2024-03-16T08:53:03.433 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-16T08:53:03.433 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/norstats}} --name pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602314 --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602314/orig.config.yaml 2024-03-16T08:53:03.486 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:53:03.590 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602314 branch: wip-batrick-testing-20240314.183243 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/norstats}} email: pdonnell@redhat.com first_in_suite: false job_id: '7602314' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240314.183243 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client dirsize rbytes: false client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - slow metadata IO - SLOW_OPS - slow request sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync rbytes: false syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240314.183243 sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4770 sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af sleep_before_teardown: 0 subset: 498/512 suite: fs suite_branch: wip-batrick-testing-20240314.183243 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKMcQjK1rvItRmL4/d6pqQrhpzHSWR91fnX6TdnTY7XC2SubTf7JMs3KMhAohQyTwOx/gFFsyX2+r7kJkLGBd0Y= smithi159.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHk06dUmFhNWeMG+1AKfUShHlDg3tc4cOLl+wvCchzg4pNCgiMKmcq/SqbIonqu15hbqyZd+I95iTnmJuuafkfA= smithi194.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIUfg8irZqXC6jjvMXnUMcmg4amOQLA+dO82iErVMDRIdet6YICjTwsDO97TYxaP7VihKgqFGgElf/IZHWjhPhU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/norstats teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-15_13:00:41 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-16T08:53:03.590 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa; will attempt to use it 2024-03-16T08:53:03.591 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks 2024-03-16T08:53:03.591 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-16T08:53:03.592 INFO:teuthology.task.internal:Checking packages... 2024-03-16T08:53:03.616 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af' 2024-03-16T08:53:03.616 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T08:53:03.616 INFO:teuthology.packaging:ref: None 2024-03-16T08:53:03.616 INFO:teuthology.packaging:tag: None 2024-03-16T08:53:03.616 INFO:teuthology.packaging:branch: wip-batrick-testing-20240314.183243 2024-03-16T08:53:03.616 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T08:53:03.616 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240314.183243 2024-03-16T08:53:03.835 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2162.g3d78f5ba 2024-03-16T08:53:03.836 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-16T08:53:03.842 INFO:teuthology.task.internal:no buildpackages task found 2024-03-16T08:53:03.842 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-16T08:53:03.869 INFO:teuthology.task.internal:Saving configuration 2024-03-16T08:53:03.890 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-16T08:53:03.895 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T08:53:03.922 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602314', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 08:47:57.911507', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKMcQjK1rvItRmL4/d6pqQrhpzHSWR91fnX6TdnTY7XC2SubTf7JMs3KMhAohQyTwOx/gFFsyX2+r7kJkLGBd0Y='} 2024-03-16T08:53:03.947 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602314', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 08:47:57.908446', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHk06dUmFhNWeMG+1AKfUShHlDg3tc4cOLl+wvCchzg4pNCgiMKmcq/SqbIonqu15hbqyZd+I95iTnmJuuafkfA='} 2024-03-16T08:53:03.972 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi194.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602314', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 08:47:57.910261', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIUfg8irZqXC6jjvMXnUMcmg4amOQLA+dO82iErVMDRIdet6YICjTwsDO97TYxaP7VihKgqFGgElf/IZHWjhPhU='} 2024-03-16T08:53:03.972 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-16T08:53:03.978 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-16T08:53:03.978 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-16T08:53:03.978 INFO:teuthology.task.internal:roles: ubuntu@smithi194.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-16T08:53:03.978 INFO:teuthology.run_tasks:Running task console_log... 2024-03-16T08:53:04.094 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4139047160>, signals=[15]) 2024-03-16T08:53:04.094 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-16T08:53:04.100 INFO:teuthology.task.internal:Opening connections... 2024-03-16T08:53:04.100 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2024-03-16T08:53:04.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T08:53:04.190 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi159.front.sepia.ceph.com 2024-03-16T08:53:04.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T08:53:04.277 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi194.front.sepia.ceph.com 2024-03-16T08:53:04.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T08:53:04.366 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-16T08:53:04.372 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-03-16T08:53:04.389 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-03-16T08:53:04.390 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-03-16T08:53:04.446 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-03-16T08:53:04.447 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-03-16T08:53:04.447 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-03-16T08:53:04.447 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-03-16T08:53:04.447 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-03-16T08:53:04.447 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-03-16T08:53:04.447 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T08:53:04.447 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-03-16T08:53:04.447 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-03-16T08:53:04.447 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T08:53:04.447 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-03-16T08:53:04.447 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T08:53:04.447 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T08:53:04.447 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T08:53:04.448 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-03-16T08:53:04.473 DEBUG:teuthology.orchestra.run.smithi159:> uname -m 2024-03-16T08:53:04.491 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2024-03-16T08:53:04.491 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2024-03-16T08:53:04.548 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2024-03-16T08:53:04.548 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9" 2024-03-16T08:53:04.548 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2024-03-16T08:53:04.548 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2024-03-16T08:53:04.548 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9" 2024-03-16T08:53:04.548 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9" 2024-03-16T08:53:04.548 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T08:53:04.548 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2024-03-16T08:53:04.548 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon" 2024-03-16T08:53:04.549 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T08:53:04.549 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2024-03-16T08:53:04.549 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T08:53:04.549 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T08:53:04.549 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T08:53:04.549 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2024-03-16T08:53:04.572 DEBUG:teuthology.orchestra.run.smithi194:> uname -m 2024-03-16T08:53:04.589 INFO:teuthology.orchestra.run.smithi194.stdout:x86_64 2024-03-16T08:53:04.590 DEBUG:teuthology.orchestra.run.smithi194:> cat /etc/os-release 2024-03-16T08:53:04.646 INFO:teuthology.orchestra.run.smithi194.stdout:NAME="CentOS Stream" 2024-03-16T08:53:04.646 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION="9" 2024-03-16T08:53:04.646 INFO:teuthology.orchestra.run.smithi194.stdout:ID="centos" 2024-03-16T08:53:04.646 INFO:teuthology.orchestra.run.smithi194.stdout:ID_LIKE="rhel fedora" 2024-03-16T08:53:04.646 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION_ID="9" 2024-03-16T08:53:04.646 INFO:teuthology.orchestra.run.smithi194.stdout:PLATFORM_ID="platform:el9" 2024-03-16T08:53:04.647 INFO:teuthology.orchestra.run.smithi194.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T08:53:04.647 INFO:teuthology.orchestra.run.smithi194.stdout:ANSI_COLOR="0;31" 2024-03-16T08:53:04.647 INFO:teuthology.orchestra.run.smithi194.stdout:LOGO="fedora-logo-icon" 2024-03-16T08:53:04.647 INFO:teuthology.orchestra.run.smithi194.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T08:53:04.647 INFO:teuthology.orchestra.run.smithi194.stdout:HOME_URL="https://centos.org/" 2024-03-16T08:53:04.647 INFO:teuthology.orchestra.run.smithi194.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T08:53:04.647 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T08:53:04.647 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T08:53:04.647 INFO:teuthology.lock.ops:Updating smithi194.front.sepia.ceph.com on lock server 2024-03-16T08:53:04.673 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-16T08:53:04.682 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-16T08:53:04.702 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-16T08:53:04.703 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2024-03-16T08:53:04.705 DEBUG:teuthology.orchestra.run.smithi159:> test '!' -e /home/ubuntu/cephtest 2024-03-16T08:53:04.708 DEBUG:teuthology.orchestra.run.smithi194:> test '!' -e /home/ubuntu/cephtest 2024-03-16T08:53:04.723 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-16T08:53:04.728 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-16T08:53:04.728 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2024-03-16T08:53:04.762 DEBUG:teuthology.orchestra.run.smithi159:> test -z $(ls -A /var/lib/ceph) 2024-03-16T08:53:04.764 DEBUG:teuthology.orchestra.run.smithi194:> test -z $(ls -A /var/lib/ceph) 2024-03-16T08:53:04.790 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-16T08:53:04.870 INFO:teuthology.run_tasks:Running task kernel... 2024-03-16T08:53:04.888 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-16T08:53:04.888 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-16T08:53:04.888 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-16T08:53:04.888 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-16T08:53:04.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T08:53:04.906 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-03-16T08:53:04.963 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-361.el9.x86_64 2024-03-16T08:53:04.963 INFO:teuthology.task.kernel:Running kernel on smithi112: 5.14.0-361.el9.x86_64 2024-03-16T08:53:04.963 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum install -y kernel 2024-03-16T08:53:07.094 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-16T08:53:08.486 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS-9 - BaseOS 6.7 MB/s | 2.0 MB 00:00 2024-03-16T08:53:22.625 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 595 kB/s | 8.0 MB 00:13 2024-03-16T08:53:24.859 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-03-16T08:53:28.944 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.1 MB 00:00 2024-03-16T08:53:30.444 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 15 kB 00:00 2024-03-16T08:53:36.529 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 3.5 MB/s | 21 MB 00:05 2024-03-16T08:53:41.579 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-16T08:53:43.777 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-16T08:53:43.777 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-16T08:53:43.875 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-03-16T08:53:43.887 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-03-16T08:53:43.887 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2024-03-16T08:53:43.887 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-03-16T08:53:43.887 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-03-16T08:53:43.887 INFO:teuthology.orchestra.run.smithi112.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-16T08:53:43.887 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-03-16T08:53:43.887 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-16T08:53:43.887 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-16T08:53:43.887 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-16T08:53:43.888 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T08:53:43.888 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-03-16T08:53:43.888 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-03-16T08:53:43.888 INFO:teuthology.orchestra.run.smithi112.stdout:Install 4 Packages 2024-03-16T08:53:43.888 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T08:53:43.888 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 100 M 2024-03-16T08:53:43.888 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 123 M 2024-03-16T08:53:43.888 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-03-16T08:53:46.180 INFO:teuthology.orchestra.run.smithi112.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 2.7 MB/s | 6.0 MB 00:02 2024-03-16T08:54:01.054 INFO:teuthology.orchestra.run.smithi112.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 1.2 MB/s | 21 MB 00:17 2024-03-16T08:54:25.293 INFO:teuthology.orchestra.run.smithi112.stdout:(3/4): kernel-modules-core-5.14.0-427.el9.x86_6 875 kB/s | 33 MB 00:39 2024-03-16T08:54:25.452 INFO:teuthology.orchestra.run.smithi112.stdout:(4/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 970 kB/s | 39 MB 00:41 2024-03-16T08:54:25.452 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-03-16T08:54:25.452 INFO:teuthology.orchestra.run.smithi112.stdout:Total 2.4 MB/s | 100 MB 00:41 2024-03-16T08:54:26.323 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-03-16T08:54:26.688 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-03-16T08:54:26.688 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-03-16T08:54:28.033 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-03-16T08:54:28.033 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-03-16T08:54:30.970 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-03-16T08:54:31.605 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-16T08:54:31.745 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-16T08:54:34.512 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-16T08:54:34.696 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-16T08:54:39.407 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-16T08:54:39.591 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-16T08:54:44.345 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-16T08:55:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-16T08:55:13.425 INFO:teuthology.orchestra.run.smithi112.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-16T08:55:13.425 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T08:55:13.509 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-16T08:55:14.027 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-16T08:55:14.027 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-16T08:55:14.027 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-16T08:55:14.027 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-16T08:55:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-16T08:55:14.448 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T08:55:14.449 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-03-16T08:55:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-16T08:55:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-16T08:55:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-16T08:55:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-16T08:55:14.449 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T08:55:14.449 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-03-16T08:55:14.977 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2024-03-16T08:55:15.132 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-16T08:55:15.132 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-16T08:55:15.132 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-16T08:55:15.133 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-16T08:55:15.133 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-16T08:55:15.133 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-16T08:55:15.133 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi112.front.sepia.ceph.com, path=None, version=distro) 2024-03-16T08:55:15.133 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2024-03-16T08:55:15.252 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-16T08:55:15.290 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-16T08:55:15.290 DEBUG:teuthology.orchestra.run.smithi112:> sudo rpm -qi grub2-tools 2024-03-16T08:55:15.341 INFO:teuthology.orchestra.run.smithi112.stdout:Name : grub2-tools 2024-03-16T08:55:15.341 INFO:teuthology.orchestra.run.smithi112.stdout:Epoch : 1 2024-03-16T08:55:15.341 INFO:teuthology.orchestra.run.smithi112.stdout:Version : 2.06 2024-03-16T08:55:15.341 INFO:teuthology.orchestra.run.smithi112.stdout:Release : 68.el9 2024-03-16T08:55:15.341 INFO:teuthology.orchestra.run.smithi112.stdout:Architecture: x86_64 2024-03-16T08:55:15.341 INFO:teuthology.orchestra.run.smithi112.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-16T08:55:15.341 INFO:teuthology.orchestra.run.smithi112.stdout:Group : Unspecified 2024-03-16T08:55:15.341 INFO:teuthology.orchestra.run.smithi112.stdout:Size : 8264254 2024-03-16T08:55:15.341 INFO:teuthology.orchestra.run.smithi112.stdout:License : GPLv3+ 2024-03-16T08:55:15.341 INFO:teuthology.orchestra.run.smithi112.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-16T08:55:15.341 INFO:teuthology.orchestra.run.smithi112.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-16T08:55:15.341 INFO:teuthology.orchestra.run.smithi112.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-16T08:55:15.341 INFO:teuthology.orchestra.run.smithi112.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-16T08:55:15.342 INFO:teuthology.orchestra.run.smithi112.stdout:Packager : builder@centos.org 2024-03-16T08:55:15.342 INFO:teuthology.orchestra.run.smithi112.stdout:Vendor : CentOS 2024-03-16T08:55:15.342 INFO:teuthology.orchestra.run.smithi112.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-16T08:55:15.342 INFO:teuthology.orchestra.run.smithi112.stdout:Summary : Support tools for GRUB. 2024-03-16T08:55:15.342 INFO:teuthology.orchestra.run.smithi112.stdout:Description : 2024-03-16T08:55:15.342 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T08:55:15.342 INFO:teuthology.orchestra.run.smithi112.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-16T08:55:15.342 INFO:teuthology.orchestra.run.smithi112.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-16T08:55:15.342 INFO:teuthology.orchestra.run.smithi112.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-16T08:55:15.342 INFO:teuthology.orchestra.run.smithi112.stdout:hardware devices. 2024-03-16T08:55:15.342 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T08:55:15.342 INFO:teuthology.orchestra.run.smithi112.stdout:This subpackage provides tools for support of all platforms. 2024-03-16T08:55:15.343 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-16T08:55:15.343 INFO:teuthology.task.kernel:Updating grub on smithi112 to boot 5.14.0-427.el9.x86_64 2024-03-16T08:55:15.344 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-16T08:55:15.950 INFO:teuthology.orchestra.run.smithi112.stderr:Generating grub configuration file ... 2024-03-16T08:55:17.384 INFO:teuthology.orchestra.run.smithi112.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-16T08:55:17.417 INFO:teuthology.orchestra.run.smithi112.stderr:done 2024-03-16T08:55:17.419 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2024-03-16T08:55:17.474 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.ap1clwQgQc 2024-03-16T08:55:17.474 DEBUG:teuthology.orchestra.run.smithi112:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ap1clwQgQc 2024-03-16T08:55:17.544 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.ap1clwQgQc 2024-03-16T08:55:17.674 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.ap1clwQgQc is 6KB 2024-03-16T08:55:17.722 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.ap1clwQgQc 2024-03-16T08:55:17.737 DEBUG:teuthology.orchestra.run.smithi112:> sudo /bin/ls /boot/loader/entries || true 2024-03-16T08:55:17.801 INFO:teuthology.orchestra.run.smithi112.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-16T08:55:17.801 INFO:teuthology.orchestra.run.smithi112.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-16T08:55:17.802 INFO:teuthology.orchestra.run.smithi112.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-16T08:55:17.802 INFO:teuthology.orchestra.run.smithi112.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-16T08:55:17.803 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-16T08:55:17.939 DEBUG:teuthology.orchestra.run.smithi112:> sudo shutdown -r now 2024-03-16T08:55:47.942 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-16T08:55:47.943 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-03-16T08:55:47.945 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T08:56:47.947 DEBUG:teuthology.orchestra.remote:timed out 2024-03-16T08:56:56.948 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-03-16T08:56:56.949 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T08:57:00.312 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-03-16T08:57:00.704 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-03-16T08:57:00.705 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-427.el9.x86_64"... 2024-03-16T08:57:00.705 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-03-16T08:57:00.719 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-427.el9.x86_64 2024-03-16T08:57:00.719 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-16T08:57:00.719 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-16T08:57:00.719 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-16T08:57:01.720 DEBUG:teuthology.parallel:result is None 2024-03-16T08:57:01.720 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-16T08:57:01.729 INFO:teuthology.task.internal:Creating test directory... 2024-03-16T08:57:01.730 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-16T08:57:01.732 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-16T08:57:01.735 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-16T08:57:01.753 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-16T08:57:01.765 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-16T08:57:01.790 INFO:teuthology.task.internal:Creating archive directory... 2024-03-16T08:57:01.790 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-16T08:57:01.792 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-16T08:57:01.794 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-16T08:57:01.910 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-16T08:57:01.918 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-16T08:57:01.918 DEBUG:teuthology.orchestra.run.smithi159:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-16T08:57:01.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T08:57:01.935 DEBUG:teuthology.orchestra.run.smithi194:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-16T08:57:01.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T08:57:01.952 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-16T08:57:01.954 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-16T08:57:01.978 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-16T08:57:02.011 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T08:57:02.016 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T08:57:02.032 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T08:57:02.042 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T08:57:02.065 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T08:57:02.067 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T08:57:02.069 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-16T08:57:02.078 INFO:teuthology.task.internal:Configuring sudo... 2024-03-16T08:57:02.078 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-16T08:57:02.080 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-16T08:57:02.086 DEBUG:teuthology.orchestra.run.smithi194:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-16T08:57:02.139 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-16T08:57:02.152 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-16T08:57:02.152 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-16T08:57:02.154 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-16T08:57:02.157 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-16T08:57:02.195 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T08:57:02.251 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T08:57:02.308 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T08:57:02.390 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T08:57:02.433 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T08:57:02.433 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-16T08:57:02.528 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T08:57:02.573 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T08:57:02.628 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T08:57:02.711 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T08:57:02.756 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-16T08:57:02.756 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-16T08:57:02.824 DEBUG:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T08:57:02.870 DEBUG:teuthology.orchestra.run.smithi194:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T08:57:02.927 DEBUG:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T08:57:03.011 DEBUG:teuthology.orchestra.run.smithi194:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T08:57:03.055 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T08:57:03.055 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-16T08:57:03.123 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2024-03-16T08:57:03.126 DEBUG:teuthology.orchestra.run.smithi159:> sudo service rsyslog restart 2024-03-16T08:57:03.128 DEBUG:teuthology.orchestra.run.smithi194:> sudo service rsyslog restart 2024-03-16T08:57:03.164 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T08:57:03.164 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T08:57:03.207 INFO:teuthology.orchestra.run.smithi194.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T08:57:03.676 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-16T08:57:03.688 INFO:teuthology.task.internal:Starting timer... 2024-03-16T08:57:03.688 INFO:teuthology.run_tasks:Running task pcp... 2024-03-16T08:57:03.713 INFO:teuthology.run_tasks:Running task selinux... 2024-03-16T08:57:03.737 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-16T08:57:03.737 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2024-03-16T08:57:03.799 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2024-03-16T08:57:03.800 DEBUG:teuthology.orchestra.run.smithi159:> sudo service auditd rotate 2024-03-16T08:57:03.870 INFO:teuthology.orchestra.run.smithi159.stdout:Rotating logs: 2024-03-16T08:57:03.872 DEBUG:teuthology.orchestra.run.smithi194:> sudo service auditd rotate 2024-03-16T08:57:03.927 INFO:teuthology.orchestra.run.smithi194.stdout:Rotating logs: 2024-03-16T08:57:03.929 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-16T08:57:03.929 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2024-03-16T08:57:03.952 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2024-03-16T08:57:03.952 DEBUG:teuthology.orchestra.run.smithi159:> /usr/sbin/getenforce 2024-03-16T08:57:03.974 INFO:teuthology.orchestra.run.smithi159.stdout:Permissive 2024-03-16T08:57:03.975 DEBUG:teuthology.orchestra.run.smithi194:> /usr/sbin/getenforce 2024-03-16T08:57:03.996 INFO:teuthology.orchestra.run.smithi194.stdout:Permissive 2024-03-16T08:57:03.996 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi112.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi159.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi194.front.sepia.ceph.com': 'permissive'} 2024-03-16T08:57:03.996 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T08:57:04.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T08:57:04.024 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T08:57:04.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T08:57:04.053 DEBUG:teuthology.orchestra.run.smithi194:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T08:57:04.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T08:57:04.080 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-16T08:57:04.080 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2024-03-16T08:57:04.247 DEBUG:teuthology.orchestra.run.smithi159:> sudo /usr/sbin/setenforce permissive 2024-03-16T08:57:04.284 DEBUG:teuthology.orchestra.run.smithi194:> sudo /usr/sbin/setenforce permissive 2024-03-16T08:57:04.338 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-16T08:57:04.347 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-16T08:57:04.353 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-16T08:57:04.485 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-16T08:57:04.500 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-16T08:57:04.502 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi112.front.sepia.ceph.com,smithi159.front.sepia.ceph.com,smithi194.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-16T09:01:43.609 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi112.front.sepia.ceph.com'), Remote(name='ubuntu@smithi159.front.sepia.ceph.com'), Remote(name='ubuntu@smithi194.front.sepia.ceph.com')] 2024-03-16T09:01:43.610 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-03-16T09:01:43.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:01:43.704 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-03-16T09:01:43.784 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-03-16T09:01:43.784 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-03-16T09:01:43.785 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:01:43.874 DEBUG:teuthology.orchestra.run.smithi159:> true 2024-03-16T09:01:43.941 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-03-16T09:01:43.941 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-03-16T09:01:43.942 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:01:44.031 DEBUG:teuthology.orchestra.run.smithi194:> true 2024-03-16T09:01:44.097 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-03-16T09:01:44.098 INFO:teuthology.run_tasks:Running task clock... 2024-03-16T09:01:44.135 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-16T09:01:44.135 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-16T09:01:44.136 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T09:01:44.138 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-16T09:01:44.138 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T09:01:44.140 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-16T09:01:44.141 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T09:01:44.169 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-16T09:01:44.171 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-16T09:01:44.172 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-16T09:01:44.185 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-16T09:01:44.188 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-16T09:01:44.190 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-16T09:01:44.214 INFO:teuthology.orchestra.run.smithi159.stderr:sudo: ntpd: command not found 2024-03-16T09:01:44.218 INFO:teuthology.orchestra.run.smithi194.stderr:sudo: ntpd: command not found 2024-03-16T09:01:44.221 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2024-03-16T09:01:44.228 INFO:teuthology.orchestra.run.smithi159.stdout:506 Cannot talk to daemon 2024-03-16T09:01:44.231 INFO:teuthology.orchestra.run.smithi194.stdout:506 Cannot talk to daemon 2024-03-16T09:01:44.235 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2024-03-16T09:01:44.245 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-16T09:01:44.248 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-16T09:01:44.253 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-16T09:01:44.261 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-16T09:01:44.264 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-16T09:01:44.270 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-16T09:01:44.306 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-03-16T09:01:44.308 INFO:teuthology.orchestra.run.smithi194.stderr:bash: line 1: ntpq: command not found 2024-03-16T09:01:44.310 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T09:01:44.310 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-03-16T09:01:44.311 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:01:44.311 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:01:44.311 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:01:44.311 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:01:44.313 INFO:teuthology.orchestra.run.smithi194.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T09:01:44.313 INFO:teuthology.orchestra.run.smithi194.stdout:=============================================================================== 2024-03-16T09:01:44.313 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:01:44.313 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:01:44.313 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:01:44.314 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:01:44.321 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-03-16T09:01:44.324 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T09:01:44.324 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-03-16T09:01:44.325 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:01:44.325 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:01:44.325 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:01:44.325 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:01:44.326 INFO:teuthology.run_tasks:Running task install... 2024-03-16T09:01:44.338 DEBUG:teuthology.task.install:project ceph 2024-03-16T09:01:44.339 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'}} 2024-03-16T09:01:44.339 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'} 2024-03-16T09:01:44.339 INFO:teuthology.task.install:Using flavor: default 2024-03-16T09:01:44.347 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-16T09:01:44.347 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-16T09:01:44.347 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'tag': None, 'wait_for_package': False} 2024-03-16T09:01:44.348 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:01:44.350 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'tag': None, 'wait_for_package': False} 2024-03-16T09:01:44.350 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:01:44.351 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'tag': None, 'wait_for_package': False} 2024-03-16T09:01:44.351 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:01:44.585 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/ 2024-03-16T09:01:44.586 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2162.g3d78f5ba 2024-03-16T09:01:44.603 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/ 2024-03-16T09:01:44.604 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2162.g3d78f5ba 2024-03-16T09:01:44.618 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/ 2024-03-16T09:01:44.618 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2162.g3d78f5ba 2024-03-16T09:01:44.695 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-16T09:01:44.698 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T09:01:44.698 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-16T09:01:44.710 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-16T09:01:44.710 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-16T09:01:44.710 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-16T09:01:44.726 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-16T09:01:44.726 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-16T09:01:44.738 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-16T09:01:44.738 DEBUG:teuthology.orchestra.run.smithi159:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-16T09:01:44.793 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-16T09:01:44.805 DEBUG:teuthology.orchestra.run.smithi159:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-16T09:01:44.845 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-16T09:01:44.846 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:01:44.846 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-16T09:01:44.872 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-16T09:01:44.875 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-16T09:01:44.875 DEBUG:teuthology.orchestra.run.smithi194:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-16T09:01:44.885 DEBUG:teuthology.orchestra.run.smithi159:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-16T09:01:44.938 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2024-03-16T09:01:44.940 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2024-03-16T09:01:44.945 DEBUG:teuthology.orchestra.run.smithi194:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-16T09:01:44.951 INFO:teuthology.orchestra.run.smithi159.stdout:check_obsoletes = 1 2024-03-16T09:01:44.953 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum clean all 2024-03-16T09:01:45.026 DEBUG:teuthology.orchestra.run.smithi194:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-16T09:01:45.093 INFO:teuthology.orchestra.run.smithi194.stdout:check_obsoletes = 1 2024-03-16T09:01:45.094 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum clean all 2024-03-16T09:01:45.179 INFO:teuthology.orchestra.run.smithi112.stdout:74 files removed 2024-03-16T09:01:45.189 INFO:teuthology.orchestra.run.smithi159.stdout:74 files removed 2024-03-16T09:01:45.209 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-16T09:01:45.217 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-16T09:01:45.329 INFO:teuthology.orchestra.run.smithi194.stdout:74 files removed 2024-03-16T09:01:45.356 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-16T09:01:45.868 INFO:teuthology.orchestra.run.smithi159.stdout:ceph packages for x86_64 236 kB/s | 83 kB 00:00 2024-03-16T09:01:45.908 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 220 kB/s | 83 kB 00:00 2024-03-16T09:01:46.055 INFO:teuthology.orchestra.run.smithi194.stdout:ceph packages for x86_64 204 kB/s | 83 kB 00:00 2024-03-16T09:01:46.294 INFO:teuthology.orchestra.run.smithi159.stdout:ceph noarch packages 26 kB/s | 10 kB 00:00 2024-03-16T09:01:46.334 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-16T09:01:46.464 INFO:teuthology.orchestra.run.smithi194.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-03-16T09:01:46.711 INFO:teuthology.orchestra.run.smithi159.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-03-16T09:01:46.768 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-03-16T09:01:46.798 INFO:teuthology.orchestra.run.smithi194.stdout:ceph source packages 6.2 kB/s | 1.9 kB 00:00 2024-03-16T09:01:47.139 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-03-16T09:01:47.229 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-16T09:01:47.250 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-16T09:01:48.556 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-9 - BaseOS 6.4 MB/s | 2.0 MB 00:00 2024-03-16T09:01:48.634 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS-9 - BaseOS 7.6 MB/s | 2.0 MB 00:00 2024-03-16T09:01:48.639 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.0 MB 00:00 2024-03-16T09:01:50.706 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.0 MB 00:01 2024-03-16T09:01:58.097 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 901 kB/s | 8.0 MB 00:09 2024-03-16T09:01:58.474 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - BaseOS 864 kB/s | 8.0 MB 00:09 2024-03-16T09:02:00.539 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-03-16T09:02:04.440 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.1 MB 00:00 2024-03-16T09:02:06.057 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 15 kB 00:00 2024-03-16T09:02:08.428 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 21 MB 00:02 2024-03-16T09:02:13.342 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-03-16T09:02:15.871 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T09:02:15.872 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T09:02:15.884 INFO:teuthology.orchestra.run.smithi112.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-16T09:02:15.885 INFO:teuthology.orchestra.run.smithi112.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-16T09:02:15.886 INFO:teuthology.orchestra.run.smithi112.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-16T09:02:15.887 INFO:teuthology.orchestra.run.smithi112.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-16T09:02:15.889 INFO:teuthology.orchestra.run.smithi112.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-16T09:02:15.891 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-16T09:02:15.892 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-16T09:02:15.972 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-03-16T09:02:16.006 INFO:teuthology.orchestra.run.smithi112.stdout:============================================================================================ 2024-03-16T09:02:16.006 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-03-16T09:02:16.006 INFO:teuthology.orchestra.run.smithi112.stdout:============================================================================================ 2024-03-16T09:02:16.006 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-03-16T09:02:16.006 INFO:teuthology.orchestra.run.smithi112.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-16T09:02:16.006 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 6.6 k 2024-03-16T09:02:16.007 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.4 M 2024-03-16T09:02:16.007 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 890 k 2024-03-16T09:02:16.007 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 145 k 2024-03-16T09:02:16.007 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 1.5 M 2024-03-16T09:02:16.007 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 139 k 2024-03-16T09:02:16.007 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 1.8 M 2024-03-16T09:02:16.007 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 7.4 M 2024-03-16T09:02:16.007 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 49 k 2024-03-16T09:02:16.007 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 9.8 M 2024-03-16T09:02:16.007 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 48 M 2024-03-16T09:02:16.007 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 264 k 2024-03-16T09:02:16.007 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 748 k 2024-03-16T09:02:16.007 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 221 k 2024-03-16T09:02:16.008 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 27 k 2024-03-16T09:02:16.008 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 18 k 2024-03-16T09:02:16.008 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-16T09:02:16.008 INFO:teuthology.orchestra.run.smithi112.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-16T09:02:16.008 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-16T09:02:16.008 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 32 k 2024-03-16T09:02:16.008 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 747 k 2024-03-16T09:02:16.008 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 127 k 2024-03-16T09:02:16.008 INFO:teuthology.orchestra.run.smithi112.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-16T09:02:16.008 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-16T09:02:16.008 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-16T09:02:16.008 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-16T09:02:16.008 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-16T09:02:16.009 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-16T09:02:16.009 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 162 k 2024-03-16T09:02:16.009 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 320 k 2024-03-16T09:02:16.009 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 298 k 2024-03-16T09:02:16.009 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 99 k 2024-03-16T09:02:16.009 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 85 k 2024-03-16T09:02:16.009 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-16T09:02:16.009 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 170 k 2024-03-16T09:02:16.009 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-16T09:02:16.009 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2024-03-16T09:02:16.009 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-16T09:02:16.009 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-16T09:02:16.009 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-16T09:02:16.010 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-16T09:02:16.010 INFO:teuthology.orchestra.run.smithi112.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-16T09:02:16.010 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-16T09:02:16.010 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-16T09:02:16.010 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-16T09:02:16.010 INFO:teuthology.orchestra.run.smithi112.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-16T09:02:16.010 INFO:teuthology.orchestra.run.smithi112.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-16T09:02:16.010 INFO:teuthology.orchestra.run.smithi112.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-16T09:02:16.010 INFO:teuthology.orchestra.run.smithi112.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-16T09:02:16.010 INFO:teuthology.orchestra.run.smithi112.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-16T09:02:16.010 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.4 M 2024-03-16T09:02:16.010 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-16T09:02:16.010 INFO:teuthology.orchestra.run.smithi112.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-16T09:02:16.011 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-16T09:02:16.011 INFO:teuthology.orchestra.run.smithi112.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-16T09:02:16.011 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-16T09:02:16.011 INFO:teuthology.orchestra.run.smithi112.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-16T09:02:16.011 INFO:teuthology.orchestra.run.smithi112.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-16T09:02:16.011 INFO:teuthology.orchestra.run.smithi112.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-16T09:02:16.011 INFO:teuthology.orchestra.run.smithi112.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-16T09:02:16.011 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-16T09:02:16.011 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-03-16T09:02:16.011 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-16T09:02:16.011 INFO:teuthology.orchestra.run.smithi112.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-16T09:02:16.011 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-16T09:02:16.011 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 21 M 2024-03-16T09:02:16.011 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 34 k 2024-03-16T09:02:16.011 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 2.4 M 2024-03-16T09:02:16.012 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 248 k 2024-03-16T09:02:16.012 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 4.7 M 2024-03-16T09:02:16.012 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 17 M 2024-03-16T09:02:16.012 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 16 k 2024-03-16T09:02:16.012 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 25 k 2024-03-16T09:02:16.012 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-16T09:02:16.012 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-16T09:02:16.012 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-16T09:02:16.012 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-16T09:02:16.012 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-16T09:02:16.012 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-16T09:02:16.012 INFO:teuthology.orchestra.run.smithi112.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-16T09:02:16.012 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-16T09:02:16.012 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-16T09:02:16.013 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 163 k 2024-03-16T09:02:16.013 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-16T09:02:16.013 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-16T09:02:16.013 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-16T09:02:16.013 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-16T09:02:16.013 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-16T09:02:16.013 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 501 k 2024-03-16T09:02:16.013 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-16T09:02:16.013 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.1 M 2024-03-16T09:02:16.013 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-16T09:02:16.013 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-16T09:02:16.013 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-16T09:02:16.014 INFO:teuthology.orchestra.run.smithi112.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-16T09:02:16.014 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-16T09:02:16.014 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-16T09:02:16.014 INFO:teuthology.orchestra.run.smithi112.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-16T09:02:16.014 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-16T09:02:16.014 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-16T09:02:16.014 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-16T09:02:16.014 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-16T09:02:16.014 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-16T09:02:16.014 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-16T09:02:16.014 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-16T09:02:16.014 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-16T09:02:16.014 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-16T09:02:16.015 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-16T09:02:16.015 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-16T09:02:16.015 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 45 k 2024-03-16T09:02:16.015 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 131 k 2024-03-16T09:02:16.015 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-16T09:02:16.015 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-16T09:02:16.015 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-16T09:02:16.015 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-16T09:02:16.015 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-16T09:02:16.015 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-16T09:02:16.015 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-16T09:02:16.015 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-16T09:02:16.015 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-16T09:02:16.016 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-16T09:02:16.016 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-16T09:02:16.016 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-16T09:02:16.016 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-16T09:02:16.016 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-16T09:02:16.016 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-16T09:02:16.016 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-16T09:02:16.016 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-16T09:02:16.016 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-16T09:02:16.016 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-16T09:02:16.016 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-16T09:02:16.016 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-16T09:02:16.016 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-16T09:02:16.016 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-16T09:02:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-16T09:02:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-16T09:02:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-16T09:02:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-16T09:02:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-16T09:02:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-16T09:02:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-16T09:02:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-16T09:02:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-16T09:02:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-16T09:02:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-16T09:02:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-16T09:02:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-16T09:02:16.017 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-16T09:02:16.018 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-16T09:02:16.018 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-16T09:02:16.018 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-16T09:02:16.018 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-16T09:02:16.018 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-16T09:02:16.018 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-16T09:02:16.018 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-16T09:02:16.018 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-16T09:02:16.018 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-16T09:02:16.018 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-16T09:02:16.018 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-16T09:02:16.018 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-16T09:02:16.018 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-16T09:02:16.018 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-16T09:02:16.019 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-16T09:02:16.019 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-16T09:02:16.019 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-16T09:02:16.019 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-16T09:02:16.019 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-16T09:02:16.019 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-16T09:02:16.019 INFO:teuthology.orchestra.run.smithi112.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-16T09:02:16.019 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-16T09:02:16.019 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-16T09:02:16.019 INFO:teuthology.orchestra.run.smithi112.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-16T09:02:16.019 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-16T09:02:16.019 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-16T09:02:16.019 INFO:teuthology.orchestra.run.smithi112.stdout:Installing weak dependencies: 2024-03-16T09:02:16.020 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-16T09:02:16.020 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:02:16.020 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-03-16T09:02:16.020 INFO:teuthology.orchestra.run.smithi112.stdout:============================================================================================ 2024-03-16T09:02:16.020 INFO:teuthology.orchestra.run.smithi112.stdout:Install 150 Packages 2024-03-16T09:02:16.020 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 24 Packages 2024-03-16T09:02:16.020 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:02:16.021 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 224 M 2024-03-16T09:02:16.021 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-03-16T09:02:16.919 INFO:teuthology.orchestra.run.smithi112.stdout:(1/174): ceph-19.0.0-2162.g3d78f5ba.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-03-16T09:02:17.227 INFO:teuthology.orchestra.run.smithi112.stdout:(2/174): ceph-fuse-19.0.0-2162.g3d78f5ba.el9.x8 2.8 MB/s | 890 kB 00:00 2024-03-16T09:02:17.302 INFO:teuthology.orchestra.run.smithi112.stdout:(3/174): ceph-immutable-object-cache-19.0.0-216 1.9 MB/s | 145 kB 00:00 2024-03-16T09:02:18.144 INFO:teuthology.orchestra.run.smithi112.stdout:(4/174): ceph-common-19.0.0-2162.g3d78f5ba.el9. 15 MB/s | 21 MB 00:01 2024-03-16T09:02:18.186 INFO:teuthology.orchestra.run.smithi112.stdout:(5/174): ceph-mds-19.0.0-2162.g3d78f5ba.el9.x86 2.7 MB/s | 2.4 MB 00:00 2024-03-16T09:02:18.270 INFO:teuthology.orchestra.run.smithi112.stdout:(6/174): ceph-base-19.0.0-2162.g3d78f5ba.el9.x8 3.5 MB/s | 5.4 MB 00:01 2024-03-16T09:02:18.304 INFO:teuthology.orchestra.run.smithi112.stdout:(7/174): ceph-mgr-19.0.0-2162.g3d78f5ba.el9.x86 9.4 MB/s | 1.5 MB 00:00 2024-03-16T09:02:18.987 INFO:teuthology.orchestra.run.smithi112.stdout:(8/174): ceph-radosgw-19.0.0-2162.g3d78f5ba.el9 14 MB/s | 9.8 MB 00:00 2024-03-16T09:02:19.045 INFO:teuthology.orchestra.run.smithi112.stdout:(9/174): ceph-selinux-19.0.0-2162.g3d78f5ba.el9 433 kB/s | 25 kB 00:00 2024-03-16T09:02:19.204 INFO:teuthology.orchestra.run.smithi112.stdout:(10/174): ceph-mon-19.0.0-2162.g3d78f5ba.el9.x8 4.6 MB/s | 4.7 MB 00:01 2024-03-16T09:02:19.313 INFO:teuthology.orchestra.run.smithi112.stdout:(11/174): cephfs-mirror-19.0.0-2162.g3d78f5ba.e 2.0 MB/s | 221 kB 00:00 2024-03-16T09:02:19.405 INFO:teuthology.orchestra.run.smithi112.stdout:(12/174): cephfs-shell-19.0.0-2162.g3d78f5ba.el 295 kB/s | 27 kB 00:00 2024-03-16T09:02:19.480 INFO:teuthology.orchestra.run.smithi112.stdout:(13/174): libcephfs-devel-19.0.0-2162.g3d78f5ba 422 kB/s | 32 kB 00:00 2024-03-16T09:02:19.705 INFO:teuthology.orchestra.run.smithi112.stdout:(14/174): libcephfs2-19.0.0-2162.g3d78f5ba.el9. 3.2 MB/s | 747 kB 00:00 2024-03-16T09:02:19.789 INFO:teuthology.orchestra.run.smithi112.stdout:(15/174): libcephsqlite-19.0.0-2162.g3d78f5ba.e 1.9 MB/s | 163 kB 00:00 2024-03-16T09:02:20.006 INFO:teuthology.orchestra.run.smithi112.stdout:(16/174): ceph-osd-19.0.0-2162.g3d78f5ba.el9.x8 9.7 MB/s | 17 MB 00:01 2024-03-16T09:02:20.031 INFO:teuthology.orchestra.run.smithi112.stdout:(17/174): librados-devel-19.0.0-2162.g3d78f5ba. 524 kB/s | 127 kB 00:00 2024-03-16T09:02:20.081 INFO:teuthology.orchestra.run.smithi112.stdout:(18/174): libradosstriper1-19.0.0-2162.g3d78f5b 6.5 MB/s | 501 kB 00:00 2024-03-16T09:02:20.165 INFO:teuthology.orchestra.run.smithi112.stdout:(19/174): python3-ceph-argparse-19.0.0-2162.g3d 542 kB/s | 45 kB 00:00 2024-03-16T09:02:20.248 INFO:teuthology.orchestra.run.smithi112.stdout:(20/174): python3-ceph-common-19.0.0-2162.g3d78 1.5 MB/s | 131 kB 00:00 2024-03-16T09:02:20.332 INFO:teuthology.orchestra.run.smithi112.stdout:(21/174): python3-cephfs-19.0.0-2162.g3d78f5ba. 1.9 MB/s | 162 kB 00:00 2024-03-16T09:02:20.432 INFO:teuthology.orchestra.run.smithi112.stdout:(22/174): python3-rados-19.0.0-2162.g3d78f5ba.e 3.1 MB/s | 320 kB 00:00 2024-03-16T09:02:20.524 INFO:teuthology.orchestra.run.smithi112.stdout:(23/174): python3-rbd-19.0.0-2162.g3d78f5ba.el9 3.2 MB/s | 298 kB 00:00 2024-03-16T09:02:20.564 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - AppStream 917 kB/s | 19 MB 00:21 2024-03-16T09:02:20.616 INFO:teuthology.orchestra.run.smithi112.stdout:(24/174): python3-rgw-19.0.0-2162.g3d78f5ba.el9 1.1 MB/s | 99 kB 00:00 2024-03-16T09:02:20.700 INFO:teuthology.orchestra.run.smithi112.stdout:(25/174): rbd-fuse-19.0.0-2162.g3d78f5ba.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-16T09:02:21.017 INFO:teuthology.orchestra.run.smithi112.stdout:(26/174): librgw2-19.0.0-2162.g3d78f5ba.el9.x86 5.2 MB/s | 5.1 MB 00:00 2024-03-16T09:02:21.076 INFO:teuthology.orchestra.run.smithi112.stdout:(27/174): rbd-mirror-19.0.0-2162.g3d78f5ba.el9. 8.2 MB/s | 3.1 MB 00:00 2024-03-16T09:02:21.101 INFO:teuthology.orchestra.run.smithi112.stdout:(28/174): rbd-nbd-19.0.0-2162.g3d78f5ba.el9.x86 2.0 MB/s | 170 kB 00:00 2024-03-16T09:02:21.151 INFO:teuthology.orchestra.run.smithi112.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2162.g 451 kB/s | 34 kB 00:00 2024-03-16T09:02:21.176 INFO:teuthology.orchestra.run.smithi112.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2162.g3d78f5b 1.8 MB/s | 139 kB 00:00 2024-03-16T09:02:21.394 INFO:teuthology.orchestra.run.smithi112.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2162.g3d78f 7.4 MB/s | 1.8 MB 00:00 2024-03-16T09:02:21.486 INFO:teuthology.orchestra.run.smithi112.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2162.g3d 2.7 MB/s | 248 kB 00:00 2024-03-16T09:02:21.569 INFO:teuthology.orchestra.run.smithi112.stdout:(33/174): ceph-mgr-rook-19.0.0-2162.g3d78f5ba.e 593 kB/s | 49 kB 00:00 2024-03-16T09:02:22.036 INFO:teuthology.orchestra.run.smithi112.stdout:(34/174): ceph-test-19.0.0-2162.g3d78f5ba.el9.x 16 MB/s | 48 MB 00:02 2024-03-16T09:02:22.061 INFO:teuthology.orchestra.run.smithi112.stdout:(35/174): ceph-prometheus-alerts-19.0.0-2162.g3 33 kB/s | 16 kB 00:00 2024-03-16T09:02:22.103 INFO:teuthology.orchestra.run.smithi112.stdout:(36/174): ceph-volume-19.0.0-2162.g3d78f5ba.el9 3.9 MB/s | 264 kB 00:00 2024-03-16T09:02:22.154 INFO:teuthology.orchestra.run.smithi112.stdout:(37/174): cephfs-top-19.0.0-2162.g3d78f5ba.el9. 366 kB/s | 18 kB 00:00 2024-03-16T09:02:22.187 INFO:teuthology.orchestra.run.smithi112.stdout:(38/174): cephadm-19.0.0-2162.g3d78f5ba.el9.noa 5.8 MB/s | 748 kB 00:00 2024-03-16T09:02:22.329 INFO:teuthology.orchestra.run.smithi112.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 662 kB 00:00 2024-03-16T09:02:22.363 INFO:teuthology.orchestra.run.smithi112.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 6.1 MB/s | 920 kB 00:00 2024-03-16T09:02:22.389 INFO:teuthology.orchestra.run.smithi112.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-16T09:02:22.413 INFO:teuthology.orchestra.run.smithi112.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-16T09:02:22.438 INFO:teuthology.orchestra.run.smithi112.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 638 kB/s | 32 kB 00:00 2024-03-16T09:02:22.522 INFO:teuthology.orchestra.run.smithi112.stdout:(44/174): ceph-mgr-diskprediction-local-19.0.0- 5.5 MB/s | 7.4 MB 00:01 2024-03-16T09:02:22.572 INFO:teuthology.orchestra.run.smithi112.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-03-16T09:02:22.598 INFO:teuthology.orchestra.run.smithi112.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 109 kB/s | 17 kB 00:00 2024-03-16T09:02:22.623 INFO:teuthology.orchestra.run.smithi112.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 10 kB 00:00 2024-03-16T09:02:22.648 INFO:teuthology.orchestra.run.smithi112.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-03-16T09:02:22.690 INFO:teuthology.orchestra.run.smithi112.stdout:(49/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 159 kB 00:00 2024-03-16T09:02:22.723 INFO:teuthology.orchestra.run.smithi112.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 661 kB 00:00 2024-03-16T09:02:22.749 INFO:teuthology.orchestra.run.smithi112.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 578 kB 00:00 2024-03-16T09:02:22.774 INFO:teuthology.orchestra.run.smithi112.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 46 kB 00:00 2024-03-16T09:02:22.808 INFO:teuthology.orchestra.run.smithi112.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.3 MB/s | 282 kB 00:00 2024-03-16T09:02:22.833 INFO:teuthology.orchestra.run.smithi112.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2024-03-16T09:02:22.858 INFO:teuthology.orchestra.run.smithi112.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 688 kB/s | 34 kB 00:00 2024-03-16T09:02:22.884 INFO:teuthology.orchestra.run.smithi112.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-16T09:02:22.959 INFO:teuthology.orchestra.run.smithi112.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-03-16T09:02:23.001 INFO:teuthology.orchestra.run.smithi112.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 11 MB/s | 1.5 MB 00:00 2024-03-16T09:02:23.026 INFO:teuthology.orchestra.run.smithi112.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 5.7 MB/s | 811 kB 00:00 2024-03-16T09:02:23.051 INFO:teuthology.orchestra.run.smithi112.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.5 MB/s | 134 kB 00:00 2024-03-16T09:02:23.093 INFO:teuthology.orchestra.run.smithi112.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-03-16T09:02:23.202 INFO:teuthology.orchestra.run.smithi112.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 32 MB/s | 5.7 MB 00:00 2024-03-16T09:02:23.227 INFO:teuthology.orchestra.run.smithi112.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 444 kB/s | 78 kB 00:00 2024-03-16T09:02:23.311 INFO:teuthology.orchestra.run.smithi112.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 5.8 MB 00:00 2024-03-16T09:02:23.336 INFO:teuthology.orchestra.run.smithi112.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 186 kB/s | 24 kB 00:00 2024-03-16T09:02:23.361 INFO:teuthology.orchestra.run.smithi112.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 138 kB/s | 18 kB 00:00 2024-03-16T09:02:23.378 INFO:teuthology.orchestra.run.smithi112.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-16T09:02:23.404 INFO:teuthology.orchestra.run.smithi112.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 474 kB/s | 32 kB 00:00 2024-03-16T09:02:23.487 INFO:teuthology.orchestra.run.smithi112.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-03-16T09:02:23.512 INFO:teuthology.orchestra.run.smithi112.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 217 kB 00:00 2024-03-16T09:02:23.537 INFO:teuthology.orchestra.run.smithi112.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 174 kB 00:00 2024-03-16T09:02:23.562 INFO:teuthology.orchestra.run.smithi112.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 925 kB/s | 69 kB 00:00 2024-03-16T09:02:23.588 INFO:teuthology.orchestra.run.smithi112.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 820 kB/s | 41 kB 00:00 2024-03-16T09:02:23.613 INFO:teuthology.orchestra.run.smithi112.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-16T09:02:23.638 INFO:teuthology.orchestra.run.smithi112.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-16T09:02:23.663 INFO:teuthology.orchestra.run.smithi112.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 952 kB/s | 47 kB 00:00 2024-03-16T09:02:23.689 INFO:teuthology.orchestra.run.smithi112.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-16T09:02:23.714 INFO:teuthology.orchestra.run.smithi112.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 567 kB/s | 43 kB 00:00 2024-03-16T09:02:24.106 INFO:teuthology.orchestra.run.smithi112.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-03-16T09:02:24.123 INFO:teuthology.orchestra.run.smithi112.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 95 kB/s | 39 kB 00:00 2024-03-16T09:02:24.140 INFO:teuthology.orchestra.run.smithi112.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 97 kB/s | 41 kB 00:00 2024-03-16T09:02:24.165 INFO:teuthology.orchestra.run.smithi112.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-03-16T09:02:24.190 INFO:teuthology.orchestra.run.smithi112.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-16T09:02:24.207 INFO:teuthology.orchestra.run.smithi112.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 941 kB/s | 63 kB 00:00 2024-03-16T09:02:24.224 INFO:teuthology.orchestra.run.smithi112.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 746 kB/s | 44 kB 00:00 2024-03-16T09:02:24.249 INFO:teuthology.orchestra.run.smithi112.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-16T09:02:24.283 INFO:teuthology.orchestra.run.smithi112.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-03-16T09:02:24.316 INFO:teuthology.orchestra.run.smithi112.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-16T09:02:24.342 INFO:teuthology.orchestra.run.smithi112.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-03-16T09:02:24.367 INFO:teuthology.orchestra.run.smithi112.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-03-16T09:02:24.392 INFO:teuthology.orchestra.run.smithi112.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 187 kB 00:00 2024-03-16T09:02:24.510 INFO:teuthology.orchestra.run.smithi112.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-03-16T09:02:24.535 INFO:teuthology.orchestra.run.smithi112.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-03-16T09:02:24.560 INFO:teuthology.orchestra.run.smithi112.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-16T09:02:24.594 INFO:teuthology.orchestra.run.smithi112.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.5 MB/s | 803 kB 00:00 2024-03-16T09:02:24.802 INFO:teuthology.orchestra.run.smithi112.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 80 kB/s | 21 kB 00:00 2024-03-16T09:02:24.827 INFO:teuthology.orchestra.run.smithi112.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 95 kB/s | 25 kB 00:00 2024-03-16T09:02:26.178 INFO:teuthology.orchestra.run.smithi112.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 184 kB/s | 248 kB 00:01 2024-03-16T09:02:26.436 INFO:teuthology.orchestra.run.smithi112.stdout:(99/174): python3-devel-3.9.17-2.el9.x86_64.rpm 150 kB/s | 245 kB 00:01 2024-03-16T09:02:26.653 INFO:teuthology.orchestra.run.smithi112.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 161 kB/s | 35 kB 00:00 2024-03-16T09:02:26.745 INFO:teuthology.orchestra.run.smithi112.stdout:(101/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 486 kB/s | 44 kB 00:00 2024-03-16T09:02:26.954 INFO:teuthology.orchestra.run.smithi112.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 450 kB/s | 94 kB 00:00 2024-03-16T09:02:27.012 INFO:teuthology.orchestra.run.smithi112.stdout:(103/174): qatlib-23.11.0-1.el9.x86_64.rpm 259 kB/s | 216 kB 00:00 2024-03-16T09:02:27.229 INFO:teuthology.orchestra.run.smithi112.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 81 kB/s | 22 kB 00:00 2024-03-16T09:02:27.771 INFO:teuthology.orchestra.run.smithi112.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-03-16T09:02:27.796 INFO:teuthology.orchestra.run.smithi112.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-03-16T09:02:27.872 INFO:teuthology.orchestra.run.smithi112.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-03-16T09:02:27.969 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - AppStream 533 kB/s | 19 MB 00:36 2024-03-16T09:02:28.039 INFO:teuthology.orchestra.run.smithi112.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 80 kB/s | 19 kB 00:00 2024-03-16T09:02:28.139 INFO:teuthology.orchestra.run.smithi112.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-03-16T09:02:28.272 INFO:teuthology.orchestra.run.smithi112.stdout:(110/174): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-03-16T09:02:28.298 INFO:teuthology.orchestra.run.smithi112.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 306 kB/s | 49 kB 00:00 2024-03-16T09:02:28.365 INFO:teuthology.orchestra.run.smithi112.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 733 kB/s | 67 kB 00:00 2024-03-16T09:02:28.457 INFO:teuthology.orchestra.run.smithi112.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 952 kB/s | 151 kB 00:00 2024-03-16T09:02:28.499 INFO:teuthology.orchestra.run.smithi112.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.1 MB/s | 838 kB 00:00 2024-03-16T09:02:28.591 INFO:teuthology.orchestra.run.smithi112.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 473 kB/s | 43 kB 00:00 2024-03-16T09:02:28.624 INFO:teuthology.orchestra.run.smithi112.stdout:(116/174): python3-asyncssh-2.13.2-4.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-03-16T09:02:28.683 INFO:teuthology.orchestra.run.smithi112.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 351 kB/s | 32 kB 00:00 2024-03-16T09:02:28.717 INFO:teuthology.orchestra.run.smithi112.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-03-16T09:02:28.784 INFO:teuthology.orchestra.run.smithi112.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-03-16T09:02:28.809 INFO:teuthology.orchestra.run.smithi112.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-03-16T09:02:28.884 INFO:teuthology.orchestra.run.smithi112.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-03-16T09:02:28.910 INFO:teuthology.orchestra.run.smithi112.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 352 kB/s | 35 kB 00:00 2024-03-16T09:02:28.985 INFO:teuthology.orchestra.run.smithi112.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-03-16T09:02:29.085 INFO:teuthology.orchestra.run.smithi112.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-03-16T09:02:29.177 INFO:teuthology.orchestra.run.smithi112.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2024-03-16T09:02:29.261 INFO:teuthology.orchestra.run.smithi112.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-03-16T09:02:29.345 INFO:teuthology.orchestra.run.smithi112.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 278 kB/s | 23 kB 00:00 2024-03-16T09:02:29.428 INFO:teuthology.orchestra.run.smithi112.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 233 kB/s | 19 kB 00:00 2024-03-16T09:02:29.512 INFO:teuthology.orchestra.run.smithi112.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 236 kB/s | 20 kB 00:00 2024-03-16T09:02:29.622 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - CRB 1.1 MB/s | 6.1 MB 00:05 2024-03-16T09:02:29.629 INFO:teuthology.orchestra.run.smithi112.stdout:(130/174): python3-kubernetes-26.1.0-2.el9.noar 8.8 MB/s | 1.0 MB 00:00 2024-03-16T09:02:29.737 INFO:teuthology.orchestra.run.smithi112.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.5 MB/s | 2.0 MB 00:00 2024-03-16T09:02:29.763 INFO:teuthology.orchestra.run.smithi112.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 347 kB/s | 46 kB 00:00 2024-03-16T09:02:29.830 INFO:teuthology.orchestra.run.smithi112.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 859 kB/s | 79 kB 00:00 2024-03-16T09:02:29.855 INFO:teuthology.orchestra.run.smithi112.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 628 kB/s | 58 kB 00:00 2024-03-16T09:02:29.922 INFO:teuthology.orchestra.run.smithi112.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-03-16T09:02:29.947 INFO:teuthology.orchestra.run.smithi112.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-03-16T09:02:30.014 INFO:teuthology.orchestra.run.smithi112.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 979 kB/s | 90 kB 00:00 2024-03-16T09:02:30.039 INFO:teuthology.orchestra.run.smithi112.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-03-16T09:02:30.106 INFO:teuthology.orchestra.run.smithi112.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-03-16T09:02:30.140 INFO:teuthology.orchestra.run.smithi112.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-03-16T09:02:30.198 INFO:teuthology.orchestra.run.smithi112.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 642 kB/s | 59 kB 00:00 2024-03-16T09:02:30.232 INFO:teuthology.orchestra.run.smithi112.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 389 kB/s | 36 kB 00:00 2024-03-16T09:02:30.290 INFO:teuthology.orchestra.run.smithi112.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 555 kB/s | 51 kB 00:00 2024-03-16T09:02:30.333 INFO:teuthology.orchestra.run.smithi112.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-03-16T09:02:30.383 INFO:teuthology.orchestra.run.smithi112.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 975 kB/s | 90 kB 00:00 2024-03-16T09:02:30.433 INFO:teuthology.orchestra.run.smithi112.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-03-16T09:02:30.484 INFO:teuthology.orchestra.run.smithi112.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-03-16T09:02:30.534 INFO:teuthology.orchestra.run.smithi112.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-03-16T09:02:30.901 INFO:teuthology.orchestra.run.smithi112.stdout:(149/174): librados2-19.0.0-2162.g3d78f5ba.el9. 9.3 MB/s | 3.4 MB 00:00 2024-03-16T09:02:31.043 INFO:teuthology.orchestra.run.smithi112.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 2.8 MB/s | 1.6 MB 00:00 2024-03-16T09:02:31.085 INFO:teuthology.orchestra.run.smithi112.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-16T09:02:31.148 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 15 kB 00:00 2024-03-16T09:02:31.219 INFO:teuthology.orchestra.run.smithi112.stdout:(152/174): librbd1-19.0.0-2162.g3d78f5ba.el9.x8 9.9 MB/s | 3.1 MB 00:00 2024-03-16T09:02:31.319 INFO:teuthology.orchestra.run.smithi112.stdout:(153/174): NetworkManager-1.46.0-1.el9.x86_64.r 9.8 MB/s | 2.3 MB 00:00 2024-03-16T09:02:31.391 INFO:teuthology.orchestra.run.smithi112.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 589 kB/s | 42 kB 00:00 2024-03-16T09:02:31.436 INFO:teuthology.orchestra.run.smithi112.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 5.4 MB/s | 248 kB 00:00 2024-03-16T09:02:31.478 INFO:teuthology.orchestra.run.smithi112.stdout:(156/174): NetworkManager-libnm-1.46.0-1.el9.x8 7.0 MB/s | 1.8 MB 00:00 2024-03-16T09:02:31.503 INFO:teuthology.orchestra.run.smithi112.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 1.0 MB/s | 71 kB 00:00 2024-03-16T09:02:31.528 INFO:teuthology.orchestra.run.smithi112.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 724 kB/s | 36 kB 00:00 2024-03-16T09:02:31.554 INFO:teuthology.orchestra.run.smithi112.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.7 MB/s | 192 kB 00:00 2024-03-16T09:02:31.579 INFO:teuthology.orchestra.run.smithi112.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 256 kB 00:00 2024-03-16T09:02:31.605 INFO:teuthology.orchestra.run.smithi112.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 459 kB/s | 23 kB 00:00 2024-03-16T09:02:31.630 INFO:teuthology.orchestra.run.smithi112.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-16T09:02:31.646 INFO:teuthology.orchestra.run.smithi112.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.6 MB/s | 155 kB 00:00 2024-03-16T09:02:31.672 INFO:teuthology.orchestra.run.smithi112.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 3.2 MB/s | 136 kB 00:00 2024-03-16T09:02:31.689 INFO:teuthology.orchestra.run.smithi112.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.5 MB/s | 64 kB 00:00 2024-03-16T09:02:31.706 INFO:teuthology.orchestra.run.smithi112.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 856 kB/s | 29 kB 00:00 2024-03-16T09:02:31.773 INFO:teuthology.orchestra.run.smithi112.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 14 MB/s | 1.2 MB 00:00 2024-03-16T09:02:31.840 INFO:teuthology.orchestra.run.smithi112.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 16 MB/s | 2.2 MB 00:00 2024-03-16T09:02:31.890 INFO:teuthology.orchestra.run.smithi112.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 9.1 MB/s | 465 kB 00:00 2024-03-16T09:02:31.940 INFO:teuthology.orchestra.run.smithi112.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 625 kB/s | 31 kB 00:00 2024-03-16T09:02:31.982 INFO:teuthology.orchestra.run.smithi112.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 11 MB/s | 2.3 MB 00:00 2024-03-16T09:02:32.041 INFO:teuthology.orchestra.run.smithi112.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 11 MB/s | 1.1 MB 00:00 2024-03-16T09:02:32.166 INFO:teuthology.orchestra.run.smithi112.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 405 kB/s | 74 kB 00:00 2024-03-16T09:02:32.258 INFO:teuthology.orchestra.run.smithi194.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-03-16T09:02:36.341 INFO:teuthology.orchestra.run.smithi112.stdout:(174/174): openssl-devel-3.0.7-27.el9.x86_64.rp 360 kB/s | 4.1 MB 00:11 2024-03-16T09:02:36.345 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-03-16T09:02:36.345 INFO:teuthology.orchestra.run.smithi112.stdout:Total 11 MB/s | 224 MB 00:20 2024-03-16T09:02:36.817 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-03-16T09:02:37.126 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-03-16T09:02:37.127 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-03-16T09:02:37.232 INFO:teuthology.orchestra.run.smithi194.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-16T09:02:39.149 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-03-16T09:02:39.149 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-03-16T09:02:39.705 INFO:teuthology.orchestra.run.smithi194.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T09:02:39.706 INFO:teuthology.orchestra.run.smithi194.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T09:02:39.721 INFO:teuthology.orchestra.run.smithi194.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-16T09:02:39.722 INFO:teuthology.orchestra.run.smithi194.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-16T09:02:39.723 INFO:teuthology.orchestra.run.smithi194.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-16T09:02:39.725 INFO:teuthology.orchestra.run.smithi194.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-16T09:02:39.727 INFO:teuthology.orchestra.run.smithi194.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-16T09:02:39.730 INFO:teuthology.orchestra.run.smithi194.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-16T09:02:39.731 INFO:teuthology.orchestra.run.smithi194.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-16T09:02:39.813 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-03-16T09:02:39.834 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================================ 2024-03-16T09:02:39.834 INFO:teuthology.orchestra.run.smithi194.stdout: Package Arch Version Repository Size 2024-03-16T09:02:39.834 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================================ 2024-03-16T09:02:39.834 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-03-16T09:02:39.834 INFO:teuthology.orchestra.run.smithi194.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-16T09:02:39.834 INFO:teuthology.orchestra.run.smithi194.stdout: ceph x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 6.6 k 2024-03-16T09:02:39.834 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-base x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.4 M 2024-03-16T09:02:39.834 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 890 k 2024-03-16T09:02:39.834 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 145 k 2024-03-16T09:02:39.835 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 1.5 M 2024-03-16T09:02:39.835 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 139 k 2024-03-16T09:02:39.835 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 1.8 M 2024-03-16T09:02:39.835 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 7.4 M 2024-03-16T09:02:39.835 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-rook noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 49 k 2024-03-16T09:02:39.835 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-radosgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 9.8 M 2024-03-16T09:02:39.835 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-test x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 48 M 2024-03-16T09:02:39.835 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-volume noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 264 k 2024-03-16T09:02:39.835 INFO:teuthology.orchestra.run.smithi194.stdout: cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 748 k 2024-03-16T09:02:39.835 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 221 k 2024-03-16T09:02:39.835 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-shell x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 27 k 2024-03-16T09:02:39.835 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-top noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 18 k 2024-03-16T09:02:39.835 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-16T09:02:39.836 INFO:teuthology.orchestra.run.smithi194.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-16T09:02:39.836 INFO:teuthology.orchestra.run.smithi194.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-16T09:02:39.836 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 32 k 2024-03-16T09:02:39.836 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 747 k 2024-03-16T09:02:39.836 INFO:teuthology.orchestra.run.smithi194.stdout: librados-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 127 k 2024-03-16T09:02:39.836 INFO:teuthology.orchestra.run.smithi194.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-16T09:02:39.836 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-16T09:02:39.836 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-16T09:02:39.836 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-16T09:02:39.836 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-16T09:02:39.836 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-16T09:02:39.836 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cephfs x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 162 k 2024-03-16T09:02:39.836 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rados x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 320 k 2024-03-16T09:02:39.837 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 298 k 2024-03-16T09:02:39.837 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 99 k 2024-03-16T09:02:39.837 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 85 k 2024-03-16T09:02:39.837 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-16T09:02:39.837 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-nbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 170 k 2024-03-16T09:02:39.837 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-16T09:02:39.837 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrading: 2024-03-16T09:02:39.837 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-16T09:02:39.837 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-16T09:02:39.837 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-16T09:02:39.837 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-16T09:02:39.837 INFO:teuthology.orchestra.run.smithi194.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-16T09:02:39.837 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-16T09:02:39.838 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-16T09:02:39.838 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-16T09:02:39.838 INFO:teuthology.orchestra.run.smithi194.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-16T09:02:39.838 INFO:teuthology.orchestra.run.smithi194.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-16T09:02:39.838 INFO:teuthology.orchestra.run.smithi194.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-16T09:02:39.838 INFO:teuthology.orchestra.run.smithi194.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-16T09:02:39.838 INFO:teuthology.orchestra.run.smithi194.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-16T09:02:39.838 INFO:teuthology.orchestra.run.smithi194.stdout: librados2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.4 M 2024-03-16T09:02:39.838 INFO:teuthology.orchestra.run.smithi194.stdout: librbd1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-16T09:02:39.838 INFO:teuthology.orchestra.run.smithi194.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-16T09:02:39.838 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-16T09:02:39.838 INFO:teuthology.orchestra.run.smithi194.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-16T09:02:39.838 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-16T09:02:39.839 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-16T09:02:39.839 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-16T09:02:39.839 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-16T09:02:39.839 INFO:teuthology.orchestra.run.smithi194.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-16T09:02:39.839 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-16T09:02:39.839 INFO:teuthology.orchestra.run.smithi194.stdout:Installing dependencies: 2024-03-16T09:02:39.839 INFO:teuthology.orchestra.run.smithi194.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-16T09:02:39.839 INFO:teuthology.orchestra.run.smithi194.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-16T09:02:39.839 INFO:teuthology.orchestra.run.smithi194.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-16T09:02:39.839 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 21 M 2024-03-16T09:02:39.839 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 34 k 2024-03-16T09:02:39.839 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mds x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 2.4 M 2024-03-16T09:02:39.839 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 248 k 2024-03-16T09:02:39.840 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mon x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 4.7 M 2024-03-16T09:02:39.840 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-osd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 17 M 2024-03-16T09:02:39.840 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 16 k 2024-03-16T09:02:39.840 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-selinux x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 25 k 2024-03-16T09:02:39.840 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-16T09:02:39.840 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-16T09:02:39.840 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-16T09:02:39.840 INFO:teuthology.orchestra.run.smithi194.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-16T09:02:39.840 INFO:teuthology.orchestra.run.smithi194.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-16T09:02:39.840 INFO:teuthology.orchestra.run.smithi194.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-16T09:02:39.840 INFO:teuthology.orchestra.run.smithi194.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-16T09:02:39.840 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-16T09:02:39.840 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-16T09:02:39.841 INFO:teuthology.orchestra.run.smithi194.stdout: libcephsqlite x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 163 k 2024-03-16T09:02:39.841 INFO:teuthology.orchestra.run.smithi194.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-16T09:02:39.841 INFO:teuthology.orchestra.run.smithi194.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-16T09:02:39.841 INFO:teuthology.orchestra.run.smithi194.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-16T09:02:39.841 INFO:teuthology.orchestra.run.smithi194.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-16T09:02:39.841 INFO:teuthology.orchestra.run.smithi194.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-16T09:02:39.841 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 501 k 2024-03-16T09:02:39.841 INFO:teuthology.orchestra.run.smithi194.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-16T09:02:39.841 INFO:teuthology.orchestra.run.smithi194.stdout: librgw2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.1 M 2024-03-16T09:02:39.841 INFO:teuthology.orchestra.run.smithi194.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-16T09:02:39.841 INFO:teuthology.orchestra.run.smithi194.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-16T09:02:39.841 INFO:teuthology.orchestra.run.smithi194.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-16T09:02:39.841 INFO:teuthology.orchestra.run.smithi194.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-16T09:02:39.842 INFO:teuthology.orchestra.run.smithi194.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-16T09:02:39.842 INFO:teuthology.orchestra.run.smithi194.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-16T09:02:39.842 INFO:teuthology.orchestra.run.smithi194.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-16T09:02:39.842 INFO:teuthology.orchestra.run.smithi194.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-16T09:02:39.842 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-16T09:02:39.842 INFO:teuthology.orchestra.run.smithi194.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-16T09:02:39.842 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-16T09:02:39.842 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-16T09:02:39.842 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-16T09:02:39.842 INFO:teuthology.orchestra.run.smithi194.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-16T09:02:39.842 INFO:teuthology.orchestra.run.smithi194.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-16T09:02:39.842 INFO:teuthology.orchestra.run.smithi194.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-16T09:02:39.843 INFO:teuthology.orchestra.run.smithi194.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-16T09:02:39.843 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-16T09:02:39.843 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-argparse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 45 k 2024-03-16T09:02:39.843 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 131 k 2024-03-16T09:02:39.843 INFO:teuthology.orchestra.run.smithi194.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-16T09:02:39.843 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-16T09:02:39.843 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-16T09:02:39.843 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-16T09:02:39.843 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-16T09:02:39.843 INFO:teuthology.orchestra.run.smithi194.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-16T09:02:39.843 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-16T09:02:39.843 INFO:teuthology.orchestra.run.smithi194.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-16T09:02:39.844 INFO:teuthology.orchestra.run.smithi194.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-16T09:02:39.844 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-16T09:02:39.844 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-16T09:02:39.844 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-16T09:02:39.844 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-16T09:02:39.844 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-16T09:02:39.844 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-16T09:02:39.845 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-16T09:02:39.845 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-16T09:02:39.845 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-16T09:02:39.845 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-16T09:02:39.845 INFO:teuthology.orchestra.run.smithi194.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-16T09:02:39.845 INFO:teuthology.orchestra.run.smithi194.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-16T09:02:39.845 INFO:teuthology.orchestra.run.smithi194.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-16T09:02:39.845 INFO:teuthology.orchestra.run.smithi194.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-16T09:02:39.846 INFO:teuthology.orchestra.run.smithi194.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-16T09:02:39.846 INFO:teuthology.orchestra.run.smithi194.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-16T09:02:39.846 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-16T09:02:39.846 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-16T09:02:39.846 INFO:teuthology.orchestra.run.smithi194.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-16T09:02:39.846 INFO:teuthology.orchestra.run.smithi194.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-16T09:02:39.846 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-16T09:02:39.847 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-16T09:02:39.847 INFO:teuthology.orchestra.run.smithi194.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-16T09:02:39.847 INFO:teuthology.orchestra.run.smithi194.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-16T09:02:39.847 INFO:teuthology.orchestra.run.smithi194.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-16T09:02:39.847 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-16T09:02:39.847 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-16T09:02:39.847 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-16T09:02:39.847 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-16T09:02:39.847 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-16T09:02:39.847 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-16T09:02:39.847 INFO:teuthology.orchestra.run.smithi194.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-16T09:02:39.847 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-16T09:02:39.848 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-16T09:02:39.848 INFO:teuthology.orchestra.run.smithi194.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-16T09:02:39.848 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-16T09:02:39.848 INFO:teuthology.orchestra.run.smithi194.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-16T09:02:39.848 INFO:teuthology.orchestra.run.smithi194.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-16T09:02:39.848 INFO:teuthology.orchestra.run.smithi194.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-16T09:02:39.848 INFO:teuthology.orchestra.run.smithi194.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-16T09:02:39.848 INFO:teuthology.orchestra.run.smithi194.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-16T09:02:39.848 INFO:teuthology.orchestra.run.smithi194.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-16T09:02:39.848 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-16T09:02:39.849 INFO:teuthology.orchestra.run.smithi194.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-16T09:02:39.849 INFO:teuthology.orchestra.run.smithi194.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-16T09:02:39.849 INFO:teuthology.orchestra.run.smithi194.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-16T09:02:39.849 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-16T09:02:39.849 INFO:teuthology.orchestra.run.smithi194.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-16T09:02:39.849 INFO:teuthology.orchestra.run.smithi194.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-16T09:02:39.849 INFO:teuthology.orchestra.run.smithi194.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-16T09:02:39.849 INFO:teuthology.orchestra.run.smithi194.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-16T09:02:39.850 INFO:teuthology.orchestra.run.smithi194.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-16T09:02:39.850 INFO:teuthology.orchestra.run.smithi194.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-16T09:02:39.850 INFO:teuthology.orchestra.run.smithi194.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-16T09:02:39.850 INFO:teuthology.orchestra.run.smithi194.stdout:Installing weak dependencies: 2024-03-16T09:02:39.850 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-16T09:02:39.850 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:02:39.850 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-03-16T09:02:39.851 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================================ 2024-03-16T09:02:39.851 INFO:teuthology.orchestra.run.smithi194.stdout:Install 150 Packages 2024-03-16T09:02:39.851 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrade 24 Packages 2024-03-16T09:02:39.851 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:02:39.852 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 224 M 2024-03-16T09:02:39.852 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-03-16T09:02:40.700 INFO:teuthology.orchestra.run.smithi194.stdout:(1/174): ceph-19.0.0-2162.g3d78f5ba.el9.x86_64. 32 kB/s | 6.6 kB 00:00 2024-03-16T09:02:41.292 INFO:teuthology.orchestra.run.smithi194.stdout:(2/174): ceph-fuse-19.0.0-2162.g3d78f5ba.el9.x8 1.5 MB/s | 890 kB 00:00 2024-03-16T09:02:41.475 INFO:teuthology.orchestra.run.smithi194.stdout:(3/174): ceph-immutable-object-cache-19.0.0-216 791 kB/s | 145 kB 00:00 2024-03-16T09:02:41.726 INFO:teuthology.orchestra.run.smithi194.stdout:(4/174): ceph-common-19.0.0-2162.g3d78f5ba.el9. 17 MB/s | 21 MB 00:01 2024-03-16T09:02:41.826 INFO:teuthology.orchestra.run.smithi194.stdout:(5/174): ceph-mgr-19.0.0-2162.g3d78f5ba.el9.x86 15 MB/s | 1.5 MB 00:00 2024-03-16T09:02:42.110 INFO:teuthology.orchestra.run.smithi194.stdout:(6/174): ceph-mon-19.0.0-2162.g3d78f5ba.el9.x86 17 MB/s | 4.7 MB 00:00 2024-03-16T09:02:42.518 INFO:teuthology.orchestra.run.smithi194.stdout:(7/174): ceph-mds-19.0.0-2162.g3d78f5ba.el9.x86 2.3 MB/s | 2.4 MB 00:01 2024-03-16T09:02:42.785 INFO:teuthology.orchestra.run.smithi194.stdout:(8/174): ceph-base-19.0.0-2162.g3d78f5ba.el9.x8 2.3 MB/s | 5.4 MB 00:02 2024-03-16T09:02:42.985 INFO:teuthology.orchestra.run.smithi194.stdout:(9/174): ceph-osd-19.0.0-2162.g3d78f5ba.el9.x86 19 MB/s | 17 MB 00:00 2024-03-16T09:02:43.011 INFO:teuthology.orchestra.run.smithi194.stdout:(10/174): ceph-selinux-19.0.0-2162.g3d78f5ba.el 112 kB/s | 25 kB 00:00 2024-03-16T09:02:43.127 INFO:teuthology.orchestra.run.smithi194.stdout:(11/174): cephfs-mirror-19.0.0-2162.g3d78f5ba.e 1.9 MB/s | 221 kB 00:00 2024-03-16T09:02:43.236 INFO:teuthology.orchestra.run.smithi194.stdout:(12/174): cephfs-shell-19.0.0-2162.g3d78f5ba.el 250 kB/s | 27 kB 00:00 2024-03-16T09:02:43.353 INFO:teuthology.orchestra.run.smithi194.stdout:(13/174): libcephfs-devel-19.0.0-2162.g3d78f5ba 271 kB/s | 32 kB 00:00 2024-03-16T09:02:43.745 INFO:teuthology.orchestra.run.smithi194.stdout:(14/174): libcephfs2-19.0.0-2162.g3d78f5ba.el9. 1.9 MB/s | 747 kB 00:00 2024-03-16T09:02:43.862 INFO:teuthology.orchestra.run.smithi194.stdout:(15/174): libcephsqlite-19.0.0-2162.g3d78f5ba.e 1.4 MB/s | 163 kB 00:00 2024-03-16T09:02:43.952 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - CRB 506 kB/s | 6.1 MB 00:12 2024-03-16T09:02:43.970 INFO:teuthology.orchestra.run.smithi194.stdout:(16/174): librados-devel-19.0.0-2162.g3d78f5ba. 1.1 MB/s | 127 kB 00:00 2024-03-16T09:02:44.296 INFO:teuthology.orchestra.run.smithi194.stdout:(17/174): libradosstriper1-19.0.0-2162.g3d78f5b 1.5 MB/s | 501 kB 00:00 2024-03-16T09:02:44.563 INFO:teuthology.orchestra.run.smithi194.stdout:(18/174): ceph-radosgw-19.0.0-2162.g3d78f5ba.el 4.8 MB/s | 9.8 MB 00:02 2024-03-16T09:02:44.654 INFO:teuthology.orchestra.run.smithi194.stdout:(19/174): python3-ceph-argparse-19.0.0-2162.g3d 494 kB/s | 45 kB 00:00 2024-03-16T09:02:44.763 INFO:teuthology.orchestra.run.smithi194.stdout:(20/174): python3-ceph-common-19.0.0-2162.g3d78 1.2 MB/s | 131 kB 00:00 2024-03-16T09:02:44.880 INFO:teuthology.orchestra.run.smithi194.stdout:(21/174): python3-cephfs-19.0.0-2162.g3d78f5ba. 1.4 MB/s | 162 kB 00:00 2024-03-16T09:02:45.014 INFO:teuthology.orchestra.run.smithi194.stdout:(22/174): python3-rados-19.0.0-2162.g3d78f5ba.e 2.4 MB/s | 320 kB 00:00 2024-03-16T09:02:45.436 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 15 kB 00:00 2024-03-16T09:02:45.473 INFO:teuthology.orchestra.run.smithi194.stdout:(23/174): ceph-test-19.0.0-2162.g3d78f5ba.el9.x 19 MB/s | 48 MB 00:02 2024-03-16T09:02:45.497 INFO:teuthology.orchestra.run.smithi194.stdout:(24/174): python3-rbd-19.0.0-2162.g3d78f5ba.el9 616 kB/s | 298 kB 00:00 2024-03-16T09:02:45.523 INFO:teuthology.orchestra.run.smithi194.stdout:(25/174): python3-rgw-19.0.0-2162.g3d78f5ba.el9 1.9 MB/s | 99 kB 00:00 2024-03-16T09:02:45.565 INFO:teuthology.orchestra.run.smithi194.stdout:(26/174): rbd-fuse-19.0.0-2162.g3d78f5ba.el9.x8 1.3 MB/s | 85 kB 00:00 2024-03-16T09:02:45.682 INFO:teuthology.orchestra.run.smithi194.stdout:(27/174): rbd-nbd-19.0.0-2162.g3d78f5ba.el9.x86 1.4 MB/s | 170 kB 00:00 2024-03-16T09:02:45.732 INFO:teuthology.orchestra.run.smithi194.stdout:(28/174): rbd-mirror-19.0.0-2162.g3d78f5ba.el9. 15 MB/s | 3.1 MB 00:00 2024-03-16T09:02:45.757 INFO:teuthology.orchestra.run.smithi194.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2162.g 453 kB/s | 34 kB 00:00 2024-03-16T09:02:45.782 INFO:teuthology.orchestra.run.smithi194.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2162.g3d78f5b 2.7 MB/s | 139 kB 00:00 2024-03-16T09:02:46.024 INFO:teuthology.orchestra.run.smithi194.stdout:(31/174): librgw2-19.0.0-2162.g3d78f5ba.el9.x86 3.0 MB/s | 5.1 MB 00:01 2024-03-16T09:02:46.066 INFO:teuthology.orchestra.run.smithi194.stdout:(32/174): ceph-mgr-dashboard-19.0.0-2162.g3d78f 5.8 MB/s | 1.8 MB 00:00 2024-03-16T09:02:46.092 INFO:teuthology.orchestra.run.smithi194.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2162.g3d 3.6 MB/s | 248 kB 00:00 2024-03-16T09:02:46.125 INFO:teuthology.orchestra.run.smithi194.stdout:(34/174): ceph-mgr-rook-19.0.0-2162.g3d78f5ba.e 845 kB/s | 49 kB 00:00 2024-03-16T09:02:46.196 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-03-16T09:02:46.225 INFO:teuthology.orchestra.run.smithi194.stdout:(35/174): ceph-mgr-diskprediction-local-19.0.0- 17 MB/s | 7.4 MB 00:00 2024-03-16T09:02:46.251 INFO:teuthology.orchestra.run.smithi194.stdout:(36/174): ceph-prometheus-alerts-19.0.0-2162.g3 103 kB/s | 16 kB 00:00 2024-03-16T09:02:46.276 INFO:teuthology.orchestra.run.smithi194.stdout:(37/174): ceph-volume-19.0.0-2162.g3d78f5ba.el9 1.7 MB/s | 264 kB 00:00 2024-03-16T09:02:46.304 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-16T09:02:46.310 INFO:teuthology.orchestra.run.smithi194.stdout:(38/174): cephadm-19.0.0-2162.g3d78f5ba.el9.noa 8.7 MB/s | 748 kB 00:00 2024-03-16T09:02:46.339 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-16T09:02:46.362 INFO:teuthology.orchestra.run.smithi194.stdout:(39/174): cephfs-top-19.0.0-2162.g3d78f5ba.el9. 168 kB/s | 18 kB 00:00 2024-03-16T09:02:46.402 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T09:02:46.411 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T09:02:46.455 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-16T09:02:46.468 INFO:teuthology.orchestra.run.smithi194.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 662 kB 00:00 2024-03-16T09:02:46.494 INFO:teuthology.orchestra.run.smithi194.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 996 kB/s | 106 kB 00:00 2024-03-16T09:02:46.505 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-16T09:02:46.527 INFO:teuthology.orchestra.run.smithi194.stdout:(42/174): bison-3.7.4-5.el9.x86_64.rpm 4.7 MB/s | 920 kB 00:00 2024-03-16T09:02:46.538 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-16T09:02:46.552 INFO:teuthology.orchestra.run.smithi194.stdout:(43/174): flex-2.6.4-9.el9.x86_64.rpm 3.6 MB/s | 307 kB 00:00 2024-03-16T09:02:46.566 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-16T09:02:46.577 INFO:teuthology.orchestra.run.smithi194.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 383 kB/s | 32 kB 00:00 2024-03-16T09:02:46.603 INFO:teuthology.orchestra.run.smithi194.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 342 kB/s | 17 kB 00:00 2024-03-16T09:02:46.613 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-16T09:02:46.712 INFO:teuthology.orchestra.run.smithi194.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-03-16T09:02:46.737 INFO:teuthology.orchestra.run.smithi194.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 63 kB/s | 10 kB 00:00 2024-03-16T09:02:46.762 INFO:teuthology.orchestra.run.smithi194.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 159 kB 00:00 2024-03-16T09:02:46.787 INFO:teuthology.orchestra.run.smithi194.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 588 kB/s | 44 kB 00:00 2024-03-16T09:02:46.837 INFO:teuthology.orchestra.run.smithi194.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 7.5 MB/s | 578 kB 00:00 2024-03-16T09:02:46.863 INFO:teuthology.orchestra.run.smithi194.stdout:(51/174): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 661 kB 00:00 2024-03-16T09:02:46.888 INFO:teuthology.orchestra.run.smithi194.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 463 kB/s | 46 kB 00:00 2024-03-16T09:02:46.913 INFO:teuthology.orchestra.run.smithi194.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-16T09:02:46.935 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-16T09:02:46.938 INFO:teuthology.orchestra.run.smithi194.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-16T09:02:46.963 INFO:teuthology.orchestra.run.smithi194.stdout:(55/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-16T09:02:46.989 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T09:02:46.989 INFO:teuthology.orchestra.run.smithi194.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 453 kB/s | 34 kB 00:00 2024-03-16T09:02:47.150 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T09:02:47.156 INFO:teuthology.orchestra.run.smithi194.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 22 MB/s | 4.7 MB 00:00 2024-03-16T09:02:47.182 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T09:02:47.189 INFO:teuthology.orchestra.run.smithi194.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 6.8 MB/s | 1.5 MB 00:00 2024-03-16T09:02:47.214 INFO:teuthology.orchestra.run.smithi194.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 3.5 MB/s | 811 kB 00:00 2024-03-16T09:02:47.240 INFO:teuthology.orchestra.run.smithi194.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.6 MB/s | 134 kB 00:00 2024-03-16T09:02:47.276 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T09:02:47.298 INFO:teuthology.orchestra.run.smithi194.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-03-16T09:02:47.310 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-16T09:02:47.354 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-16T09:02:47.407 INFO:teuthology.orchestra.run.smithi194.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 30 MB/s | 5.7 MB 00:00 2024-03-16T09:02:47.432 INFO:teuthology.orchestra.run.smithi194.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 404 kB/s | 78 kB 00:00 2024-03-16T09:02:47.524 INFO:teuthology.orchestra.run.smithi194.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 5.8 MB 00:00 2024-03-16T09:02:47.541 INFO:teuthology.orchestra.run.smithi194.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 186 kB/s | 24 kB 00:00 2024-03-16T09:02:47.542 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-16T09:02:47.558 INFO:teuthology.orchestra.run.smithi194.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 148 kB/s | 18 kB 00:00 2024-03-16T09:02:47.575 INFO:teuthology.orchestra.run.smithi194.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-16T09:02:47.592 INFO:teuthology.orchestra.run.smithi194.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 951 kB/s | 32 kB 00:00 2024-03-16T09:02:47.592 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-16T09:02:47.620 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T09:02:47.692 INFO:teuthology.orchestra.run.smithi194.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2024-03-16T09:02:47.715 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T09:02:47.717 INFO:teuthology.orchestra.run.smithi194.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-03-16T09:02:47.738 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-16T09:02:47.742 INFO:teuthology.orchestra.run.smithi194.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2024-03-16T09:02:47.768 INFO:teuthology.orchestra.run.smithi194.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 916 kB/s | 69 kB 00:00 2024-03-16T09:02:47.793 INFO:teuthology.orchestra.run.smithi194.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 826 kB/s | 41 kB 00:00 2024-03-16T09:02:47.794 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-16T09:02:47.818 INFO:teuthology.orchestra.run.smithi194.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-16T09:02:47.834 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-16T09:02:47.843 INFO:teuthology.orchestra.run.smithi194.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-16T09:02:47.960 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T09:02:47.960 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T09:02:47.960 INFO:teuthology.orchestra.run.smithi194.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 803 kB/s | 47 kB 00:00 2024-03-16T09:02:47.960 INFO:teuthology.orchestra.run.smithi194.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.4 MB/s | 211 kB 00:00 2024-03-16T09:02:47.961 INFO:teuthology.orchestra.run.smithi194.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 507 kB/s | 43 kB 00:00 2024-03-16T09:02:48.018 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T09:02:48.047 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T09:02:48.077 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-16T09:02:48.120 INFO:teuthology.orchestra.run.smithi194.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 204 kB/s | 39 kB 00:00 2024-03-16T09:02:48.146 INFO:teuthology.orchestra.run.smithi194.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 191 kB/s | 41 kB 00:00 2024-03-16T09:02:48.262 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-16T09:02:48.296 INFO:teuthology.orchestra.run.smithi194.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-03-16T09:02:48.322 INFO:teuthology.orchestra.run.smithi194.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 300 kB 00:00 2024-03-16T09:02:48.347 INFO:teuthology.orchestra.run.smithi194.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 283 kB/s | 56 kB 00:00 2024-03-16T09:02:48.372 INFO:teuthology.orchestra.run.smithi194.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-16T09:02:48.390 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T09:02:48.397 INFO:teuthology.orchestra.run.smithi194.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 588 kB/s | 44 kB 00:00 2024-03-16T09:02:48.422 INFO:teuthology.orchestra.run.smithi194.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-16T09:02:48.434 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T09:02:48.448 INFO:teuthology.orchestra.run.smithi194.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-16T09:02:48.481 INFO:teuthology.orchestra.run.smithi194.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-16T09:02:48.507 INFO:teuthology.orchestra.run.smithi194.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-16T09:02:48.531 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T09:02:48.532 INFO:teuthology.orchestra.run.smithi194.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-03-16T09:02:48.557 INFO:teuthology.orchestra.run.smithi194.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-16T09:02:48.585 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T09:02:48.674 INFO:teuthology.orchestra.run.smithi194.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-03-16T09:02:48.699 INFO:teuthology.orchestra.run.smithi194.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-03-16T09:02:48.724 INFO:teuthology.orchestra.run.smithi194.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-16T09:02:48.766 INFO:teuthology.orchestra.run.smithi194.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-03-16T09:02:48.806 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T09:02:48.825 INFO:teuthology.orchestra.run.smithi194.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-03-16T09:02:48.850 INFO:teuthology.orchestra.run.smithi194.stdout:(97/174): elfutils-libelf-devel-0.190-2.el9.x86 143 kB/s | 21 kB 00:00 2024-03-16T09:02:48.915 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-16T09:02:48.967 INFO:teuthology.orchestra.run.smithi194.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-03-16T09:02:48.993 INFO:teuthology.orchestra.run.smithi194.stdout:(99/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2024-03-16T09:02:49.018 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-16T09:02:49.043 INFO:teuthology.orchestra.run.smithi194.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 697 kB/s | 35 kB 00:00 2024-03-16T09:02:49.057 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-16T09:02:49.068 INFO:teuthology.orchestra.run.smithi194.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.1 MB/s | 216 kB 00:00 2024-03-16T09:02:49.084 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-16T09:02:49.085 INFO:teuthology.orchestra.run.smithi194.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-16T09:02:49.118 INFO:teuthology.orchestra.run.smithi194.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-16T09:02:49.119 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-16T09:02:49.144 INFO:teuthology.orchestra.run.smithi194.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 380 kB/s | 22 kB 00:00 2024-03-16T09:02:49.181 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 5.7 MB/s | 21 MB 00:03 2024-03-16T09:02:49.192 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-16T09:02:49.211 INFO:teuthology.orchestra.run.smithi194.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 9.2 MB/s | 863 kB 00:00 2024-03-16T09:02:49.255 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-16T09:02:49.293 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-16T09:02:49.357 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-16T09:02:49.393 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-16T09:02:49.436 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-16T09:02:49.469 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-16T09:02:49.497 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-16T09:02:49.582 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-16T09:02:49.618 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 39/198 2024-03-16T09:02:49.660 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 40/198 2024-03-16T09:02:49.690 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 41/198 2024-03-16T09:02:49.728 INFO:teuthology.orchestra.run.smithi194.stdout:(106/174): openssl-devel-3.0.7-27.el9.x86_64.rp 4.3 MB/s | 4.1 MB 00:00 2024-03-16T09:02:49.760 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-16T09:02:49.805 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-16T09:02:49.878 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-16T09:02:49.894 INFO:teuthology.orchestra.run.smithi194.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 731 kB/s | 548 kB 00:00 2024-03-16T09:02:49.994 INFO:teuthology.orchestra.run.smithi194.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 394 kB/s | 309 kB 00:00 2024-03-16T09:02:50.010 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-16T09:02:50.057 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-16T09:02:50.170 INFO:teuthology.orchestra.run.smithi194.stdout:(109/174): grpc-data-1.46.7-10.el9.noarch.rpm 44 kB/s | 19 kB 00:00 2024-03-16T09:02:50.254 INFO:teuthology.orchestra.run.smithi194.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 100 kB/s | 26 kB 00:00 2024-03-16T09:02:50.404 INFO:teuthology.orchestra.run.smithi194.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 208 kB/s | 49 kB 00:00 2024-03-16T09:02:50.421 INFO:teuthology.orchestra.run.smithi194.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 403 kB/s | 67 kB 00:00 2024-03-16T09:02:50.513 INFO:teuthology.orchestra.run.smithi194.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 7.1 MB/s | 4.4 MB 00:00 2024-03-16T09:02:50.569 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-16T09:02:50.645 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-16T09:02:50.730 INFO:teuthology.orchestra.run.smithi194.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 838 kB 00:00 2024-03-16T09:02:50.737 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-16T09:02:50.755 INFO:teuthology.orchestra.run.smithi194.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 2.2 MB/s | 548 kB 00:00 2024-03-16T09:02:50.790 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T09:02:50.798 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T09:02:50.822 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-16T09:02:50.830 INFO:teuthology.orchestra.run.smithi194.stdout:(116/174): luarocks-3.9.2-1.el9.noarch.rpm 355 kB/s | 151 kB 00:00 2024-03-16T09:02:50.870 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-16T09:02:50.916 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 53/198 2024-03-16T09:02:50.965 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-16T09:02:51.005 INFO:teuthology.orchestra.run.smithi194.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 86 kB/s | 15 kB 00:00 2024-03-16T09:02:51.181 INFO:teuthology.orchestra.run.smithi194.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 76 kB/s | 32 kB 00:00 2024-03-16T09:02:51.206 INFO:teuthology.orchestra.run.smithi194.stdout:(119/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 91 kB/s | 43 kB 00:00 2024-03-16T09:02:51.221 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-16T09:02:51.248 INFO:teuthology.orchestra.run.smithi194.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 709 kB/s | 172 kB 00:00 2024-03-16T09:02:51.373 INFO:teuthology.orchestra.run.smithi194.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 282 kB/s | 35 kB 00:00 2024-03-16T09:02:51.398 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-16T09:02:51.520 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-16T09:02:51.532 INFO:teuthology.orchestra.run.smithi194.stdout:(122/174): python3-cherrypy-18.6.1-2.el9.noarch 1.0 MB/s | 358 kB 00:00 2024-03-16T09:02:51.557 INFO:teuthology.orchestra.run.smithi194.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 1.1 MB/s | 203 kB 00:00 2024-03-16T09:02:51.611 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-16T09:02:51.615 INFO:teuthology.orchestra.run.smithi194.stdout:(124/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 741 kB/s | 303 kB 00:00 2024-03-16T09:02:51.724 INFO:teuthology.orchestra.run.smithi194.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-03-16T09:02:51.750 INFO:teuthology.orchestra.run.smithi194.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 752 kB/s | 144 kB 00:00 2024-03-16T09:02:51.756 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-16T09:02:51.797 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-16T09:02:51.799 INFO:teuthology.orchestra.run.smithi194.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 58 kB/s | 11 kB 00:00 2024-03-16T09:02:51.841 INFO:teuthology.orchestra.run.smithi194.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-03-16T09:02:51.851 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-16T09:02:51.867 INFO:teuthology.orchestra.run.smithi194.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 198 kB/s | 23 kB 00:00 2024-03-16T09:02:51.923 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-16T09:02:51.942 INFO:teuthology.orchestra.run.smithi194.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 137 kB/s | 19 kB 00:00 2024-03-16T09:02:51.956 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-16T09:02:51.984 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-16T09:02:52.001 INFO:teuthology.orchestra.run.smithi194.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 124 kB/s | 20 kB 00:00 2024-03-16T09:02:52.030 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-16T09:02:52.067 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-16T09:02:52.084 INFO:teuthology.orchestra.run.smithi194.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 326 kB/s | 46 kB 00:00 2024-03-16T09:02:52.126 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-16T09:02:52.143 INFO:teuthology.orchestra.run.smithi194.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 557 kB/s | 79 kB 00:00 2024-03-16T09:02:52.169 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-16T09:02:52.176 INFO:teuthology.orchestra.run.smithi194.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 3.3 MB/s | 1.0 MB 00:00 2024-03-16T09:02:52.243 INFO:teuthology.orchestra.run.smithi194.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 364 kB/s | 58 kB 00:00 2024-03-16T09:02:52.300 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-16T09:02:52.349 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-16T09:02:52.368 INFO:teuthology.orchestra.run.smithi194.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 85 kB/s | 16 kB 00:00 2024-03-16T09:02:52.379 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-16T09:02:52.420 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-16T09:02:52.523 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-16T09:02:52.527 INFO:teuthology.orchestra.run.smithi194.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 317 kB/s | 90 kB 00:00 2024-03-16T09:02:52.554 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-16T09:02:52.569 INFO:teuthology.orchestra.run.smithi194.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 141 kB/s | 28 kB 00:00 2024-03-16T09:02:52.594 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-16T09:02:52.602 INFO:teuthology.orchestra.run.smithi194.stdout:(139/174): python3-pecan-1.4.2-3.el9.noarch.rpm 593 kB/s | 272 kB 00:00 2024-03-16T09:02:52.620 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-16T09:02:52.644 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-16T09:02:52.722 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-16T09:02:52.728 INFO:teuthology.orchestra.run.smithi194.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-03-16T09:02:52.753 INFO:teuthology.orchestra.run.smithi194.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 393 kB/s | 59 kB 00:00 2024-03-16T09:02:52.778 INFO:teuthology.orchestra.run.smithi194.stdout:(142/174): python3-repoze-lru-0.7-16.el9.noarch 122 kB/s | 31 kB 00:00 2024-03-16T09:02:52.845 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-16T09:02:52.929 INFO:teuthology.orchestra.run.smithi194.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 178 kB/s | 36 kB 00:00 2024-03-16T09:02:52.941 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-16T09:02:52.954 INFO:teuthology.orchestra.run.smithi194.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 254 kB/s | 51 kB 00:00 2024-03-16T09:02:52.979 INFO:teuthology.orchestra.run.smithi194.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.1 MB/s | 230 kB 00:00 2024-03-16T09:02:52.985 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-16T09:02:53.042 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-16T09:02:53.088 INFO:teuthology.orchestra.run.smithi194.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 565 kB/s | 90 kB 00:00 2024-03-16T09:02:53.098 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-16T09:02:53.104 INFO:teuthology.orchestra.run.smithi194.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 160 kB/s | 20 kB 00:00 2024-03-16T09:02:53.138 INFO:teuthology.orchestra.run.smithi194.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.3 MB/s | 427 kB 00:00 2024-03-16T09:02:53.230 INFO:teuthology.orchestra.run.smithi194.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-03-16T09:02:53.281 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-16T09:02:53.354 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-16T09:02:53.372 INFO:teuthology.orchestra.run.smithi194.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 5.9 MB/s | 1.6 MB 00:00 2024-03-16T09:02:53.414 INFO:teuthology.orchestra.run.smithi194.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-16T09:02:53.492 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-16T09:02:53.538 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-16T09:02:53.569 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-16T09:02:53.600 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-16T09:02:53.628 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-16T09:02:53.631 INFO:teuthology.orchestra.run.smithi194.stdout:(152/174): NetworkManager-1.46.0-1.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-03-16T09:02:53.662 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-16T09:02:53.681 INFO:teuthology.orchestra.run.smithi194.stdout:(153/174): librbd1-19.0.0-2162.g3d78f5ba.el9.x8 7.0 MB/s | 3.1 MB 00:00 2024-03-16T09:02:53.701 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-16T09:02:53.723 INFO:teuthology.orchestra.run.smithi194.stdout:(154/174): NetworkManager-libnm-1.46.0-1.el9.x8 20 MB/s | 1.8 MB 00:00 2024-03-16T09:02:53.724 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-16T09:02:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-16T09:02:53.773 INFO:teuthology.orchestra.run.smithi194.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 4.9 MB/s | 248 kB 00:00 2024-03-16T09:02:53.799 INFO:teuthology.orchestra.run.smithi194.stdout:(156/174): NetworkManager-team-1.46.0-1.el9.x86 361 kB/s | 42 kB 00:00 2024-03-16T09:02:53.824 INFO:teuthology.orchestra.run.smithi194.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-03-16T09:02:53.829 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-16T09:02:53.849 INFO:teuthology.orchestra.run.smithi194.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 728 kB/s | 36 kB 00:00 2024-03-16T09:02:53.865 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-16T09:02:53.874 INFO:teuthology.orchestra.run.smithi194.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.8 MB/s | 192 kB 00:00 2024-03-16T09:02:53.898 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-16T09:02:53.925 INFO:teuthology.orchestra.run.smithi194.stdout:(160/174): librados2-19.0.0-2162.g3d78f5ba.el9. 4.3 MB/s | 3.4 MB 00:00 2024-03-16T09:02:53.939 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-16T09:02:53.950 INFO:teuthology.orchestra.run.smithi194.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 306 kB/s | 23 kB 00:00 2024-03-16T09:02:53.974 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-16T09:02:53.975 INFO:teuthology.orchestra.run.smithi194.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.0 MB/s | 256 kB 00:00 2024-03-16T09:02:54.000 INFO:teuthology.orchestra.run.smithi194.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-03-16T09:02:54.026 INFO:teuthology.orchestra.run.smithi194.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-03-16T09:02:54.051 INFO:teuthology.orchestra.run.smithi194.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-16T09:02:54.061 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-16T09:02:54.076 INFO:teuthology.orchestra.run.smithi194.stdout:(166/174): libblkid-2.37.4-18.el9.x86_64.rpm 717 kB/s | 108 kB 00:00 2024-03-16T09:02:54.101 INFO:teuthology.orchestra.run.smithi194.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 381 kB/s | 29 kB 00:00 2024-03-16T09:02:54.135 INFO:teuthology.orchestra.run.smithi194.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 14 MB/s | 1.2 MB 00:00 2024-03-16T09:02:54.143 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-16T09:02:54.153 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-16T09:02:54.188 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-16T09:02:54.193 INFO:teuthology.orchestra.run.smithi194.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 7.8 MB/s | 465 kB 00:00 2024-03-16T09:02:54.220 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-16T09:02:54.235 INFO:teuthology.orchestra.run.smithi194.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 14 MB/s | 2.2 MB 00:00 2024-03-16T09:02:54.261 INFO:teuthology.orchestra.run.smithi194.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 465 kB/s | 31 kB 00:00 2024-03-16T09:02:54.266 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-16T09:02:54.311 INFO:teuthology.orchestra.run.smithi194.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 11 MB/s | 2.3 MB 00:00 2024-03-16T09:02:54.346 INFO:teuthology.orchestra.run.smithi194.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 9.9 MB/s | 1.1 MB 00:00 2024-03-16T09:02:54.364 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-16T09:02:54.370 INFO:teuthology.orchestra.run.smithi194.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 684 kB/s | 74 kB 00:00 2024-03-16T09:02:54.377 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-03-16T09:02:54.377 INFO:teuthology.orchestra.run.smithi194.stdout:Total 15 MB/s | 224 MB 00:14 2024-03-16T09:02:54.412 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-16T09:02:54.846 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-03-16T09:02:55.125 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-03-16T09:02:55.125 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-03-16T09:02:55.125 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-16T09:02:55.168 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-16T09:02:55.202 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-16T09:02:55.251 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-16T09:02:55.338 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-16T09:02:55.369 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-16T09:02:55.547 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-16T09:02:55.575 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-16T09:02:55.651 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-16T09:02:56.298 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-16T09:02:56.412 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-16T09:02:56.643 INFO:teuthology.orchestra.run.smithi159.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T09:02:56.644 INFO:teuthology.orchestra.run.smithi159.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T09:02:56.659 INFO:teuthology.orchestra.run.smithi159.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-16T09:02:56.660 INFO:teuthology.orchestra.run.smithi159.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-16T09:02:56.661 INFO:teuthology.orchestra.run.smithi159.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-16T09:02:56.662 INFO:teuthology.orchestra.run.smithi159.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-16T09:02:56.664 INFO:teuthology.orchestra.run.smithi159.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-16T09:02:56.667 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-16T09:02:56.669 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-16T09:02:56.752 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-03-16T09:02:56.787 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================ 2024-03-16T09:02:56.787 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-03-16T09:02:56.787 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================ 2024-03-16T09:02:56.787 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-03-16T09:02:56.787 INFO:teuthology.orchestra.run.smithi159.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-16T09:02:56.787 INFO:teuthology.orchestra.run.smithi159.stdout: ceph x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 6.6 k 2024-03-16T09:02:56.787 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.4 M 2024-03-16T09:02:56.787 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 890 k 2024-03-16T09:02:56.787 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 145 k 2024-03-16T09:02:56.787 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 1.5 M 2024-03-16T09:02:56.788 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 139 k 2024-03-16T09:02:56.788 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 1.8 M 2024-03-16T09:02:56.788 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 7.4 M 2024-03-16T09:02:56.788 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 49 k 2024-03-16T09:02:56.788 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 9.8 M 2024-03-16T09:02:56.788 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 48 M 2024-03-16T09:02:56.788 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 264 k 2024-03-16T09:02:56.788 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 748 k 2024-03-16T09:02:56.788 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 221 k 2024-03-16T09:02:56.788 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-shell x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 27 k 2024-03-16T09:02:56.788 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 18 k 2024-03-16T09:02:56.788 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-16T09:02:56.788 INFO:teuthology.orchestra.run.smithi159.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-16T09:02:56.789 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-16T09:02:56.789 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 32 k 2024-03-16T09:02:56.789 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 747 k 2024-03-16T09:02:56.789 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 127 k 2024-03-16T09:02:56.789 INFO:teuthology.orchestra.run.smithi159.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-16T09:02:56.789 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-16T09:02:56.789 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-16T09:02:56.789 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-16T09:02:56.789 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-16T09:02:56.789 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-16T09:02:56.789 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 162 k 2024-03-16T09:02:56.789 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 320 k 2024-03-16T09:02:56.790 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 298 k 2024-03-16T09:02:56.790 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 99 k 2024-03-16T09:02:56.790 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 85 k 2024-03-16T09:02:56.790 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-16T09:02:56.790 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 170 k 2024-03-16T09:02:56.790 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-16T09:02:56.790 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrading: 2024-03-16T09:02:56.790 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-16T09:02:56.790 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-16T09:02:56.790 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-16T09:02:56.790 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-16T09:02:56.790 INFO:teuthology.orchestra.run.smithi159.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-16T09:02:56.791 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-16T09:02:56.791 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-16T09:02:56.791 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-16T09:02:56.791 INFO:teuthology.orchestra.run.smithi159.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-16T09:02:56.791 INFO:teuthology.orchestra.run.smithi159.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-16T09:02:56.791 INFO:teuthology.orchestra.run.smithi159.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-16T09:02:56.791 INFO:teuthology.orchestra.run.smithi159.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-16T09:02:56.791 INFO:teuthology.orchestra.run.smithi159.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-16T09:02:56.791 INFO:teuthology.orchestra.run.smithi159.stdout: librados2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.4 M 2024-03-16T09:02:56.791 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-16T09:02:56.791 INFO:teuthology.orchestra.run.smithi159.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-16T09:02:56.791 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-16T09:02:56.791 INFO:teuthology.orchestra.run.smithi159.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-16T09:02:56.791 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-16T09:02:56.792 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-16T09:02:56.792 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-16T09:02:56.792 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-16T09:02:56.792 INFO:teuthology.orchestra.run.smithi159.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-16T09:02:56.792 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-16T09:02:56.792 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2024-03-16T09:02:56.792 INFO:teuthology.orchestra.run.smithi159.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-16T09:02:56.792 INFO:teuthology.orchestra.run.smithi159.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-16T09:02:56.792 INFO:teuthology.orchestra.run.smithi159.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-16T09:02:56.792 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 21 M 2024-03-16T09:02:56.792 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 34 k 2024-03-16T09:02:56.792 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 2.4 M 2024-03-16T09:02:56.792 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 248 k 2024-03-16T09:02:56.792 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 4.7 M 2024-03-16T09:02:56.793 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 17 M 2024-03-16T09:02:56.793 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 16 k 2024-03-16T09:02:56.793 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 25 k 2024-03-16T09:02:56.793 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-16T09:02:56.793 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-16T09:02:56.794 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-16T09:02:56.794 INFO:teuthology.orchestra.run.smithi159.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-16T09:02:56.794 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-16T09:02:56.794 INFO:teuthology.orchestra.run.smithi159.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-16T09:02:56.794 INFO:teuthology.orchestra.run.smithi159.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-16T09:02:56.794 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-16T09:02:56.794 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-16T09:02:56.794 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 163 k 2024-03-16T09:02:56.794 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-16T09:02:56.794 INFO:teuthology.orchestra.run.smithi159.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-16T09:02:56.794 INFO:teuthology.orchestra.run.smithi159.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-16T09:02:56.795 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-16T09:02:56.795 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-16T09:02:56.795 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 501 k 2024-03-16T09:02:56.795 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-16T09:02:56.795 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.1 M 2024-03-16T09:02:56.795 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-16T09:02:56.795 INFO:teuthology.orchestra.run.smithi159.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-16T09:02:56.795 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-16T09:02:56.795 INFO:teuthology.orchestra.run.smithi159.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-16T09:02:56.795 INFO:teuthology.orchestra.run.smithi159.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-16T09:02:56.795 INFO:teuthology.orchestra.run.smithi159.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-16T09:02:56.795 INFO:teuthology.orchestra.run.smithi159.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-16T09:02:56.795 INFO:teuthology.orchestra.run.smithi159.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-16T09:02:56.796 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-16T09:02:56.796 INFO:teuthology.orchestra.run.smithi159.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-16T09:02:56.796 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-16T09:02:56.796 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-16T09:02:56.796 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-16T09:02:56.796 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-16T09:02:56.796 INFO:teuthology.orchestra.run.smithi159.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-16T09:02:56.796 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-16T09:02:56.796 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-16T09:02:56.796 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-16T09:02:56.796 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 45 k 2024-03-16T09:02:56.796 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 131 k 2024-03-16T09:02:56.796 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-16T09:02:56.796 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-16T09:02:56.796 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-16T09:02:56.797 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-16T09:02:56.797 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-16T09:02:56.797 INFO:teuthology.orchestra.run.smithi159.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-16T09:02:56.797 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-16T09:02:56.797 INFO:teuthology.orchestra.run.smithi159.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-16T09:02:56.797 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-16T09:02:56.797 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-16T09:02:56.797 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-16T09:02:56.797 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-16T09:02:56.798 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-16T09:02:56.798 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-16T09:02:56.798 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-16T09:02:56.798 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-16T09:02:56.798 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-16T09:02:56.798 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-16T09:02:56.798 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-16T09:02:56.798 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-16T09:02:56.798 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-16T09:02:56.798 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-16T09:02:56.798 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-16T09:02:56.798 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-16T09:02:56.798 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-16T09:02:56.801 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-16T09:02:56.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-16T09:02:56.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-16T09:02:56.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-16T09:02:56.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-16T09:02:56.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-16T09:02:56.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-16T09:02:56.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-16T09:02:56.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-16T09:02:56.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-16T09:02:56.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-16T09:02:56.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-16T09:02:56.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-16T09:02:56.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-16T09:02:56.802 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-16T09:02:56.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-16T09:02:56.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-16T09:02:56.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-16T09:02:56.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-16T09:02:56.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-16T09:02:56.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-16T09:02:56.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-16T09:02:56.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-16T09:02:56.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-16T09:02:56.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-16T09:02:56.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-16T09:02:56.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-16T09:02:56.803 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-16T09:02:56.804 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-16T09:02:56.804 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-16T09:02:56.804 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-16T09:02:56.804 INFO:teuthology.orchestra.run.smithi159.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-16T09:02:56.804 INFO:teuthology.orchestra.run.smithi159.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-16T09:02:56.804 INFO:teuthology.orchestra.run.smithi159.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-16T09:02:56.804 INFO:teuthology.orchestra.run.smithi159.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-16T09:02:56.804 INFO:teuthology.orchestra.run.smithi159.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-16T09:02:56.804 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-16T09:02:56.804 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-16T09:02:56.804 INFO:teuthology.orchestra.run.smithi159.stdout:Installing weak dependencies: 2024-03-16T09:02:56.804 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-16T09:02:56.804 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-16T09:02:56.804 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-03-16T09:02:56.805 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================ 2024-03-16T09:02:56.805 INFO:teuthology.orchestra.run.smithi159.stdout:Install 150 Packages 2024-03-16T09:02:56.805 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrade 24 Packages 2024-03-16T09:02:56.805 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-16T09:02:56.805 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 224 M 2024-03-16T09:02:56.805 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-03-16T09:02:56.842 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-03-16T09:02:56.842 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-03-16T09:02:57.372 INFO:teuthology.orchestra.run.smithi159.stdout:(1/174): ceph-19.0.0-2162.g3d78f5ba.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-03-16T09:02:57.547 INFO:teuthology.orchestra.run.smithi159.stdout:(2/174): ceph-fuse-19.0.0-2162.g3d78f5ba.el9.x8 5.0 MB/s | 890 kB 00:00 2024-03-16T09:02:57.589 INFO:teuthology.orchestra.run.smithi159.stdout:(3/174): ceph-immutable-object-cache-19.0.0-216 3.4 MB/s | 145 kB 00:00 2024-03-16T09:02:57.617 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-16T09:02:57.707 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-16T09:02:57.738 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-16T09:02:57.769 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-16T09:02:57.796 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-16T09:02:57.822 INFO:teuthology.orchestra.run.smithi159.stdout:(4/174): ceph-mds-19.0.0-2162.g3d78f5ba.el9.x86 10 MB/s | 2.4 MB 00:00 2024-03-16T09:02:57.965 INFO:teuthology.orchestra.run.smithi159.stdout:(5/174): ceph-base-19.0.0-2162.g3d78f5ba.el9.x8 6.9 MB/s | 5.4 MB 00:00 2024-03-16T09:02:57.998 INFO:teuthology.orchestra.run.smithi159.stdout:(6/174): ceph-mgr-19.0.0-2162.g3d78f5ba.el9.x86 8.5 MB/s | 1.5 MB 00:00 2024-03-16T09:02:58.108 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-16T09:02:58.143 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-16T09:02:58.232 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-16T09:02:58.264 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 125/198 2024-03-16T09:02:58.301 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-16T09:02:58.357 INFO:teuthology.orchestra.run.smithi159.stdout:(7/174): ceph-mon-19.0.0-2162.g3d78f5ba.el9.x86 12 MB/s | 4.7 MB 00:00 2024-03-16T09:02:58.762 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-16T09:02:58.796 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-16T09:02:58.850 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-16T09:02:58.872 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 129/198 2024-03-16T09:02:59.240 INFO:teuthology.orchestra.run.smithi159.stdout:(8/174): ceph-radosgw-19.0.0-2162.g3d78f5ba.el9 11 MB/s | 9.8 MB 00:00 2024-03-16T09:02:59.407 INFO:teuthology.orchestra.run.smithi159.stdout:(9/174): ceph-osd-19.0.0-2162.g3d78f5ba.el9.x86 12 MB/s | 17 MB 00:01 2024-03-16T09:02:59.424 INFO:teuthology.orchestra.run.smithi159.stdout:(10/174): ceph-selinux-19.0.0-2162.g3d78f5ba.el 137 kB/s | 25 kB 00:00 2024-03-16T09:02:59.499 INFO:teuthology.orchestra.run.smithi159.stdout:(11/174): cephfs-mirror-19.0.0-2162.g3d78f5ba.e 2.9 MB/s | 221 kB 00:00 2024-03-16T09:02:59.566 INFO:teuthology.orchestra.run.smithi159.stdout:(12/174): cephfs-shell-19.0.0-2162.g3d78f5ba.el 406 kB/s | 27 kB 00:00 2024-03-16T09:02:59.633 INFO:teuthology.orchestra.run.smithi159.stdout:(13/174): libcephfs-devel-19.0.0-2162.g3d78f5ba 475 kB/s | 32 kB 00:00 2024-03-16T09:02:59.742 INFO:teuthology.orchestra.run.smithi159.stdout:(14/174): libcephfs2-19.0.0-2162.g3d78f5ba.el9. 6.7 MB/s | 747 kB 00:00 2024-03-16T09:02:59.834 INFO:teuthology.orchestra.run.smithi159.stdout:(15/174): libcephsqlite-19.0.0-2162.g3d78f5ba.e 1.7 MB/s | 163 kB 00:00 2024-03-16T09:02:59.917 INFO:teuthology.orchestra.run.smithi159.stdout:(16/174): librados-devel-19.0.0-2162.g3d78f5ba. 1.5 MB/s | 127 kB 00:00 2024-03-16T09:03:00.018 INFO:teuthology.orchestra.run.smithi159.stdout:(17/174): libradosstriper1-19.0.0-2162.g3d78f5b 4.9 MB/s | 501 kB 00:00 2024-03-16T09:03:00.518 INFO:teuthology.orchestra.run.smithi159.stdout:(18/174): librgw2-19.0.0-2162.g3d78f5ba.el9.x86 10 MB/s | 5.1 MB 00:00 2024-03-16T09:03:00.593 INFO:teuthology.orchestra.run.smithi159.stdout:(19/174): python3-ceph-argparse-19.0.0-2162.g3d 603 kB/s | 45 kB 00:00 2024-03-16T09:03:00.693 INFO:teuthology.orchestra.run.smithi159.stdout:(20/174): python3-ceph-common-19.0.0-2162.g3d78 1.3 MB/s | 131 kB 00:00 2024-03-16T09:03:00.777 INFO:teuthology.orchestra.run.smithi159.stdout:(21/174): python3-cephfs-19.0.0-2162.g3d78f5ba. 1.9 MB/s | 162 kB 00:00 2024-03-16T09:03:00.878 INFO:teuthology.orchestra.run.smithi159.stdout:(22/174): python3-rados-19.0.0-2162.g3d78f5ba.e 3.1 MB/s | 320 kB 00:00 2024-03-16T09:03:00.978 INFO:teuthology.orchestra.run.smithi159.stdout:(23/174): python3-rbd-19.0.0-2162.g3d78f5ba.el9 2.9 MB/s | 298 kB 00:00 2024-03-16T09:03:01.061 INFO:teuthology.orchestra.run.smithi159.stdout:(24/174): python3-rgw-19.0.0-2162.g3d78f5ba.el9 1.2 MB/s | 99 kB 00:00 2024-03-16T09:03:01.127 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T09:03:01.145 INFO:teuthology.orchestra.run.smithi159.stdout:(25/174): rbd-fuse-19.0.0-2162.g3d78f5ba.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-16T09:03:01.162 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T09:03:01.454 INFO:teuthology.orchestra.run.smithi159.stdout:(26/174): rbd-mirror-19.0.0-2162.g3d78f5ba.el9. 10 MB/s | 3.1 MB 00:00 2024-03-16T09:03:01.537 INFO:teuthology.orchestra.run.smithi159.stdout:(27/174): rbd-nbd-19.0.0-2162.g3d78f5ba.el9.x86 2.0 MB/s | 170 kB 00:00 2024-03-16T09:03:01.613 INFO:teuthology.orchestra.run.smithi159.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2162.g 453 kB/s | 34 kB 00:00 2024-03-16T09:03:01.688 INFO:teuthology.orchestra.run.smithi159.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2162.g3d78f5b 1.8 MB/s | 139 kB 00:00 2024-03-16T09:03:01.703 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T09:03:01.740 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-16T09:03:01.818 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-16T09:03:01.818 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-16T09:03:01.818 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-16T09:03:01.818 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:03:01.849 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-16T09:03:02.339 INFO:teuthology.orchestra.run.smithi159.stdout:(30/174): ceph-test-19.0.0-2162.g3d78f5ba.el9.x 16 MB/s | 48 MB 00:02 2024-03-16T09:03:02.389 INFO:teuthology.orchestra.run.smithi159.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2162.g3d78f 2.6 MB/s | 1.8 MB 00:00 2024-03-16T09:03:02.481 INFO:teuthology.orchestra.run.smithi159.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2162.g3d 2.7 MB/s | 248 kB 00:00 2024-03-16T09:03:02.581 INFO:teuthology.orchestra.run.smithi159.stdout:(33/174): ceph-mgr-rook-19.0.0-2162.g3d78f5ba.e 494 kB/s | 49 kB 00:00 2024-03-16T09:03:02.656 INFO:teuthology.orchestra.run.smithi159.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2162.g3 219 kB/s | 16 kB 00:00 2024-03-16T09:03:02.740 INFO:teuthology.orchestra.run.smithi159.stdout:(35/174): ceph-volume-19.0.0-2162.g3d78f5ba.el9 3.1 MB/s | 264 kB 00:00 2024-03-16T09:03:02.824 INFO:teuthology.orchestra.run.smithi159.stdout:(36/174): ceph-mgr-diskprediction-local-19.0.0- 15 MB/s | 7.4 MB 00:00 2024-03-16T09:03:02.857 INFO:teuthology.orchestra.run.smithi159.stdout:(37/174): cephfs-top-19.0.0-2162.g3d78f5ba.el9. 550 kB/s | 18 kB 00:00 2024-03-16T09:03:02.891 INFO:teuthology.orchestra.run.smithi159.stdout:(38/174): cephadm-19.0.0-2162.g3d78f5ba.el9.noa 4.8 MB/s | 748 kB 00:00 2024-03-16T09:03:03.041 INFO:teuthology.orchestra.run.smithi159.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 662 kB 00:00 2024-03-16T09:03:03.075 INFO:teuthology.orchestra.run.smithi159.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 5.7 MB/s | 920 kB 00:00 2024-03-16T09:03:03.100 INFO:teuthology.orchestra.run.smithi159.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-16T09:03:03.134 INFO:teuthology.orchestra.run.smithi159.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2024-03-16T09:03:03.151 INFO:teuthology.orchestra.run.smithi159.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 635 kB/s | 32 kB 00:00 2024-03-16T09:03:03.243 INFO:teuthology.orchestra.run.smithi159.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-03-16T09:03:03.259 INFO:teuthology.orchestra.run.smithi159.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 160 kB/s | 17 kB 00:00 2024-03-16T09:03:03.278 INFO:teuthology.orchestra.run.smithi159.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 301 kB/s | 10 kB 00:00 2024-03-16T09:03:03.302 INFO:teuthology.orchestra.run.smithi159.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-03-16T09:03:03.319 INFO:teuthology.orchestra.run.smithi159.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-16T09:03:03.344 INFO:teuthology.orchestra.run.smithi159.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 16 MB/s | 661 kB 00:00 2024-03-16T09:03:03.369 INFO:teuthology.orchestra.run.smithi159.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 578 kB 00:00 2024-03-16T09:03:03.385 INFO:teuthology.orchestra.run.smithi159.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.2 MB/s | 46 kB 00:00 2024-03-16T09:03:03.410 INFO:teuthology.orchestra.run.smithi159.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 282 kB 00:00 2024-03-16T09:03:03.435 INFO:teuthology.orchestra.run.smithi159.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-16T09:03:03.461 INFO:teuthology.orchestra.run.smithi159.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-16T09:03:03.478 INFO:teuthology.orchestra.run.smithi159.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 816 kB/s | 34 kB 00:00 2024-03-16T09:03:03.604 INFO:teuthology.orchestra.run.smithi159.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-03-16T09:03:03.645 INFO:teuthology.orchestra.run.smithi159.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 9.2 MB/s | 1.5 MB 00:00 2024-03-16T09:03:03.671 INFO:teuthology.orchestra.run.smithi159.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-03-16T09:03:03.696 INFO:teuthology.orchestra.run.smithi159.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.7 MB/s | 134 kB 00:00 2024-03-16T09:03:03.726 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-03-16T09:03:03.830 INFO:teuthology.orchestra.run.smithi159.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 36 MB/s | 5.7 MB 00:00 2024-03-16T09:03:03.842 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-16T09:03:03.872 INFO:teuthology.orchestra.run.smithi159.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-03-16T09:03:03.884 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-16T09:03:03.897 INFO:teuthology.orchestra.run.smithi159.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 78 kB 00:00 2024-03-16T09:03:03.915 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T09:03:03.924 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T09:03:03.965 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-16T09:03:04.018 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-16T09:03:04.031 INFO:teuthology.orchestra.run.smithi159.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 5.8 MB 00:00 2024-03-16T09:03:04.052 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-16T09:03:04.056 INFO:teuthology.orchestra.run.smithi159.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 156 kB/s | 24 kB 00:00 2024-03-16T09:03:04.081 INFO:teuthology.orchestra.run.smithi159.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 361 kB/s | 18 kB 00:00 2024-03-16T09:03:04.086 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-16T09:03:04.106 INFO:teuthology.orchestra.run.smithi159.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-16T09:03:04.128 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-16T09:03:04.132 INFO:teuthology.orchestra.run.smithi159.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 632 kB/s | 32 kB 00:00 2024-03-16T09:03:04.366 INFO:teuthology.orchestra.run.smithi159.stdout:(68/174): ceph-common-19.0.0-2162.g3d78f5ba.el9 2.9 MB/s | 21 MB 00:07 2024-03-16T09:03:04.441 INFO:teuthology.orchestra.run.smithi159.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 4.8 MB 00:00 2024-03-16T09:03:04.448 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-16T09:03:04.466 INFO:teuthology.orchestra.run.smithi159.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 655 kB/s | 217 kB 00:00 2024-03-16T09:03:04.491 INFO:teuthology.orchestra.run.smithi159.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 174 kB 00:00 2024-03-16T09:03:04.503 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T09:03:04.517 INFO:teuthology.orchestra.run.smithi159.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-16T09:03:04.533 INFO:teuthology.orchestra.run.smithi159.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 618 kB/s | 41 kB 00:00 2024-03-16T09:03:04.559 INFO:teuthology.orchestra.run.smithi159.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-16T09:03:04.601 INFO:teuthology.orchestra.run.smithi159.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-16T09:03:04.624 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T09:03:04.626 INFO:teuthology.orchestra.run.smithi159.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 211 kB 00:00 2024-03-16T09:03:04.643 INFO:teuthology.orchestra.run.smithi159.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 564 kB/s | 47 kB 00:00 2024-03-16T09:03:04.654 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T09:03:04.668 INFO:teuthology.orchestra.run.smithi159.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 640 kB/s | 43 kB 00:00 2024-03-16T09:03:04.745 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T09:03:04.775 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-16T09:03:04.818 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-16T09:03:05.000 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-16T09:03:05.056 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-16T09:03:05.068 INFO:teuthology.orchestra.run.smithi159.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 91 kB/s | 39 kB 00:00 2024-03-16T09:03:05.084 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T09:03:05.093 INFO:teuthology.orchestra.run.smithi159.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 96 kB/s | 41 kB 00:00 2024-03-16T09:03:05.190 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T09:03:05.210 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-16T09:03:05.257 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-16T09:03:05.278 INFO:teuthology.orchestra.run.smithi159.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-03-16T09:03:05.297 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-16T09:03:05.303 INFO:teuthology.orchestra.run.smithi159.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 300 kB 00:00 2024-03-16T09:03:05.320 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T09:03:05.328 INFO:teuthology.orchestra.run.smithi159.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 243 kB/s | 56 kB 00:00 2024-03-16T09:03:05.336 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T09:03:05.353 INFO:teuthology.orchestra.run.smithi159.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 842 kB/s | 63 kB 00:00 2024-03-16T09:03:05.387 INFO:teuthology.orchestra.run.smithi159.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 757 kB/s | 44 kB 00:00 2024-03-16T09:03:05.412 INFO:teuthology.orchestra.run.smithi159.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 943 kB/s | 79 kB 00:00 2024-03-16T09:03:05.437 INFO:teuthology.orchestra.run.smithi159.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.8 MB/s | 241 kB 00:00 2024-03-16T09:03:05.488 INFO:teuthology.orchestra.run.smithi159.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-16T09:03:05.513 INFO:teuthology.orchestra.run.smithi159.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-03-16T09:03:05.528 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T09:03:05.538 INFO:teuthology.orchestra.run.smithi159.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2024-03-16T09:03:05.559 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T09:03:05.563 INFO:teuthology.orchestra.run.smithi159.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-03-16T09:03:05.591 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-16T09:03:05.672 INFO:teuthology.orchestra.run.smithi159.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-16T09:03:05.687 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-16T09:03:05.697 INFO:teuthology.orchestra.run.smithi159.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-03-16T09:03:05.723 INFO:teuthology.orchestra.run.smithi159.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-16T09:03:05.748 INFO:teuthology.orchestra.run.smithi159.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 427 kB/s | 21 kB 00:00 2024-03-16T09:03:05.773 INFO:teuthology.orchestra.run.smithi159.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 506 kB/s | 25 kB 00:00 2024-03-16T09:03:05.793 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T09:03:05.807 INFO:teuthology.orchestra.run.smithi159.stdout:(97/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-03-16T09:03:05.836 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T09:03:05.840 INFO:teuthology.orchestra.run.smithi159.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-03-16T09:03:05.866 INFO:teuthology.orchestra.run.smithi159.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-03-16T09:03:05.891 INFO:teuthology.orchestra.run.smithi159.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 216 kB 00:00 2024-03-16T09:03:05.913 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T09:03:05.916 INFO:teuthology.orchestra.run.smithi159.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 694 kB/s | 35 kB 00:00 2024-03-16T09:03:05.944 INFO:teuthology.orchestra.run.smithi159.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 853 kB/s | 44 kB 00:00 2024-03-16T09:03:05.969 INFO:teuthology.orchestra.run.smithi159.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-16T09:03:05.971 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T09:03:06.061 INFO:teuthology.orchestra.run.smithi159.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 13 MB/s | 4.1 MB 00:00 2024-03-16T09:03:06.192 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T09:03:06.299 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-16T09:03:06.361 INFO:teuthology.orchestra.run.smithi159.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-03-16T09:03:06.386 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-16T09:03:06.434 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-16T09:03:06.436 INFO:teuthology.orchestra.run.smithi159.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 4.1 MB/s | 309 kB 00:00 2024-03-16T09:03:06.461 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-16T09:03:06.486 INFO:teuthology.orchestra.run.smithi159.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 391 kB/s | 19 kB 00:00 2024-03-16T09:03:06.496 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-16T09:03:06.546 INFO:teuthology.orchestra.run.smithi159.stdout:(108/174): lua-devel-5.4.4-4.el9.x86_64.rpm 37 kB/s | 22 kB 00:00 2024-03-16T09:03:06.570 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-16T09:03:06.624 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-16T09:03:06.662 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-16T09:03:06.696 INFO:teuthology.orchestra.run.smithi159.stdout:(109/174): libarrow-9.0.0-10.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-03-16T09:03:06.713 INFO:teuthology.orchestra.run.smithi159.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 156 kB/s | 26 kB 00:00 2024-03-16T09:03:06.726 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-16T09:03:06.746 INFO:teuthology.orchestra.run.smithi159.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-03-16T09:03:06.761 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-16T09:03:06.805 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-16T09:03:06.806 INFO:teuthology.orchestra.run.smithi159.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-03-16T09:03:06.830 INFO:teuthology.orchestra.run.smithi159.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 575 kB/s | 67 kB 00:00 2024-03-16T09:03:06.837 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-16T09:03:06.865 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-16T09:03:06.872 INFO:teuthology.orchestra.run.smithi159.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-03-16T09:03:06.931 INFO:teuthology.orchestra.run.smithi159.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 747 kB/s | 43 kB 00:00 2024-03-16T09:03:06.934 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-16T09:03:06.970 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 39/198 2024-03-16T09:03:06.990 INFO:teuthology.orchestra.run.smithi159.stdout:(116/174): python3-asyncssh-2.13.2-4.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-03-16T09:03:07.013 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 40/198 2024-03-16T09:03:07.014 INFO:teuthology.orchestra.run.smithi159.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 385 kB/s | 32 kB 00:00 2024-03-16T09:03:07.042 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 41/198 2024-03-16T09:03:07.048 INFO:teuthology.orchestra.run.smithi159.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-03-16T09:03:07.073 INFO:teuthology.orchestra.run.smithi159.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-03-16T09:03:07.119 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-16T09:03:07.124 INFO:teuthology.orchestra.run.smithi159.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-03-16T09:03:07.149 INFO:teuthology.orchestra.run.smithi159.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-03-16T09:03:07.165 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-16T09:03:07.183 INFO:teuthology.orchestra.run.smithi159.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 603 kB/s | 35 kB 00:00 2024-03-16T09:03:07.208 INFO:teuthology.orchestra.run.smithi159.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-03-16T09:03:07.238 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-16T09:03:07.267 INFO:teuthology.orchestra.run.smithi159.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-03-16T09:03:07.325 INFO:teuthology.orchestra.run.smithi159.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-03-16T09:03:07.373 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-16T09:03:07.375 INFO:teuthology.orchestra.run.smithi159.stdout:(126/174): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-03-16T09:03:07.401 INFO:teuthology.orchestra.run.smithi159.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-03-16T09:03:07.425 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-16T09:03:07.435 INFO:teuthology.orchestra.run.smithi159.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 395 kB/s | 23 kB 00:00 2024-03-16T09:03:07.460 INFO:teuthology.orchestra.run.smithi159.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-03-16T09:03:07.519 INFO:teuthology.orchestra.run.smithi159.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-03-16T09:03:07.552 INFO:teuthology.orchestra.run.smithi159.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-03-16T09:03:07.569 INFO:teuthology.orchestra.run.smithi159.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 925 kB/s | 46 kB 00:00 2024-03-16T09:03:07.611 INFO:teuthology.orchestra.run.smithi159.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-03-16T09:03:07.628 INFO:teuthology.orchestra.run.smithi159.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 984 kB/s | 58 kB 00:00 2024-03-16T09:03:07.670 INFO:teuthology.orchestra.run.smithi159.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-03-16T09:03:07.696 INFO:teuthology.orchestra.run.smithi159.stdout:(136/174): protobuf-compiler-3.14.0-13.el9.x86_ 500 kB/s | 863 kB 00:01 2024-03-16T09:03:07.712 INFO:teuthology.orchestra.run.smithi159.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 195 kB/s | 16 kB 00:00 2024-03-16T09:03:07.738 INFO:teuthology.orchestra.run.smithi159.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-03-16T09:03:07.763 INFO:teuthology.orchestra.run.smithi159.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 567 kB/s | 28 kB 00:00 2024-03-16T09:03:07.796 INFO:teuthology.orchestra.run.smithi159.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-03-16T09:03:07.822 INFO:teuthology.orchestra.run.smithi159.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-03-16T09:03:07.847 INFO:teuthology.orchestra.run.smithi159.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-03-16T09:03:07.864 INFO:teuthology.orchestra.run.smithi159.stdout:(143/174): python3-repoze-lru-0.7-16.el9.noarch 204 kB/s | 31 kB 00:00 2024-03-16T09:03:07.881 INFO:teuthology.orchestra.run.smithi159.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 865 kB/s | 51 kB 00:00 2024-03-16T09:03:07.906 INFO:teuthology.orchestra.run.smithi159.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-03-16T09:03:07.940 INFO:teuthology.orchestra.run.smithi159.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 7.2 MB/s | 427 kB 00:00 2024-03-16T09:03:07.942 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-16T09:03:07.957 INFO:teuthology.orchestra.run.smithi159.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-03-16T09:03:07.982 INFO:teuthology.orchestra.run.smithi159.stdout:(148/174): python3-websocket-client-1.2.3-2.el9 761 kB/s | 90 kB 00:00 2024-03-16T09:03:08.016 INFO:teuthology.orchestra.run.smithi159.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-03-16T09:03:08.028 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-16T09:03:08.058 INFO:teuthology.orchestra.run.smithi159.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-03-16T09:03:08.099 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-16T09:03:08.100 INFO:teuthology.orchestra.run.smithi159.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-16T09:03:08.149 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T09:03:08.157 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T09:03:08.189 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-16T09:03:08.229 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-16T09:03:08.267 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 53/198 2024-03-16T09:03:08.283 INFO:teuthology.orchestra.run.smithi159.stdout:(152/174): librados2-19.0.0-2162.g3d78f5ba.el9. 11 MB/s | 3.4 MB 00:00 2024-03-16T09:03:08.297 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-16T09:03:08.325 INFO:teuthology.orchestra.run.smithi159.stdout:(153/174): NetworkManager-1.46.0-1.el9.x86_64.r 10 MB/s | 2.3 MB 00:00 2024-03-16T09:03:08.367 INFO:teuthology.orchestra.run.smithi159.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-03-16T09:03:08.409 INFO:teuthology.orchestra.run.smithi159.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-03-16T09:03:08.459 INFO:teuthology.orchestra.run.smithi159.stdout:(156/174): NetworkManager-libnm-1.46.0-1.el9.x8 10 MB/s | 1.8 MB 00:00 2024-03-16T09:03:08.485 INFO:teuthology.orchestra.run.smithi159.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 948 kB/s | 71 kB 00:00 2024-03-16T09:03:08.510 INFO:teuthology.orchestra.run.smithi159.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 725 kB/s | 36 kB 00:00 2024-03-16T09:03:08.527 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-16T09:03:08.535 INFO:teuthology.orchestra.run.smithi159.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.7 MB/s | 192 kB 00:00 2024-03-16T09:03:08.560 INFO:teuthology.orchestra.run.smithi159.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 256 kB 00:00 2024-03-16T09:03:08.586 INFO:teuthology.orchestra.run.smithi159.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 458 kB/s | 23 kB 00:00 2024-03-16T09:03:08.611 INFO:teuthology.orchestra.run.smithi159.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-16T09:03:08.670 INFO:teuthology.orchestra.run.smithi159.stdout:(163/174): librbd1-19.0.0-2162.g3d78f5ba.el9.x8 4.8 MB/s | 3.1 MB 00:00 2024-03-16T09:03:08.685 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-16T09:03:08.695 INFO:teuthology.orchestra.run.smithi159.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 155 kB 00:00 2024-03-16T09:03:08.720 INFO:teuthology.orchestra.run.smithi159.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 136 kB 00:00 2024-03-16T09:03:08.754 INFO:teuthology.orchestra.run.smithi159.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-03-16T09:03:08.757 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-16T09:03:08.779 INFO:teuthology.orchestra.run.smithi159.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 342 kB/s | 29 kB 00:00 2024-03-16T09:03:08.800 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-16T09:03:08.821 INFO:teuthology.orchestra.run.smithi159.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 12 MB/s | 1.2 MB 00:00 2024-03-16T09:03:08.871 INFO:teuthology.orchestra.run.smithi159.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 18 MB/s | 2.2 MB 00:00 2024-03-16T09:03:08.897 INFO:teuthology.orchestra.run.smithi159.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 6.1 MB/s | 465 kB 00:00 2024-03-16T09:03:08.914 INFO:teuthology.orchestra.run.smithi159.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 743 kB/s | 31 kB 00:00 2024-03-16T09:03:08.926 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-16T09:03:08.964 INFO:teuthology.orchestra.run.smithi159.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-03-16T09:03:08.987 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-16T09:03:08.989 INFO:teuthology.orchestra.run.smithi159.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 989 kB/s | 74 kB 00:00 2024-03-16T09:03:09.033 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-16T09:03:09.048 INFO:teuthology.orchestra.run.smithi159.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 8.5 MB/s | 2.3 MB 00:00 2024-03-16T09:03:09.052 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-03-16T09:03:09.052 INFO:teuthology.orchestra.run.smithi159.stdout:Total 18 MB/s | 224 MB 00:12 2024-03-16T09:03:09.097 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-16T09:03:09.129 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-16T09:03:09.165 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-16T09:03:09.212 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-16T09:03:09.249 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-16T09:03:09.291 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-16T09:03:09.334 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-16T09:03:09.467 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-16T09:03:09.506 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-03-16T09:03:09.614 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-16T09:03:09.644 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-16T09:03:09.686 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-16T09:03:09.780 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-03-16T09:03:09.780 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-03-16T09:03:09.781 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-16T09:03:09.819 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-16T09:03:09.865 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-16T09:03:09.891 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-16T09:03:09.916 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-16T09:03:09.987 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-16T09:03:10.107 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-16T09:03:10.215 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-16T09:03:10.256 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-16T09:03:10.305 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-16T09:03:10.360 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-16T09:03:10.562 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-16T09:03:10.610 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-16T09:03:10.747 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-16T09:03:10.783 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-16T09:03:10.815 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-16T09:03:10.846 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-16T09:03:10.874 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-16T09:03:10.917 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-16T09:03:10.956 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-16T09:03:10.987 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-16T09:03:11.017 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-16T09:03:11.094 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-16T09:03:11.127 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-16T09:03:11.161 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-16T09:03:11.202 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-16T09:03:11.236 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-16T09:03:11.326 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-16T09:03:11.406 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-16T09:03:11.451 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-16T09:03:11.482 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-16T09:03:11.513 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-03-16T09:03:11.514 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-03-16T09:03:11.546 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-16T09:03:11.645 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-16T09:03:11.692 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-16T09:03:12.408 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-16T09:03:12.455 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-16T09:03:12.489 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-16T09:03:12.538 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-16T09:03:12.625 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-16T09:03:12.657 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-16T09:03:12.833 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-16T09:03:12.863 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-16T09:03:12.940 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-16T09:03:13.282 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-16T09:03:13.282 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2024-03-16T09:03:13.282 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2024-03-16T09:03:13.282 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2024-03-16T09:03:13.283 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2024-03-16T09:03:13.283 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2024-03-16T09:03:13.283 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /root 2024-03-16T09:03:13.283 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2024-03-16T09:03:13.283 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:03:13.444 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-16T09:03:13.462 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-16T09:03:13.462 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:13.463 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-16T09:03:13.463 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:03:13.568 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-16T09:03:13.676 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-16T09:03:14.872 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-16T09:03:14.938 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-16T09:03:14.954 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-16T09:03:14.963 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-16T09:03:14.963 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:14.963 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-16T09:03:14.963 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T09:03:14.964 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T09:03:14.964 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:03:14.984 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-16T09:03:15.007 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-16T09:03:15.026 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-16T09:03:15.180 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-16T09:03:15.203 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-16T09:03:15.203 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:15.203 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-16T09:03:15.203 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T09:03:15.203 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T09:03:15.203 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:03:15.338 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-16T09:03:15.382 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-16T09:03:15.463 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-16T09:03:15.494 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 125/198 2024-03-16T09:03:15.547 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-16T09:03:15.668 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-16T09:03:15.690 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-16T09:03:15.690 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:15.691 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-16T09:03:15.691 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T09:03:15.691 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T09:03:15.691 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:03:15.783 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-16T09:03:16.006 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-16T09:03:16.034 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-16T09:03:16.105 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-16T09:03:16.128 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 129/198 2024-03-16T09:03:16.141 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-16T09:03:16.164 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-16T09:03:16.196 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 139/198 2024-03-16T09:03:16.255 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 140/198 2024-03-16T09:03:16.279 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-16T09:03:16.512 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-16T09:03:16.546 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-16T09:03:17.514 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-16T09:03:17.538 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-16T09:03:17.639 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-16T09:03:17.795 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 144/198 2024-03-16T09:03:17.824 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-16T09:03:17.847 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-16T09:03:17.847 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:17.847 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-16T09:03:17.847 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T09:03:17.847 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T09:03:17.847 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:03:17.891 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-16T09:03:17.902 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-16T09:03:17.952 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 147/198 2024-03-16T09:03:18.381 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T09:03:18.416 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T09:03:18.469 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-03-16T09:03:18.577 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-16T09:03:18.611 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-16T09:03:18.642 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T09:03:18.651 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T09:03:18.699 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-16T09:03:18.753 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-16T09:03:18.786 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-16T09:03:18.817 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-16T09:03:18.850 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-16T09:03:18.850 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-16T09:03:18.865 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-16T09:03:18.866 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-16T09:03:18.866 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:18.866 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-16T09:03:18.866 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T09:03:18.866 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T09:03:18.866 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:03:18.910 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-16T09:03:18.932 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-16T09:03:18.932 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:18.933 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-16T09:03:18.933 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:03:18.960 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T09:03:18.986 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T09:03:18.995 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-16T09:03:19.008 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T09:03:19.009 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:19.009 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-16T09:03:19.009 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T09:03:19.009 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T09:03:19.009 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:03:19.076 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-16T09:03:19.076 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-16T09:03:19.076 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-16T09:03:19.076 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:03:19.104 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-16T09:03:19.187 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-16T09:03:19.257 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T09:03:19.320 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-16T09:03:19.343 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-16T09:03:19.343 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:19.343 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-16T09:03:19.343 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T09:03:19.343 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T09:03:19.343 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:03:19.387 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T09:03:19.425 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T09:03:19.508 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T09:03:19.546 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-16T09:03:19.583 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-16T09:03:19.763 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-16T09:03:19.827 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-16T09:03:19.856 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T09:03:19.958 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T09:03:19.981 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-16T09:03:20.054 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-16T09:03:20.102 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-16T09:03:20.125 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T09:03:20.140 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T09:03:20.332 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T09:03:20.364 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T09:03:20.404 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-16T09:03:20.507 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-16T09:03:20.623 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T09:03:20.683 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T09:03:20.744 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T09:03:20.860 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T09:03:21.069 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T09:03:21.179 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-16T09:03:21.269 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-16T09:03:21.307 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-16T09:03:21.334 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-16T09:03:21.377 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-16T09:03:21.451 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-16T09:03:21.505 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-16T09:03:21.544 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-16T09:03:21.607 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-16T09:03:21.651 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-16T09:03:21.703 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-16T09:03:21.735 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-16T09:03:21.764 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-16T09:03:21.832 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-16T09:03:21.869 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 39/198 2024-03-16T09:03:21.912 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 40/198 2024-03-16T09:03:21.941 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 41/198 2024-03-16T09:03:22.008 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-16T09:03:22.055 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-16T09:03:22.128 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-16T09:03:22.269 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-16T09:03:22.332 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-16T09:03:22.852 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-16T09:03:22.936 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-16T09:03:23.016 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-16T09:03:23.065 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T09:03:23.073 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T09:03:23.105 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-16T09:03:23.146 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-16T09:03:23.184 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 53/198 2024-03-16T09:03:23.213 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-16T09:03:23.438 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-16T09:03:23.606 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-16T09:03:23.683 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-16T09:03:23.735 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-16T09:03:23.849 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-16T09:03:23.870 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-16T09:03:23.908 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 154/198 2024-03-16T09:03:23.914 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-16T09:03:23.937 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 155/198 2024-03-16T09:03:23.965 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-16T09:03:24.142 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 157/198 2024-03-16T09:03:24.257 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-16T09:03:24.264 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-16T09:03:24.303 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 159/198 2024-03-16T09:03:24.329 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-16T09:03:24.366 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-16T09:03:24.369 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-16T09:03:24.403 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-16T09:03:24.421 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-16T09:03:24.450 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-16T09:03:24.455 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-16T09:03:24.480 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 163/198 2024-03-16T09:03:24.504 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-16T09:03:24.542 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-16T09:03:24.603 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-16T09:03:24.643 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-16T09:03:24.671 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-16T09:03:24.707 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-16T09:03:24.791 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-16T09:03:24.939 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-16T09:03:25.077 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-16T09:03:25.126 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-16T09:03:25.156 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-16T09:03:25.198 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-16T09:03:25.293 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-16T09:03:25.339 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-16T09:03:25.344 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T09:03:25.379 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T09:03:25.380 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-16T09:03:25.399 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T09:03:25.405 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-16T09:03:25.430 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-16T09:03:25.518 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-16T09:03:25.646 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-16T09:03:25.757 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-16T09:03:25.797 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-16T09:03:25.854 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-16T09:03:25.909 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-16T09:03:26.101 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-16T09:03:26.151 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-16T09:03:26.204 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 170/198 2024-03-16T09:03:26.274 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-16T09:03:26.280 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-16T09:03:26.317 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-16T09:03:26.318 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-16T09:03:26.356 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-16T09:03:26.395 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-16T09:03:26.396 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-16T09:03:26.423 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-16T09:03:26.427 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-16T09:03:26.452 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-16T09:03:26.474 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-16T09:03:26.478 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-16T09:03:26.504 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-16T09:03:26.504 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T09:03:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-16T09:03:26.561 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-16T09:03:26.592 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-16T09:03:26.592 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T09:03:26.621 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-16T09:03:26.622 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T09:03:26.668 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-16T09:03:26.702 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-16T09:03:26.714 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T09:03:26.735 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-16T09:03:26.740 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-16T09:03:26.773 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-16T09:03:26.777 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-16T09:03:26.799 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-16T09:03:26.811 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-16T09:03:26.824 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-16T09:03:26.859 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-16T09:03:26.896 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-16T09:03:26.901 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-16T09:03:26.929 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-16T09:03:26.945 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-16T09:03:26.971 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-16T09:03:26.997 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-16T09:03:26.998 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-16T09:03:27.021 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-16T09:03:27.028 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T09:03:27.028 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T09:03:27.051 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-16T09:03:27.082 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-16T09:03:27.129 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-16T09:03:27.207 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T09:03:27.207 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T09:03:27.227 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-16T09:03:27.273 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T09:03:27.283 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-16T09:03:27.298 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-16T09:03:27.324 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-16T09:03:27.348 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-16T09:03:27.375 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-16T09:03:27.440 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T09:03:28.008 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-16T09:03:28.056 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-16T09:03:28.090 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-16T09:03:28.137 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-16T09:03:28.226 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-16T09:03:28.274 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-16T09:03:28.452 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-16T09:03:28.480 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-16T09:03:28.558 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-16T09:03:29.200 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-16T09:03:29.321 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-16T09:03:30.534 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-16T09:03:30.614 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-16T09:03:30.645 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-16T09:03:30.676 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-16T09:03:30.704 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-16T09:03:30.912 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-16T09:03:30.912 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /sys 2024-03-16T09:03:30.912 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /proc 2024-03-16T09:03:30.912 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /mnt 2024-03-16T09:03:30.912 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /var/tmp 2024-03-16T09:03:30.912 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /home 2024-03-16T09:03:30.912 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /root 2024-03-16T09:03:30.912 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /tmp 2024-03-16T09:03:30.912 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:03:31.016 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-16T09:03:31.042 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-16T09:03:31.072 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-16T09:03:31.090 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-16T09:03:31.090 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:31.090 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-16T09:03:31.090 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:03:31.133 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-16T09:03:31.164 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 125/198 2024-03-16T09:03:31.199 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-16T09:03:31.664 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-16T09:03:31.695 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-16T09:03:31.766 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-16T09:03:31.788 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 129/198 2024-03-16T09:03:32.363 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T09:03:32.363 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 1/198 2024-03-16T09:03:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2/198 2024-03-16T09:03:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 3/198 2024-03-16T09:03:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 4/198 2024-03-16T09:03:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 5/198 2024-03-16T09:03:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 6/198 2024-03-16T09:03:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 7/198 2024-03-16T09:03:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 8/198 2024-03-16T09:03:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 9/198 2024-03-16T09:03:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T09:03:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T09:03:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-16T09:03:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 13/198 2024-03-16T09:03:32.364 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 14/198 2024-03-16T09:03:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 15/198 2024-03-16T09:03:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T09:03:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 17/198 2024-03-16T09:03:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 18/198 2024-03-16T09:03:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 19/198 2024-03-16T09:03:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 20/198 2024-03-16T09:03:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 21/198 2024-03-16T09:03:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 22/198 2024-03-16T09:03:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 23/198 2024-03-16T09:03:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 24/198 2024-03-16T09:03:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 25/198 2024-03-16T09:03:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 26/198 2024-03-16T09:03:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 27/198 2024-03-16T09:03:32.365 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 28/198 2024-03-16T09:03:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 29/198 2024-03-16T09:03:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 30/198 2024-03-16T09:03:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 31/198 2024-03-16T09:03:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 32/198 2024-03-16T09:03:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 33/198 2024-03-16T09:03:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 34/198 2024-03-16T09:03:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 35/198 2024-03-16T09:03:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 36/198 2024-03-16T09:03:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 37/198 2024-03-16T09:03:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 38/198 2024-03-16T09:03:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 39/198 2024-03-16T09:03:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-16T09:03:32.366 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-16T09:03:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-16T09:03:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-16T09:03:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-16T09:03:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-16T09:03:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-16T09:03:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-16T09:03:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-16T09:03:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-16T09:03:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-16T09:03:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-16T09:03:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-16T09:03:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-16T09:03:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-16T09:03:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-16T09:03:32.367 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-16T09:03:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-16T09:03:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-16T09:03:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-16T09:03:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-16T09:03:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-16T09:03:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-16T09:03:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-16T09:03:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-16T09:03:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-16T09:03:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-16T09:03:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-16T09:03:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-16T09:03:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-16T09:03:32.368 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-16T09:03:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-16T09:03:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-16T09:03:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-16T09:03:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-16T09:03:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-16T09:03:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-16T09:03:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-16T09:03:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-16T09:03:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-16T09:03:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-16T09:03:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-16T09:03:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-16T09:03:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-16T09:03:32.369 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-16T09:03:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-16T09:03:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-16T09:03:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-16T09:03:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-16T09:03:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-16T09:03:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-16T09:03:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-16T09:03:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-16T09:03:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-16T09:03:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-16T09:03:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-16T09:03:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-16T09:03:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-16T09:03:32.370 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-16T09:03:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-16T09:03:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-16T09:03:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-16T09:03:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-16T09:03:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-16T09:03:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-16T09:03:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-16T09:03:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-16T09:03:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-16T09:03:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-16T09:03:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-16T09:03:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-16T09:03:32.371 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-16T09:03:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-16T09:03:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-16T09:03:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-16T09:03:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-16T09:03:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-16T09:03:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-16T09:03:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-16T09:03:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-16T09:03:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-16T09:03:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-16T09:03:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-16T09:03:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-16T09:03:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-16T09:03:32.372 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-16T09:03:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-16T09:03:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-16T09:03:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-16T09:03:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-16T09:03:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-16T09:03:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-16T09:03:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-16T09:03:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-16T09:03:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-16T09:03:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-16T09:03:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-16T09:03:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-16T09:03:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-16T09:03:32.373 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-16T09:03:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-16T09:03:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-16T09:03:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-16T09:03:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-16T09:03:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-16T09:03:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-16T09:03:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-16T09:03:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-16T09:03:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-16T09:03:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-16T09:03:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-16T09:03:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T09:03:32.374 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-16T09:03:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-16T09:03:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-16T09:03:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-16T09:03:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-16T09:03:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-16T09:03:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-16T09:03:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-16T09:03:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-16T09:03:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-16T09:03:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-16T09:03:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-16T09:03:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-16T09:03:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-16T09:03:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-16T09:03:32.375 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-16T09:03:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-16T09:03:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-16T09:03:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-16T09:03:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-16T09:03:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-16T09:03:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-16T09:03:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-16T09:03:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-16T09:03:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-16T09:03:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-16T09:03:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-16T09:03:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-16T09:03:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-16T09:03:32.376 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-16T09:03:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-16T09:03:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-16T09:03:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-16T09:03:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-16T09:03:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-16T09:03:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-16T09:03:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-16T09:03:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-16T09:03:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-16T09:03:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-16T09:03:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-16T09:03:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-16T09:03:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-16T09:03:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-16T09:03:32.377 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-16T09:03:32.378 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-16T09:03:32.575 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-16T09:03:32.601 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-16T09:03:32.601 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:32.601 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-16T09:03:32.601 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T09:03:32.601 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T09:03:32.601 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:03:32.826 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-16T09:03:32.848 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-16T09:03:32.848 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:32.848 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-16T09:03:32.848 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T09:03:32.849 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T09:03:32.849 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:03:33.293 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-16T09:03:33.315 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-16T09:03:33.315 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:33.315 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-16T09:03:33.315 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T09:03:33.315 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T09:03:33.315 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:03:33.411 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-16T09:03:33.890 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-16T09:03:33.917 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-16T09:03:33.949 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 139/198 2024-03-16T09:03:34.008 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 140/198 2024-03-16T09:03:34.032 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T09:03:34.032 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-16T09:03:34.068 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T09:03:34.265 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-16T09:03:34.300 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-16T09:03:34.604 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-16T09:03:34.639 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-16T09:03:34.719 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-16T09:03:34.719 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-16T09:03:34.719 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-16T09:03:34.719 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-16T09:03:34.748 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-16T09:03:35.261 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-16T09:03:35.291 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-16T09:03:35.392 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-16T09:03:35.554 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 144/198 2024-03-16T09:03:35.593 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-16T09:03:35.617 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-16T09:03:35.617 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:35.617 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-16T09:03:35.617 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T09:03:35.617 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T09:03:35.617 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:03:35.630 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-16T09:03:35.630 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:03:35.631 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2024-03-16T09:03:35.631 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-16T09:03:35.631 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-16T09:03:35.631 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-16T09:03:35.631 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-16T09:03:35.631 INFO:teuthology.orchestra.run.smithi112.stdout: acl-2.3.1-4.el9.x86_64 2024-03-16T09:03:35.631 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-16T09:03:35.631 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-16T09:03:35.631 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-16T09:03:35.631 INFO:teuthology.orchestra.run.smithi112.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-16T09:03:35.631 INFO:teuthology.orchestra.run.smithi112.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-16T09:03:35.631 INFO:teuthology.orchestra.run.smithi112.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-16T09:03:35.631 INFO:teuthology.orchestra.run.smithi112.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-16T09:03:35.631 INFO:teuthology.orchestra.run.smithi112.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-16T09:03:35.632 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.632 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.632 INFO:teuthology.orchestra.run.smithi112.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-16T09:03:35.632 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-16T09:03:35.632 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-16T09:03:35.632 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-16T09:03:35.632 INFO:teuthology.orchestra.run.smithi112.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-16T09:03:35.632 INFO:teuthology.orchestra.run.smithi112.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-16T09:03:35.632 INFO:teuthology.orchestra.run.smithi112.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-16T09:03:35.632 INFO:teuthology.orchestra.run.smithi112.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-16T09:03:35.632 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-16T09:03:35.632 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-03-16T09:03:35.632 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-16T09:03:35.632 INFO:teuthology.orchestra.run.smithi112.stdout: automake-1.16.2-8.el9.noarch 2024-03-16T09:03:35.633 INFO:teuthology.orchestra.run.smithi112.stdout: bison-3.7.4-5.el9.x86_64 2024-03-16T09:03:35.633 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-16T09:03:35.633 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.633 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.633 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.633 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.633 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:35.633 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.633 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.633 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.633 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:35.633 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:35.633 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:35.634 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:35.634 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:35.634 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.634 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.634 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:35.634 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.634 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.634 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.634 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:35.634 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:35.634 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.634 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.635 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:35.635 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-16T09:03:35.635 INFO:teuthology.orchestra.run.smithi112.stdout: flex-2.6.4-9.el9.x86_64 2024-03-16T09:03:35.635 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-16T09:03:35.635 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-16T09:03:35.635 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-16T09:03:35.635 INFO:teuthology.orchestra.run.smithi112.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-16T09:03:35.635 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-16T09:03:35.635 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-16T09:03:35.635 INFO:teuthology.orchestra.run.smithi112.stdout: jq-1.6-16.el9.x86_64 2024-03-16T09:03:35.635 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-16T09:03:35.635 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-16T09:03:35.635 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-16T09:03:35.635 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.636 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.636 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.636 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-16T09:03:35.636 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-16T09:03:35.636 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-16T09:03:35.636 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-16T09:03:35.636 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-16T09:03:35.636 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.636 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.636 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-16T09:03:35.636 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.636 INFO:teuthology.orchestra.run.smithi112.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-16T09:03:35.636 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-16T09:03:35.636 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-16T09:03:35.637 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-16T09:03:35.637 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-16T09:03:35.637 INFO:teuthology.orchestra.run.smithi112.stdout: lua-5.4.4-4.el9.x86_64 2024-03-16T09:03:35.637 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-16T09:03:35.637 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-16T09:03:35.637 INFO:teuthology.orchestra.run.smithi112.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-16T09:03:35.637 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-16T09:03:35.637 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-16T09:03:35.637 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-16T09:03:35.637 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-16T09:03:35.637 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-13.11-1.el9.x86_64 2024-03-16T09:03:35.637 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-16T09:03:35.637 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-16T09:03:35.637 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-16T09:03:35.637 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-16T09:03:35.638 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-16T09:03:35.638 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-16T09:03:35.638 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-16T09:03:35.638 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-16T09:03:35.638 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-16T09:03:35.638 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-16T09:03:35.638 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.638 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.638 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.638 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-16T09:03:35.638 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-16T09:03:35.638 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-16T09:03:35.638 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-16T09:03:35.639 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-16T09:03:35.639 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-16T09:03:35.639 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-16T09:03:35.639 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-16T09:03:35.639 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-16T09:03:35.639 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-16T09:03:35.639 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-16T09:03:35.639 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-16T09:03:35.639 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-16T09:03:35.639 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-16T09:03:35.639 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-16T09:03:35.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-16T09:03:35.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-16T09:03:35.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-16T09:03:35.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-16T09:03:35.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-16T09:03:35.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-16T09:03:35.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-16T09:03:35.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-16T09:03:35.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-16T09:03:35.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-16T09:03:35.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-16T09:03:35.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-16T09:03:35.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-16T09:03:35.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-16T09:03:35.640 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-16T09:03:35.641 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply-3.11-14.el9.noarch 2024-03-16T09:03:35.641 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-16T09:03:35.641 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-16T09:03:35.641 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-16T09:03:35.641 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-16T09:03:35.641 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-16T09:03:35.641 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-16T09:03:35.641 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-16T09:03:35.641 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-16T09:03:35.641 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-16T09:03:35.641 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.641 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.641 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-16T09:03:35.642 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-16T09:03:35.642 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-16T09:03:35.642 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.642 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-16T09:03:35.642 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-16T09:03:35.642 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-16T09:03:35.642 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-16T09:03:35.642 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-16T09:03:35.642 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-16T09:03:35.642 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-16T09:03:35.642 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-16T09:03:35.642 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-16T09:03:35.642 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-16T09:03:35.643 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-16T09:03:35.643 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-16T09:03:35.643 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-16T09:03:35.643 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-16T09:03:35.643 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-16T09:03:35.643 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.643 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.643 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:35.643 INFO:teuthology.orchestra.run.smithi112.stdout: re2-1:20211101-3.el9.x86_64 2024-03-16T09:03:35.643 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-16T09:03:35.643 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-16T09:03:35.643 INFO:teuthology.orchestra.run.smithi112.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-16T09:03:35.643 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-16T09:03:35.644 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-16T09:03:35.644 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-16T09:03:35.644 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:03:35.644 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-03-16T09:03:35.661 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-16T09:03:35.671 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-16T09:03:35.716 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 147/198 2024-03-16T09:03:36.087 DEBUG:teuthology.parallel:result is None 2024-03-16T09:03:36.587 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-16T09:03:36.613 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-16T09:03:36.635 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-16T09:03:36.636 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:36.636 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-16T09:03:36.636 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T09:03:36.636 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T09:03:36.636 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:03:36.680 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-16T09:03:36.702 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-16T09:03:36.702 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:36.702 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-16T09:03:36.702 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:03:36.747 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T09:03:36.769 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T09:03:36.770 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:36.770 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-16T09:03:36.770 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T09:03:36.770 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T09:03:36.770 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:03:37.090 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-16T09:03:37.113 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-16T09:03:37.113 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:37.113 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-16T09:03:37.113 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T09:03:37.113 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T09:03:37.113 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:03:41.552 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-16T09:03:41.595 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 154/198 2024-03-16T09:03:41.632 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 155/198 2024-03-16T09:03:41.659 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-16T09:03:41.838 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 157/198 2024-03-16T09:03:41.952 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-16T09:03:42.001 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 159/198 2024-03-16T09:03:42.040 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-16T09:03:42.092 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-16T09:03:42.127 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-16T09:03:42.160 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 163/198 2024-03-16T09:03:42.200 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-16T09:03:42.230 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-16T09:03:42.284 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-16T09:03:42.345 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-16T09:03:42.378 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-16T09:03:42.928 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T09:03:42.967 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T09:03:42.987 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T09:03:43.806 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 170/198 2024-03-16T09:03:43.887 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-16T09:03:43.927 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-16T09:03:44.008 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-16T09:03:44.048 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-16T09:03:44.073 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-16T09:03:44.098 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-16T09:03:44.125 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-16T09:03:44.125 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T09:03:44.213 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T09:03:44.242 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-16T09:03:44.243 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T09:03:44.327 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T09:03:44.353 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-16T09:03:44.395 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-16T09:03:44.420 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-16T09:03:44.445 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-16T09:03:44.478 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-16T09:03:44.517 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-16T09:03:44.550 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-16T09:03:44.575 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-16T09:03:44.600 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-16T09:03:44.625 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-16T09:03:44.651 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-16T09:03:44.657 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T09:03:44.657 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T09:03:44.837 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T09:03:44.837 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T09:03:44.928 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T09:03:44.954 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-16T09:03:44.978 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-16T09:03:45.011 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-16T09:03:45.046 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-16T09:03:45.109 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T09:03:46.444 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-16T09:03:46.444 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /sys 2024-03-16T09:03:46.445 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /proc 2024-03-16T09:03:46.445 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /mnt 2024-03-16T09:03:46.445 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /var/tmp 2024-03-16T09:03:46.445 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /home 2024-03-16T09:03:46.445 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /root 2024-03-16T09:03:46.445 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /tmp 2024-03-16T09:03:46.445 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-16T09:03:46.619 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-16T09:03:46.633 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-16T09:03:46.633 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:46.633 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-16T09:03:46.634 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-16T09:03:48.131 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-16T09:03:48.156 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-16T09:03:48.157 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:48.157 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-16T09:03:48.157 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T09:03:48.157 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T09:03:48.157 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-16T09:03:48.382 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-16T09:03:48.404 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-16T09:03:48.404 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:48.404 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-16T09:03:48.404 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T09:03:48.404 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T09:03:48.405 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-16T09:03:48.841 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-16T09:03:48.864 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-16T09:03:48.864 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:48.864 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-16T09:03:48.864 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T09:03:48.864 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T09:03:48.864 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-16T09:03:48.968 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-16T09:03:49.368 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-16T09:03:49.399 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-16T09:03:49.431 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 139/198 2024-03-16T09:03:49.491 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 140/198 2024-03-16T09:03:49.514 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-16T09:03:49.747 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-16T09:03:49.781 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-16T09:03:49.909 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T09:03:49.910 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 1/198 2024-03-16T09:03:49.910 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2/198 2024-03-16T09:03:49.910 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 3/198 2024-03-16T09:03:49.910 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 4/198 2024-03-16T09:03:49.910 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 5/198 2024-03-16T09:03:49.910 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 6/198 2024-03-16T09:03:49.910 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 7/198 2024-03-16T09:03:49.910 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 8/198 2024-03-16T09:03:49.910 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 9/198 2024-03-16T09:03:49.910 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T09:03:49.910 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T09:03:49.911 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-16T09:03:49.911 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 13/198 2024-03-16T09:03:49.911 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 14/198 2024-03-16T09:03:49.911 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 15/198 2024-03-16T09:03:49.911 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T09:03:49.911 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 17/198 2024-03-16T09:03:49.911 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 18/198 2024-03-16T09:03:49.911 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 19/198 2024-03-16T09:03:49.911 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 20/198 2024-03-16T09:03:49.911 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 21/198 2024-03-16T09:03:49.911 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 22/198 2024-03-16T09:03:49.911 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 23/198 2024-03-16T09:03:49.911 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 24/198 2024-03-16T09:03:49.911 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 25/198 2024-03-16T09:03:49.912 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 26/198 2024-03-16T09:03:49.912 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 27/198 2024-03-16T09:03:49.912 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 28/198 2024-03-16T09:03:49.912 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 29/198 2024-03-16T09:03:49.912 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 30/198 2024-03-16T09:03:49.912 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 31/198 2024-03-16T09:03:49.912 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 32/198 2024-03-16T09:03:49.912 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 33/198 2024-03-16T09:03:49.912 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 34/198 2024-03-16T09:03:49.912 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 35/198 2024-03-16T09:03:49.912 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 36/198 2024-03-16T09:03:49.912 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 37/198 2024-03-16T09:03:49.912 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 38/198 2024-03-16T09:03:49.912 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 39/198 2024-03-16T09:03:49.912 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-16T09:03:49.913 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-16T09:03:49.913 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-16T09:03:49.913 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-16T09:03:49.913 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-16T09:03:49.913 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-16T09:03:49.913 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-16T09:03:49.913 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-16T09:03:49.913 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-16T09:03:49.913 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-16T09:03:49.913 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-16T09:03:49.913 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-16T09:03:49.913 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-16T09:03:49.913 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-16T09:03:49.914 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-16T09:03:49.914 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-16T09:03:49.914 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-16T09:03:49.914 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-16T09:03:49.914 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-16T09:03:49.914 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-16T09:03:49.914 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-16T09:03:49.914 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-16T09:03:49.914 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-16T09:03:49.914 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-16T09:03:49.914 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-16T09:03:49.914 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-16T09:03:49.914 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-16T09:03:49.915 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-16T09:03:49.915 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-16T09:03:49.915 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-16T09:03:49.915 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-16T09:03:49.915 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-16T09:03:49.915 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-16T09:03:49.915 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-16T09:03:49.915 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-16T09:03:49.915 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-16T09:03:49.915 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-16T09:03:49.915 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-16T09:03:49.915 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-16T09:03:49.915 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-16T09:03:49.915 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-16T09:03:49.915 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-16T09:03:49.916 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-16T09:03:49.916 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-16T09:03:49.916 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-16T09:03:49.916 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-16T09:03:49.916 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-16T09:03:49.916 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-16T09:03:49.916 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-16T09:03:49.916 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-16T09:03:49.916 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-16T09:03:49.916 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-16T09:03:49.916 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-16T09:03:49.916 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-16T09:03:49.916 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-16T09:03:49.916 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-16T09:03:49.917 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-16T09:03:49.917 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-16T09:03:49.917 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-16T09:03:49.917 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-16T09:03:49.917 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-16T09:03:49.917 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-16T09:03:49.917 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-16T09:03:49.917 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-16T09:03:49.917 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-16T09:03:49.917 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-16T09:03:49.917 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-16T09:03:49.917 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-16T09:03:49.917 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-16T09:03:49.918 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-16T09:03:49.918 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-16T09:03:49.918 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-16T09:03:49.918 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-16T09:03:49.918 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-16T09:03:49.918 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-16T09:03:49.918 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-16T09:03:49.918 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-16T09:03:49.918 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-16T09:03:49.918 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-16T09:03:49.918 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-16T09:03:49.918 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-16T09:03:49.918 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-16T09:03:49.919 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-16T09:03:49.919 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-16T09:03:49.919 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-16T09:03:49.919 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-16T09:03:49.919 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-16T09:03:49.919 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-16T09:03:49.919 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-16T09:03:49.919 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-16T09:03:49.919 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-16T09:03:49.919 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-16T09:03:49.919 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-16T09:03:49.919 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-16T09:03:49.919 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-16T09:03:49.919 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-16T09:03:49.920 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-16T09:03:49.920 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-16T09:03:49.920 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-16T09:03:49.920 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-16T09:03:49.920 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-16T09:03:49.920 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-16T09:03:49.920 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-16T09:03:49.920 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-16T09:03:49.920 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-16T09:03:49.920 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-16T09:03:49.920 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-16T09:03:49.920 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-16T09:03:49.920 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-16T09:03:49.920 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-16T09:03:49.921 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-16T09:03:49.921 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T09:03:49.921 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-16T09:03:49.921 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-16T09:03:49.921 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-16T09:03:49.921 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-16T09:03:49.921 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-16T09:03:49.921 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-16T09:03:49.921 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-16T09:03:49.921 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-16T09:03:49.921 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-16T09:03:49.921 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-16T09:03:49.921 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-16T09:03:49.921 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-16T09:03:49.922 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-16T09:03:49.922 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-16T09:03:49.922 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-16T09:03:49.922 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-16T09:03:49.922 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-16T09:03:49.922 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-16T09:03:49.922 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-16T09:03:49.922 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-16T09:03:49.922 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-16T09:03:49.922 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-16T09:03:49.922 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-16T09:03:49.922 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-16T09:03:49.923 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-16T09:03:49.923 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-16T09:03:49.923 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-16T09:03:49.923 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-16T09:03:49.923 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-16T09:03:49.923 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-16T09:03:49.923 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-16T09:03:49.923 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-16T09:03:49.923 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-16T09:03:49.923 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-16T09:03:49.923 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-16T09:03:49.923 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-16T09:03:49.923 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-16T09:03:49.923 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-16T09:03:49.924 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-16T09:03:49.924 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-16T09:03:49.924 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-16T09:03:49.924 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-16T09:03:49.924 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-16T09:03:49.924 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-16T09:03:49.924 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-16T09:03:49.924 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-16T09:03:50.744 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-16T09:03:50.774 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-16T09:03:50.873 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-16T09:03:51.053 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 144/198 2024-03-16T09:03:51.092 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-16T09:03:51.114 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-16T09:03:51.115 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:51.115 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-16T09:03:51.115 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T09:03:51.115 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T09:03:51.115 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-16T09:03:51.160 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-16T09:03:51.170 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-16T09:03:51.216 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 147/198 2024-03-16T09:03:52.084 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-16T09:03:52.112 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-16T09:03:52.134 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-16T09:03:52.134 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:52.135 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-16T09:03:52.135 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T09:03:52.135 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T09:03:52.135 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-16T09:03:52.179 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-16T09:03:52.201 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-16T09:03:52.201 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:52.201 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-16T09:03:52.201 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-16T09:03:52.255 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T09:03:52.277 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T09:03:52.277 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:52.277 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-16T09:03:52.277 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T09:03:52.278 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T09:03:52.278 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-16T09:03:52.581 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-16T09:03:52.603 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-16T09:03:52.603 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T09:03:52.604 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-16T09:03:52.604 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T09:03:52.604 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T09:03:52.604 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-16T09:03:52.886 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-16T09:03:52.887 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:03:52.887 INFO:teuthology.orchestra.run.smithi194.stdout:Upgraded: 2024-03-16T09:03:52.887 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-16T09:03:52.887 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-16T09:03:52.887 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-16T09:03:52.887 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-16T09:03:52.887 INFO:teuthology.orchestra.run.smithi194.stdout: acl-2.3.1-4.el9.x86_64 2024-03-16T09:03:52.887 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-16T09:03:52.887 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-16T09:03:52.887 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-16T09:03:52.887 INFO:teuthology.orchestra.run.smithi194.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-16T09:03:52.888 INFO:teuthology.orchestra.run.smithi194.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-16T09:03:52.888 INFO:teuthology.orchestra.run.smithi194.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-16T09:03:52.888 INFO:teuthology.orchestra.run.smithi194.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-16T09:03:52.888 INFO:teuthology.orchestra.run.smithi194.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-16T09:03:52.888 INFO:teuthology.orchestra.run.smithi194.stdout: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.888 INFO:teuthology.orchestra.run.smithi194.stdout: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.888 INFO:teuthology.orchestra.run.smithi194.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-16T09:03:52.888 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-16T09:03:52.888 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-16T09:03:52.888 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-16T09:03:52.888 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-16T09:03:52.889 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-16T09:03:52.889 INFO:teuthology.orchestra.run.smithi194.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-16T09:03:52.889 INFO:teuthology.orchestra.run.smithi194.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-16T09:03:52.889 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-16T09:03:52.889 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-03-16T09:03:52.889 INFO:teuthology.orchestra.run.smithi194.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-16T09:03:52.889 INFO:teuthology.orchestra.run.smithi194.stdout: automake-1.16.2-8.el9.noarch 2024-03-16T09:03:52.889 INFO:teuthology.orchestra.run.smithi194.stdout: bison-3.7.4-5.el9.x86_64 2024-03-16T09:03:52.889 INFO:teuthology.orchestra.run.smithi194.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-16T09:03:52.889 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.889 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.890 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.890 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.890 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:52.890 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.890 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.890 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.890 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:52.890 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:52.890 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:52.890 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:52.890 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:52.891 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.891 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.891 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:52.891 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.891 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.891 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.891 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:52.891 INFO:teuthology.orchestra.run.smithi194.stdout: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:52.891 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.891 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.891 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:03:52.891 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-16T09:03:52.892 INFO:teuthology.orchestra.run.smithi194.stdout: flex-2.6.4-9.el9.x86_64 2024-03-16T09:03:52.892 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-16T09:03:52.892 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-16T09:03:52.892 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-16T09:03:52.892 INFO:teuthology.orchestra.run.smithi194.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-16T09:03:52.892 INFO:teuthology.orchestra.run.smithi194.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-16T09:03:52.892 INFO:teuthology.orchestra.run.smithi194.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-16T09:03:52.892 INFO:teuthology.orchestra.run.smithi194.stdout: jq-1.6-16.el9.x86_64 2024-03-16T09:03:52.892 INFO:teuthology.orchestra.run.smithi194.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-16T09:03:52.892 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-16T09:03:52.892 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-16T09:03:52.892 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.892 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.893 INFO:teuthology.orchestra.run.smithi194.stdout: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.893 INFO:teuthology.orchestra.run.smithi194.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-16T09:03:52.893 INFO:teuthology.orchestra.run.smithi194.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-16T09:03:52.893 INFO:teuthology.orchestra.run.smithi194.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-16T09:03:52.893 INFO:teuthology.orchestra.run.smithi194.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-16T09:03:52.893 INFO:teuthology.orchestra.run.smithi194.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-16T09:03:52.893 INFO:teuthology.orchestra.run.smithi194.stdout: librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.893 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.893 INFO:teuthology.orchestra.run.smithi194.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-16T09:03:52.893 INFO:teuthology.orchestra.run.smithi194.stdout: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.893 INFO:teuthology.orchestra.run.smithi194.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-16T09:03:52.893 INFO:teuthology.orchestra.run.smithi194.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-16T09:03:52.894 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-16T09:03:52.894 INFO:teuthology.orchestra.run.smithi194.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-16T09:03:52.894 INFO:teuthology.orchestra.run.smithi194.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-16T09:03:52.894 INFO:teuthology.orchestra.run.smithi194.stdout: lua-5.4.4-4.el9.x86_64 2024-03-16T09:03:52.894 INFO:teuthology.orchestra.run.smithi194.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-16T09:03:52.894 INFO:teuthology.orchestra.run.smithi194.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-16T09:03:52.894 INFO:teuthology.orchestra.run.smithi194.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-16T09:03:52.894 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-16T09:03:52.894 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-16T09:03:52.894 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-16T09:03:52.894 INFO:teuthology.orchestra.run.smithi194.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-16T09:03:52.895 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-13.11-1.el9.x86_64 2024-03-16T09:03:52.895 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-16T09:03:52.895 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-16T09:03:52.895 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-16T09:03:52.895 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-16T09:03:52.895 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-16T09:03:52.895 INFO:teuthology.orchestra.run.smithi194.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-16T09:03:52.895 INFO:teuthology.orchestra.run.smithi194.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-16T09:03:52.895 INFO:teuthology.orchestra.run.smithi194.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-16T09:03:52.895 INFO:teuthology.orchestra.run.smithi194.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-16T09:03:52.895 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-16T09:03:52.896 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.896 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.896 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.896 INFO:teuthology.orchestra.run.smithi194.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-16T09:03:52.896 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-16T09:03:52.896 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-16T09:03:52.896 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-16T09:03:52.896 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-16T09:03:52.896 INFO:teuthology.orchestra.run.smithi194.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-16T09:03:52.896 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-16T09:03:52.896 INFO:teuthology.orchestra.run.smithi194.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-16T09:03:52.897 INFO:teuthology.orchestra.run.smithi194.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-16T09:03:52.897 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-16T09:03:52.897 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-16T09:03:52.897 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-16T09:03:52.897 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-16T09:03:52.897 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-16T09:03:52.897 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-16T09:03:52.897 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-16T09:03:52.897 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-16T09:03:52.897 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-16T09:03:52.897 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-16T09:03:52.897 INFO:teuthology.orchestra.run.smithi194.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-16T09:03:52.898 INFO:teuthology.orchestra.run.smithi194.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-16T09:03:52.898 INFO:teuthology.orchestra.run.smithi194.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-16T09:03:52.898 INFO:teuthology.orchestra.run.smithi194.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-16T09:03:52.898 INFO:teuthology.orchestra.run.smithi194.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-16T09:03:52.898 INFO:teuthology.orchestra.run.smithi194.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-16T09:03:52.898 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-16T09:03:52.898 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-16T09:03:52.898 INFO:teuthology.orchestra.run.smithi194.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-16T09:03:52.898 INFO:teuthology.orchestra.run.smithi194.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-16T09:03:52.898 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-16T09:03:52.898 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ply-3.11-14.el9.noarch 2024-03-16T09:03:52.898 INFO:teuthology.orchestra.run.smithi194.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-16T09:03:52.899 INFO:teuthology.orchestra.run.smithi194.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-16T09:03:52.899 INFO:teuthology.orchestra.run.smithi194.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-16T09:03:52.899 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-16T09:03:52.899 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-16T09:03:52.899 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-16T09:03:52.899 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-16T09:03:52.899 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-16T09:03:52.899 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-16T09:03:52.899 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.899 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.899 INFO:teuthology.orchestra.run.smithi194.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-16T09:03:52.899 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-16T09:03:52.900 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-16T09:03:52.900 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.900 INFO:teuthology.orchestra.run.smithi194.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-16T09:03:52.900 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-16T09:03:52.900 INFO:teuthology.orchestra.run.smithi194.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-16T09:03:52.900 INFO:teuthology.orchestra.run.smithi194.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-16T09:03:52.900 INFO:teuthology.orchestra.run.smithi194.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-16T09:03:52.900 INFO:teuthology.orchestra.run.smithi194.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-16T09:03:52.900 INFO:teuthology.orchestra.run.smithi194.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-16T09:03:52.900 INFO:teuthology.orchestra.run.smithi194.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-16T09:03:52.900 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-16T09:03:52.900 INFO:teuthology.orchestra.run.smithi194.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-16T09:03:52.901 INFO:teuthology.orchestra.run.smithi194.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-16T09:03:52.901 INFO:teuthology.orchestra.run.smithi194.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-16T09:03:52.901 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-16T09:03:52.901 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-16T09:03:52.901 INFO:teuthology.orchestra.run.smithi194.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-16T09:03:52.901 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.901 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.901 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:03:52.901 INFO:teuthology.orchestra.run.smithi194.stdout: re2-1:20211101-3.el9.x86_64 2024-03-16T09:03:52.901 INFO:teuthology.orchestra.run.smithi194.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-16T09:03:52.901 INFO:teuthology.orchestra.run.smithi194.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-16T09:03:52.901 INFO:teuthology.orchestra.run.smithi194.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-16T09:03:52.902 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-16T09:03:52.902 INFO:teuthology.orchestra.run.smithi194.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-16T09:03:52.902 INFO:teuthology.orchestra.run.smithi194.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-16T09:03:52.902 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:03:52.902 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-03-16T09:03:53.373 DEBUG:teuthology.parallel:result is None 2024-03-16T09:03:57.043 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-16T09:03:57.085 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 154/198 2024-03-16T09:03:57.115 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 155/198 2024-03-16T09:03:57.142 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-16T09:03:57.323 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 157/198 2024-03-16T09:03:57.443 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-16T09:03:57.509 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 159/198 2024-03-16T09:03:57.556 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-16T09:03:57.617 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-16T09:03:57.660 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-16T09:03:57.685 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 163/198 2024-03-16T09:03:57.734 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-16T09:03:57.764 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-16T09:03:57.826 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-16T09:03:57.898 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-16T09:03:57.937 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-16T09:03:58.557 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T09:03:58.594 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T09:03:58.613 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T09:03:59.431 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 170/198 2024-03-16T09:03:59.506 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-16T09:03:59.546 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-16T09:03:59.626 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-16T09:03:59.676 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-16T09:03:59.701 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-16T09:03:59.726 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-16T09:03:59.753 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-16T09:03:59.753 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T09:03:59.824 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T09:03:59.862 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-16T09:03:59.862 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T09:03:59.947 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T09:03:59.973 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-16T09:04:00.006 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-16T09:04:00.032 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-16T09:04:00.056 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-16T09:04:00.089 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-16T09:04:00.129 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-16T09:04:00.162 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-16T09:04:00.187 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-16T09:04:00.212 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-16T09:04:00.238 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-16T09:04:00.271 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-16T09:04:00.277 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T09:04:00.277 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T09:04:00.466 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T09:04:00.466 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T09:04:00.541 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T09:04:00.565 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-16T09:04:00.590 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-16T09:04:00.623 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-16T09:04:00.649 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-16T09:04:00.713 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T09:04:05.596 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T09:04:05.597 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 1/198 2024-03-16T09:04:05.597 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2/198 2024-03-16T09:04:05.597 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 3/198 2024-03-16T09:04:05.597 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 4/198 2024-03-16T09:04:05.597 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 5/198 2024-03-16T09:04:05.597 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 6/198 2024-03-16T09:04:05.597 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 7/198 2024-03-16T09:04:05.597 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 8/198 2024-03-16T09:04:05.597 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 9/198 2024-03-16T09:04:05.597 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-16T09:04:05.597 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-16T09:04:05.597 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-16T09:04:05.598 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 13/198 2024-03-16T09:04:05.598 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 14/198 2024-03-16T09:04:05.598 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 15/198 2024-03-16T09:04:05.598 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-16T09:04:05.598 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 17/198 2024-03-16T09:04:05.598 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 18/198 2024-03-16T09:04:05.598 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 19/198 2024-03-16T09:04:05.598 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 20/198 2024-03-16T09:04:05.598 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 21/198 2024-03-16T09:04:05.598 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 22/198 2024-03-16T09:04:05.598 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 23/198 2024-03-16T09:04:05.598 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 24/198 2024-03-16T09:04:05.598 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 25/198 2024-03-16T09:04:05.599 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 26/198 2024-03-16T09:04:05.599 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 27/198 2024-03-16T09:04:05.599 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 28/198 2024-03-16T09:04:05.599 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 29/198 2024-03-16T09:04:05.599 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 30/198 2024-03-16T09:04:05.599 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 31/198 2024-03-16T09:04:05.599 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 32/198 2024-03-16T09:04:05.599 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 33/198 2024-03-16T09:04:05.599 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 34/198 2024-03-16T09:04:05.599 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 35/198 2024-03-16T09:04:05.599 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 36/198 2024-03-16T09:04:05.599 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 37/198 2024-03-16T09:04:05.599 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 38/198 2024-03-16T09:04:05.600 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 39/198 2024-03-16T09:04:05.600 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-16T09:04:05.600 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-16T09:04:05.600 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-16T09:04:05.600 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-16T09:04:05.600 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-16T09:04:05.600 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-16T09:04:05.600 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-16T09:04:05.600 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-16T09:04:05.600 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-16T09:04:05.600 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-16T09:04:05.600 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-16T09:04:05.600 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-16T09:04:05.600 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-16T09:04:05.601 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-16T09:04:05.601 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-16T09:04:05.601 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-16T09:04:05.601 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-16T09:04:05.601 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-16T09:04:05.601 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-16T09:04:05.601 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-16T09:04:05.601 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-16T09:04:05.601 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-16T09:04:05.601 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-16T09:04:05.601 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-16T09:04:05.601 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-16T09:04:05.601 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-16T09:04:05.602 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-16T09:04:05.602 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-16T09:04:05.602 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-16T09:04:05.602 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-16T09:04:05.602 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-16T09:04:05.602 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-16T09:04:05.602 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-16T09:04:05.602 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-16T09:04:05.602 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-16T09:04:05.602 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-16T09:04:05.602 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-16T09:04:05.602 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-16T09:04:05.602 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-16T09:04:05.602 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-16T09:04:05.603 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-16T09:04:05.603 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-16T09:04:05.603 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-16T09:04:05.603 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-16T09:04:05.603 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-16T09:04:05.603 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-16T09:04:05.603 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-16T09:04:05.603 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-16T09:04:05.603 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-16T09:04:05.603 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-16T09:04:05.603 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-16T09:04:05.603 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-16T09:04:05.604 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-16T09:04:05.604 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-16T09:04:05.604 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-16T09:04:05.604 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-16T09:04:05.604 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-16T09:04:05.604 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-16T09:04:05.604 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-16T09:04:05.604 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-16T09:04:05.604 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-16T09:04:05.604 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-16T09:04:05.604 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-16T09:04:05.604 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-16T09:04:05.604 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-16T09:04:05.604 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-16T09:04:05.605 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-16T09:04:05.605 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-16T09:04:05.605 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-16T09:04:05.605 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-16T09:04:05.605 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-16T09:04:05.605 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-16T09:04:05.605 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-16T09:04:05.605 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-16T09:04:05.605 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-16T09:04:05.605 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-16T09:04:05.605 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-16T09:04:05.605 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-16T09:04:05.605 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-16T09:04:05.606 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-16T09:04:05.606 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-16T09:04:05.606 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-16T09:04:05.606 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-16T09:04:05.606 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-16T09:04:05.606 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-16T09:04:05.606 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-16T09:04:05.606 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-16T09:04:05.606 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-16T09:04:05.606 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-16T09:04:05.606 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-16T09:04:05.606 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-16T09:04:05.606 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-16T09:04:05.607 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-16T09:04:05.607 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-16T09:04:05.607 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-16T09:04:05.607 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-16T09:04:05.607 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-16T09:04:05.607 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-16T09:04:05.607 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-16T09:04:05.607 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-16T09:04:05.607 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-16T09:04:05.607 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-16T09:04:05.607 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-16T09:04:05.607 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-16T09:04:05.607 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-16T09:04:05.608 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-16T09:04:05.608 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-16T09:04:05.608 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-16T09:04:05.608 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-16T09:04:05.608 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-16T09:04:05.608 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-16T09:04:05.608 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-16T09:04:05.608 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-16T09:04:05.608 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-16T09:04:05.608 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-16T09:04:05.608 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-16T09:04:05.608 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-16T09:04:05.608 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-16T09:04:05.608 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-16T09:04:05.609 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-16T09:04:05.609 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-16T09:04:05.609 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-16T09:04:05.609 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-16T09:04:05.609 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-16T09:04:05.609 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-16T09:04:05.609 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-16T09:04:05.609 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-16T09:04:05.609 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-16T09:04:05.609 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-16T09:04:05.609 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-16T09:04:05.609 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-16T09:04:05.609 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-16T09:04:05.609 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-16T09:04:05.610 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-16T09:04:05.610 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-16T09:04:05.610 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-16T09:04:05.610 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-16T09:04:05.610 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-16T09:04:05.610 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-16T09:04:05.610 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-16T09:04:05.610 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-16T09:04:05.610 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-16T09:04:05.610 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-16T09:04:05.610 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-16T09:04:05.610 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-16T09:04:05.610 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-16T09:04:05.611 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-16T09:04:05.611 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-16T09:04:05.611 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-16T09:04:05.611 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-16T09:04:05.611 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-16T09:04:05.611 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-16T09:04:05.611 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-16T09:04:05.611 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-16T09:04:05.611 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-16T09:04:05.611 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-16T09:04:05.611 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-16T09:04:05.611 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-16T09:04:08.702 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-16T09:04:08.702 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-16T09:04:08.702 INFO:teuthology.orchestra.run.smithi159.stdout:Upgraded: 2024-03-16T09:04:08.702 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-16T09:04:08.702 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-16T09:04:08.702 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-16T09:04:08.703 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-16T09:04:08.703 INFO:teuthology.orchestra.run.smithi159.stdout: acl-2.3.1-4.el9.x86_64 2024-03-16T09:04:08.703 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-16T09:04:08.703 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-16T09:04:08.703 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-16T09:04:08.703 INFO:teuthology.orchestra.run.smithi159.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-16T09:04:08.703 INFO:teuthology.orchestra.run.smithi159.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-16T09:04:08.703 INFO:teuthology.orchestra.run.smithi159.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-16T09:04:08.703 INFO:teuthology.orchestra.run.smithi159.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-16T09:04:08.703 INFO:teuthology.orchestra.run.smithi159.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-16T09:04:08.703 INFO:teuthology.orchestra.run.smithi159.stdout: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.703 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.703 INFO:teuthology.orchestra.run.smithi159.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-16T09:04:08.703 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-16T09:04:08.704 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-16T09:04:08.704 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-16T09:04:08.704 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-16T09:04:08.704 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-16T09:04:08.704 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-16T09:04:08.704 INFO:teuthology.orchestra.run.smithi159.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-16T09:04:08.704 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-16T09:04:08.704 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-03-16T09:04:08.704 INFO:teuthology.orchestra.run.smithi159.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-16T09:04:08.704 INFO:teuthology.orchestra.run.smithi159.stdout: automake-1.16.2-8.el9.noarch 2024-03-16T09:04:08.704 INFO:teuthology.orchestra.run.smithi159.stdout: bison-3.7.4-5.el9.x86_64 2024-03-16T09:04:08.704 INFO:teuthology.orchestra.run.smithi159.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-16T09:04:08.704 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.705 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.705 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.705 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.705 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:04:08.705 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.705 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.705 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.705 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:04:08.705 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:04:08.705 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:04:08.705 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:04:08.705 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:04:08.705 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.705 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.706 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:04:08.706 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.706 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.706 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.706 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:04:08.706 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:04:08.706 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.706 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.706 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-16T09:04:08.706 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-16T09:04:08.706 INFO:teuthology.orchestra.run.smithi159.stdout: flex-2.6.4-9.el9.x86_64 2024-03-16T09:04:08.706 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-16T09:04:08.707 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-16T09:04:08.707 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-16T09:04:08.707 INFO:teuthology.orchestra.run.smithi159.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-16T09:04:08.707 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-16T09:04:08.707 INFO:teuthology.orchestra.run.smithi159.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-16T09:04:08.707 INFO:teuthology.orchestra.run.smithi159.stdout: jq-1.6-16.el9.x86_64 2024-03-16T09:04:08.707 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-16T09:04:08.707 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-16T09:04:08.707 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-16T09:04:08.707 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.707 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.707 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.707 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-16T09:04:08.708 INFO:teuthology.orchestra.run.smithi159.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-16T09:04:08.708 INFO:teuthology.orchestra.run.smithi159.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-16T09:04:08.708 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-16T09:04:08.708 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-16T09:04:08.708 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.708 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.708 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-16T09:04:08.708 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.708 INFO:teuthology.orchestra.run.smithi159.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-16T09:04:08.708 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-16T09:04:08.708 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-16T09:04:08.708 INFO:teuthology.orchestra.run.smithi159.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-16T09:04:08.708 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-16T09:04:08.708 INFO:teuthology.orchestra.run.smithi159.stdout: lua-5.4.4-4.el9.x86_64 2024-03-16T09:04:08.709 INFO:teuthology.orchestra.run.smithi159.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-16T09:04:08.709 INFO:teuthology.orchestra.run.smithi159.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-16T09:04:08.709 INFO:teuthology.orchestra.run.smithi159.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-16T09:04:08.709 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-16T09:04:08.709 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-16T09:04:08.709 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-16T09:04:08.709 INFO:teuthology.orchestra.run.smithi159.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-16T09:04:08.709 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-13.11-1.el9.x86_64 2024-03-16T09:04:08.709 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-16T09:04:08.709 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-16T09:04:08.709 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-16T09:04:08.709 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-16T09:04:08.709 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-16T09:04:08.709 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-16T09:04:08.710 INFO:teuthology.orchestra.run.smithi159.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-16T09:04:08.710 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-16T09:04:08.710 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-16T09:04:08.710 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-16T09:04:08.710 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.710 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.710 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.710 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-16T09:04:08.710 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-16T09:04:08.710 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-16T09:04:08.710 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-16T09:04:08.711 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-16T09:04:08.711 INFO:teuthology.orchestra.run.smithi159.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-16T09:04:08.711 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-16T09:04:08.711 INFO:teuthology.orchestra.run.smithi159.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-16T09:04:08.711 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-16T09:04:08.711 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-16T09:04:08.711 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-16T09:04:08.711 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-16T09:04:08.711 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-16T09:04:08.711 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-16T09:04:08.711 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-16T09:04:08.711 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-16T09:04:08.711 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-16T09:04:08.711 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-16T09:04:08.712 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-16T09:04:08.712 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-16T09:04:08.712 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-16T09:04:08.712 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-16T09:04:08.712 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-16T09:04:08.712 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-16T09:04:08.712 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-16T09:04:08.712 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-16T09:04:08.712 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-16T09:04:08.712 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-16T09:04:08.712 INFO:teuthology.orchestra.run.smithi159.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-16T09:04:08.712 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-16T09:04:08.712 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ply-3.11-14.el9.noarch 2024-03-16T09:04:08.712 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-16T09:04:08.713 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-16T09:04:08.713 INFO:teuthology.orchestra.run.smithi159.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-16T09:04:08.713 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-16T09:04:08.713 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-16T09:04:08.713 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-16T09:04:08.713 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-16T09:04:08.713 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-16T09:04:08.713 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-16T09:04:08.713 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.713 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.713 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-16T09:04:08.713 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-16T09:04:08.713 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-16T09:04:08.713 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.714 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-16T09:04:08.714 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-16T09:04:08.714 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-16T09:04:08.714 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-16T09:04:08.714 INFO:teuthology.orchestra.run.smithi159.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-16T09:04:08.714 INFO:teuthology.orchestra.run.smithi159.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-16T09:04:08.714 INFO:teuthology.orchestra.run.smithi159.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-16T09:04:08.714 INFO:teuthology.orchestra.run.smithi159.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-16T09:04:08.714 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-16T09:04:08.714 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-16T09:04:08.714 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-16T09:04:08.714 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-16T09:04:08.714 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-16T09:04:08.715 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-16T09:04:08.715 INFO:teuthology.orchestra.run.smithi159.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-16T09:04:08.715 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.715 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.715 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-16T09:04:08.715 INFO:teuthology.orchestra.run.smithi159.stdout: re2-1:20211101-3.el9.x86_64 2024-03-16T09:04:08.715 INFO:teuthology.orchestra.run.smithi159.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-16T09:04:08.715 INFO:teuthology.orchestra.run.smithi159.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-16T09:04:08.715 INFO:teuthology.orchestra.run.smithi159.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-16T09:04:08.715 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-16T09:04:08.715 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-16T09:04:08.716 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-16T09:04:08.716 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-16T09:04:08.716 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-03-16T09:04:09.218 DEBUG:teuthology.parallel:result is None 2024-03-16T09:04:09.219 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:04:09.362 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-16T09:04:09.387 INFO:teuthology.orchestra.run.smithi112.stdout:19.0.0-2162.g3d78f5ba.el9 2024-03-16T09:04:09.388 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2162.g3d78f5ba.el9 2024-03-16T09:04:09.388 INFO:teuthology.task.install:The correct ceph version 19.0.0-2162.g3d78f5ba is installed. 2024-03-16T09:04:09.391 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:04:09.603 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-16T09:04:09.626 INFO:teuthology.orchestra.run.smithi159.stdout:19.0.0-2162.g3d78f5ba.el9 2024-03-16T09:04:09.626 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2162.g3d78f5ba.el9 2024-03-16T09:04:09.626 INFO:teuthology.task.install:The correct ceph version 19.0.0-2162.g3d78f5ba is installed. 2024-03-16T09:04:09.628 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:04:09.774 DEBUG:teuthology.orchestra.run.smithi194:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-16T09:04:09.800 INFO:teuthology.orchestra.run.smithi194.stdout:19.0.0-2162.g3d78f5ba.el9 2024-03-16T09:04:09.800 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2162.g3d78f5ba.el9 2024-03-16T09:04:09.800 INFO:teuthology.task.install:The correct ceph version 19.0.0-2162.g3d78f5ba is installed. 2024-03-16T09:04:09.802 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-16T09:04:09.802 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T09:04:09.802 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-16T09:04:09.836 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-16T09:04:09.837 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-16T09:04:09.868 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:04:09.868 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-16T09:04:09.903 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-16T09:04:09.903 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T09:04:09.903 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2024-03-16T09:04:09.932 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-16T09:04:10.001 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-16T09:04:10.002 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/daemon-helper 2024-03-16T09:04:10.030 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-16T09:04:10.099 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:04:10.099 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/daemon-helper 2024-03-16T09:04:10.129 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-16T09:04:10.199 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-16T09:04:10.200 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T09:04:10.200 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-16T09:04:10.227 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-16T09:04:10.295 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-16T09:04:10.295 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-16T09:04:10.324 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-16T09:04:10.393 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:04:10.393 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-16T09:04:10.423 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-16T09:04:10.492 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-16T09:04:10.493 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T09:04:10.493 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2024-03-16T09:04:10.533 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-16T09:04:10.601 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-16T09:04:10.601 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/stdin-killer 2024-03-16T09:04:10.631 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-16T09:04:10.699 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:04:10.700 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/stdin-killer 2024-03-16T09:04:10.729 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-16T09:04:10.800 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-16T09:04:10.900 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client dirsize rbytes': False, 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-03-16T09:04:10.900 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:04:10.901 INFO:tasks.cephadm:Cluster fsid is 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:04:10.901 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-16T09:04:10.901 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.112', 'mon.b': '172.21.15.159', 'mon.c': '172.21.15.194'} 2024-03-16T09:04:10.901 INFO:tasks.cephadm:First mon is mon.a on smithi112 2024-03-16T09:04:10.901 INFO:tasks.cephadm:First mgr is x 2024-03-16T09:04:10.901 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-16T09:04:10.902 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname $(hostname -s) 2024-03-16T09:04:10.929 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname $(hostname -s) 2024-03-16T09:04:10.960 DEBUG:teuthology.orchestra.run.smithi194:> sudo hostname $(hostname -s) 2024-03-16T09:04:10.992 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-16T09:04:10.992 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:04:11.131 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77658/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2162-g3d78f5ba', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2162.g3d78f5ba'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-15 01:33:04.210510', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240314.183243', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-16T09:04:11.371 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-batrick-testing-20240314.183243, sha1 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:04:11.372 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/x86_64/flavors/default/cephadm 2024-03-16T09:04:11.372 DEBUG:teuthology.orchestra.run.smithi112:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-16T09:04:11.806 INFO:teuthology.orchestra.run.smithi112.stdout:-rw-r--r--. 1 ubuntu ubuntu 767536 Mar 16 09:04 /home/ubuntu/cephtest/cephadm 2024-03-16T09:04:11.806 DEBUG:teuthology.orchestra.run.smithi159:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-16T09:04:12.379 INFO:teuthology.orchestra.run.smithi159.stdout:-rw-r--r--. 1 ubuntu ubuntu 767536 Mar 16 09:04 /home/ubuntu/cephtest/cephadm 2024-03-16T09:04:12.379 DEBUG:teuthology.orchestra.run.smithi194:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-16T09:04:12.880 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r--. 1 ubuntu ubuntu 767536 Mar 16 09:04 /home/ubuntu/cephtest/cephadm 2024-03-16T09:04:12.881 DEBUG:teuthology.orchestra.run.smithi112:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-16T09:04:12.900 DEBUG:teuthology.orchestra.run.smithi159:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-16T09:04:12.919 DEBUG:teuthology.orchestra.run.smithi194:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-16T09:04:12.953 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af on all hosts... 2024-03-16T09:04:12.953 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af pull 2024-03-16T09:04:12.956 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af pull 2024-03-16T09:04:12.961 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af pull 2024-03-16T09:04:13.788 INFO:teuthology.orchestra.run.smithi112.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-16T09:04:13.797 INFO:teuthology.orchestra.run.smithi194.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-16T09:04:13.858 INFO:teuthology.orchestra.run.smithi159.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-16T09:05:06.924 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-03-16T09:05:06.924 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_version": "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)", 2024-03-16T09:05:06.925 INFO:teuthology.orchestra.run.smithi194.stdout: "image_id": "be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae", 2024-03-16T09:05:06.925 INFO:teuthology.orchestra.run.smithi194.stdout: "repo_digests": [ 2024-03-16T09:05:06.925 INFO:teuthology.orchestra.run.smithi194.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513" 2024-03-16T09:05:06.925 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2024-03-16T09:05:06.925 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-03-16T09:05:15.318 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-03-16T09:05:15.318 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_version": "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)", 2024-03-16T09:05:15.318 INFO:teuthology.orchestra.run.smithi159.stdout: "image_id": "be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae", 2024-03-16T09:05:15.318 INFO:teuthology.orchestra.run.smithi159.stdout: "repo_digests": [ 2024-03-16T09:05:15.318 INFO:teuthology.orchestra.run.smithi159.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513" 2024-03-16T09:05:15.318 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-03-16T09:05:15.318 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-03-16T09:05:21.675 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:05:21.675 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)", 2024-03-16T09:05:21.675 INFO:teuthology.orchestra.run.smithi112.stdout: "image_id": "be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae", 2024-03-16T09:05:21.675 INFO:teuthology.orchestra.run.smithi112.stdout: "repo_digests": [ 2024-03-16T09:05:21.675 INFO:teuthology.orchestra.run.smithi112.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513" 2024-03-16T09:05:21.675 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-03-16T09:05:21.675 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:05:21.699 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph 2024-03-16T09:05:21.754 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /etc/ceph 2024-03-16T09:05:21.790 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /etc/ceph 2024-03-16T09:05:21.826 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /etc/ceph 2024-03-16T09:05:21.853 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 777 /etc/ceph 2024-03-16T09:05:21.883 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 777 /etc/ceph 2024-03-16T09:05:21.914 INFO:tasks.cephadm:Writing seed config... 2024-03-16T09:05:21.915 INFO:tasks.cephadm: override: [client] client dirsize rbytes = False 2024-03-16T09:05:21.915 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-16T09:05:21.916 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-16T09:05:21.916 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-16T09:05:21.916 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-16T09:05:21.916 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-16T09:05:21.916 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-16T09:05:21.916 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-16T09:05:21.916 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-16T09:05:21.916 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-16T09:05:21.916 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-16T09:05:21.916 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-16T09:05:21.916 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-16T09:05:21.916 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-16T09:05:21.917 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-16T09:05:21.917 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-16T09:05:21.917 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-16T09:05:21.917 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-16T09:05:21.917 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-16T09:05:21.917 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-16T09:05:21.917 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-16T09:05:21.917 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-16T09:05:21.917 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-16T09:05:21.917 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-16T09:05:21.917 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-16T09:05:21.917 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-16T09:05:21.917 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-16T09:05:21.918 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-16T09:05:21.918 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-16T09:05:21.918 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-16T09:05:21.918 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-16T09:05:21.918 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-16T09:05:21.918 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-16T09:05:21.918 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-16T09:05:21.918 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-16T09:05:21.918 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-16T09:05:21.918 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-16T09:05:21.918 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-16T09:05:21.918 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-16T09:05:21.919 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-16T09:05:21.919 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-16T09:05:21.919 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-16T09:05:21.919 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-16T09:05:21.919 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-16T09:05:21.919 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-16T09:05:21.920 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T09:05:21.920 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-16T09:05:21.935 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 273be2a4-e374-11ee-95c9-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client dirsize rbytes = False client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-16T09:05:21.935 DEBUG:teuthology.orchestra.run.smithi112:mon.a> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mon.a.service 2024-03-16T09:05:21.978 DEBUG:teuthology.orchestra.run.smithi112:mgr.x> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mgr.x.service 2024-03-16T09:05:22.021 INFO:tasks.cephadm:Bootstrapping... 2024-03-16T09:05:22.021 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -v bootstrap --fsid 273be2a4-e374-11ee-95c9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.112 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-16T09:05:22.212 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-03-16T09:05:22.213 INFO:teuthology.orchestra.run.smithi112.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', '-v', 'bootstrap', '--fsid', '273be2a4-e374-11ee-95c9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.112', '--skip-admin-label'] 2024-03-16T09:05:22.237 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-16T09:05:22.237 INFO:teuthology.orchestra.run.smithi112.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-16T09:05:22.237 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying podman|docker is present... 2024-03-16T09:05:22.260 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-16T09:05:22.261 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying lvm2 is present... 2024-03-16T09:05:22.261 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying time synchronization is in place... 2024-03-16T09:05:22.268 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-16T09:05:22.268 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-16T09:05:22.274 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-16T09:05:22.274 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-03-16T09:05:22.280 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout enabled 2024-03-16T09:05:22.286 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout active 2024-03-16T09:05:22.286 INFO:teuthology.orchestra.run.smithi112.stdout:Unit chronyd.service is enabled and running 2024-03-16T09:05:22.286 INFO:teuthology.orchestra.run.smithi112.stdout:Repeating the final host check... 2024-03-16T09:05:22.309 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-16T09:05:22.309 INFO:teuthology.orchestra.run.smithi112.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-16T09:05:22.309 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl is present 2024-03-16T09:05:22.309 INFO:teuthology.orchestra.run.smithi112.stdout:lvcreate is present 2024-03-16T09:05:22.315 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-16T09:05:22.315 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-16T09:05:22.321 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-16T09:05:22.321 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-03-16T09:05:22.327 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout enabled 2024-03-16T09:05:22.334 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout active 2024-03-16T09:05:22.334 INFO:teuthology.orchestra.run.smithi112.stdout:Unit chronyd.service is enabled and running 2024-03-16T09:05:22.334 INFO:teuthology.orchestra.run.smithi112.stdout:Host looks OK 2024-03-16T09:05:22.334 INFO:teuthology.orchestra.run.smithi112.stdout:Cluster fsid: 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:05:22.334 INFO:teuthology.orchestra.run.smithi112.stdout:Acquiring lock 140409576922560 on /run/cephadm/273be2a4-e374-11ee-95c9-87774f69a715.lock 2024-03-16T09:05:22.334 INFO:teuthology.orchestra.run.smithi112.stdout:Lock 140409576922560 acquired on /run/cephadm/273be2a4-e374-11ee-95c9-87774f69a715.lock 2024-03-16T09:05:22.334 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying IP 172.21.15.112 port 3300 ... 2024-03-16T09:05:22.335 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying IP 172.21.15.112 port 6789 ... 2024-03-16T09:05:22.335 INFO:teuthology.orchestra.run.smithi112.stdout:Base mon IP(s) is [172.21.15.112:3300, 172.21.15.112:6789], mon addrv is [v2:172.21.15.112:3300,v1:172.21.15.112:6789] 2024-03-16T09:05:22.338 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.112 metric 100 2024-03-16T09:05:22.338 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.112 metric 100 2024-03-16T09:05:22.340 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-16T09:05:22.340 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-16T09:05:22.340 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1752sec hoplimit 64 pref medium 2024-03-16T09:05:22.343 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-16T09:05:22.343 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-16T09:05:22.343 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-16T09:05:22.343 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-16T09:05:22.343 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link 2024-03-16T09:05:22.343 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-16T09:05:22.344 INFO:teuthology.orchestra.run.smithi112.stdout:Mon IP `172.21.15.112` is in CIDR network `172.21.0.0/20` 2024-03-16T09:05:22.344 INFO:teuthology.orchestra.run.smithi112.stdout:Mon IP `172.21.15.112` is in CIDR network `172.21.0.0/20` 2024-03-16T09:05:22.344 INFO:teuthology.orchestra.run.smithi112.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-16T09:05:22.344 INFO:teuthology.orchestra.run.smithi112.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-16T09:05:22.345 INFO:teuthology.orchestra.run.smithi112.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-16T09:05:22.811 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae 2024-03-16T09:05:22.811 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-16T09:05:22.811 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Getting image source signatures 2024-03-16T09:05:22.811 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying blob sha256:66487374cb611c95435850c17b53ebb64c22ed9555f72cd43d678b60262a12b1 2024-03-16T09:05:22.811 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-16T09:05:22.811 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying config sha256:be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae 2024-03-16T09:05:22.811 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-16T09:05:23.732 INFO:teuthology.orchestra.run.smithi112.stdout:ceph: stdout ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev) 2024-03-16T09:05:23.732 INFO:teuthology.orchestra.run.smithi112.stdout:Ceph version: ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev) 2024-03-16T09:05:23.732 INFO:teuthology.orchestra.run.smithi112.stdout:Extracting ceph user uid/gid from container image... 2024-03-16T09:05:24.712 INFO:teuthology.orchestra.run.smithi112.stdout:stat: stdout 167 167 2024-03-16T09:05:24.712 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial keys... 2024-03-16T09:05:25.559 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQDVYPVlLeCoARAAKxiOJ5PEI7q/2uiFc0e76A== 2024-03-16T09:05:26.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQDVYPVlStwoMRAAuYiZW1j/y8Rgcmi6Y7KaYw== 2024-03-16T09:05:27.262 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQDWYPVlhFqPKBAA0vOcp0dGGpJHdM1U1SFWIw== 2024-03-16T09:05:27.262 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial monmap... 2024-03-16T09:05:28.210 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-16T09:05:28.210 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-16T09:05:28.210 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:05:28.210 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-16T09:05:28.210 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool for a [v2:172.21.15.112:3300,v1:172.21.15.112:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-16T09:05:28.210 INFO:teuthology.orchestra.run.smithi112.stdout:setting min_mon_release = quincy 2024-03-16T09:05:28.210 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: set fsid to 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:05:28.210 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-16T09:05:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:05:28.210 INFO:teuthology.orchestra.run.smithi112.stdout:Creating mon... 2024-03-16T09:05:29.191 INFO:teuthology.orchestra.run.smithi112.stdout:create mon.a on 2024-03-16T09:05:29.404 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-16T09:05:29.545 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-16T09:05:29.730 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-273be2a4-e374-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-273be2a4-e374-11ee-95c9-87774f69a715.target. 2024-03-16T09:05:29.730 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-273be2a4-e374-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-273be2a4-e374-11ee-95c9-87774f69a715.target. 2024-03-16T09:05:29.957 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-273be2a4-e374-11ee-95c9-87774f69a715@mon.a 2024-03-16T09:05:29.958 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to reset failed state of unit ceph-273be2a4-e374-11ee-95c9-87774f69a715@mon.a.service: Unit ceph-273be2a4-e374-11ee-95c9-87774f69a715@mon.a.service not loaded. 2024-03-16T09:05:30.116 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-273be2a4-e374-11ee-95c9-87774f69a715.target.wants/ceph-273be2a4-e374-11ee-95c9-87774f69a715@mon.a.service → /etc/systemd/system/ceph-273be2a4-e374-11ee-95c9-87774f69a715@.service. 2024-03-16T09:05:30.666 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T09:05:30.666 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-03-16T09:05:30.673 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T09:05:30.673 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-03-16T09:05:30.673 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-03-16T09:05:30.673 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to enable service . firewalld.service is not available 2024-03-16T09:05:30.674 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mon to start... 2024-03-16T09:05:30.674 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mon... 2024-03-16T09:05:31.676 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout cluster: 2024-03-16T09:05:31.677 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout id: 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:05:31.677 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-16T09:05:31.677 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-03-16T09:05:31.677 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout services: 2024-03-16T09:05:31.677 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.55943s) 2024-03-16T09:05:31.677 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-16T09:05:31.677 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-16T09:05:31.677 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-03-16T09:05:31.677 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout data: 2024-03-16T09:05:31.677 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-16T09:05:31.677 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-16T09:05:31.677 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-16T09:05:31.677 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout pgs: 2024-03-16T09:05:31.678 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-03-16T09:05:31.678 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.095+0000 7fc3e56c7700 1 Processor -- start 2024-03-16T09:05:31.678 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.095+0000 7fc3e56c7700 1 -- start start 2024-03-16T09:05:31.678 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.095+0000 7fc3e56c7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e0106630 0x7fc3e0106a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:31.678 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.095+0000 7fc3e56c7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e007be30 con 0x7fc3e0106630 2024-03-16T09:05:31.678 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.096+0000 7fc3deffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e0106630 0x7fc3e0106a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:31.678 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.096+0000 7fc3deffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e0106630 0x7fc3e0106a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42562/0 (socket says 172.21.15.112:42562) 2024-03-16T09:05:31.678 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.096+0000 7fc3deffd700 1 -- 172.21.15.112:0/582571600 learned_addr learned my addr 172.21.15.112:0/582571600 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:31.678 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.096+0000 7fc3deffd700 1 -- 172.21.15.112:0/582571600 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e0106f90 con 0x7fc3e0106630 2024-03-16T09:05:31.678 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.096+0000 7fc3deffd700 1 --2- 172.21.15.112:0/582571600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e0106630 0x7fc3e0106a50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3d00098a0 tx=0x7fc3d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef18140e6f3cff14 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:31.678 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.096+0000 7fc3ddffb700 1 -- 172.21.15.112:0/582571600 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3d0004160 con 0x7fc3e0106630 2024-03-16T09:05:31.678 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.096+0000 7fc3ddffb700 1 -- 172.21.15.112:0/582571600 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d0004300 con 0x7fc3e0106630 2024-03-16T09:05:31.678 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.096+0000 7fc3ddffb700 1 -- 172.21.15.112:0/582571600 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3d00036a0 con 0x7fc3e0106630 2024-03-16T09:05:31.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.097+0000 7fc3e56c7700 1 -- 172.21.15.112:0/582571600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e0106630 msgr2=0x7fc3e0106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:31.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.097+0000 7fc3e56c7700 1 --2- 172.21.15.112:0/582571600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e0106630 0x7fc3e0106a50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3d00098a0 tx=0x7fc3d0009bb0 comp rx=0 tx=0).stop 2024-03-16T09:05:31.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.097+0000 7fc3e56c7700 1 -- 172.21.15.112:0/582571600 shutdown_connections 2024-03-16T09:05:31.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.097+0000 7fc3e56c7700 1 --2- 172.21.15.112:0/582571600 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e0106630 0x7fc3e0106a50 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:31.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.097+0000 7fc3e56c7700 1 -- 172.21.15.112:0/582571600 >> 172.21.15.112:0/582571600 conn(0x7fc3e0102230 msgr2=0x7fc3e0104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:31.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.097+0000 7fc3e56c7700 1 -- 172.21.15.112:0/582571600 shutdown_connections 2024-03-16T09:05:31.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.097+0000 7fc3e56c7700 1 -- 172.21.15.112:0/582571600 wait complete. 2024-03-16T09:05:31.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.098+0000 7fc3e56c7700 1 Processor -- start 2024-03-16T09:05:31.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.098+0000 7fc3e56c7700 1 -- start start 2024-03-16T09:05:31.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.098+0000 7fc3e56c7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e0106630 0x7fc3e019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:31.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.098+0000 7fc3e56c7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e01074c0 con 0x7fc3e0106630 2024-03-16T09:05:31.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.098+0000 7fc3deffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e0106630 0x7fc3e019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:31.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.098+0000 7fc3deffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e0106630 0x7fc3e019d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42578/0 (socket says 172.21.15.112:42578) 2024-03-16T09:05:31.679 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.098+0000 7fc3deffd700 1 -- 172.21.15.112:0/398564790 learned_addr learned my addr 172.21.15.112:0/398564790 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:31.680 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.098+0000 7fc3deffd700 1 -- 172.21.15.112:0/398564790 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e019db50 con 0x7fc3e0106630 2024-03-16T09:05:31.680 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.099+0000 7fc3deffd700 1 --2- 172.21.15.112:0/398564790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e0106630 0x7fc3e019d610 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0000c00 tx=0x7fc3d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:31.680 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.099+0000 7fc3c7fff700 1 -- 172.21.15.112:0/398564790 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3d0003db0 con 0x7fc3e0106630 2024-03-16T09:05:31.680 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.099+0000 7fc3c7fff700 1 -- 172.21.15.112:0/398564790 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d0011630 con 0x7fc3e0106630 2024-03-16T09:05:31.680 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.099+0000 7fc3c7fff700 1 -- 172.21.15.112:0/398564790 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3d0011950 con 0x7fc3e0106630 2024-03-16T09:05:31.680 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.099+0000 7fc3e56c7700 1 -- 172.21.15.112:0/398564790 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e019dde0 con 0x7fc3e0106630 2024-03-16T09:05:31.680 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.099+0000 7fc3e56c7700 1 -- 172.21.15.112:0/398564790 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e01a0ad0 con 0x7fc3e0106630 2024-03-16T09:05:31.680 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.099+0000 7fc3c7fff700 1 -- 172.21.15.112:0/398564790 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc3d0011af0 con 0x7fc3e0106630 2024-03-16T09:05:31.680 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.099+0000 7fc3c7fff700 1 -- 172.21.15.112:0/398564790 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc3d001c450 con 0x7fc3e0106630 2024-03-16T09:05:31.680 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.100+0000 7fc3e56c7700 1 -- 172.21.15.112:0/398564790 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e0063900 con 0x7fc3e0106630 2024-03-16T09:05:31.680 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.102+0000 7fc3c7fff700 1 -- 172.21.15.112:0/398564790 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc3d0004260 con 0x7fc3e0106630 2024-03-16T09:05:31.680 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.173+0000 7fc3e56c7700 1 -- 172.21.15.112:0/398564790 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fc3e01a0ca0 con 0x7fc3e0106630 2024-03-16T09:05:31.680 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.173+0000 7fc3c7fff700 1 -- 172.21.15.112:0/398564790 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+316 (secure 0 0 0) 0x7fc3d001b450 con 0x7fc3e0106630 2024-03-16T09:05:31.680 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.174+0000 7fc3e56c7700 1 -- 172.21.15.112:0/398564790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e0106630 msgr2=0x7fc3e019d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:31.680 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.174+0000 7fc3e56c7700 1 --2- 172.21.15.112:0/398564790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e0106630 0x7fc3e019d610 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0000c00 tx=0x7fc3d0004750 comp rx=0 tx=0).stop 2024-03-16T09:05:31.681 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.174+0000 7fc3e56c7700 1 -- 172.21.15.112:0/398564790 shutdown_connections 2024-03-16T09:05:31.681 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.174+0000 7fc3e56c7700 1 --2- 172.21.15.112:0/398564790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3e0106630 0x7fc3e019d610 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:31.681 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.174+0000 7fc3e56c7700 1 -- 172.21.15.112:0/398564790 >> 172.21.15.112:0/398564790 conn(0x7fc3e0102230 msgr2=0x7fc3e0193c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:31.681 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.175+0000 7fc3e56c7700 1 -- 172.21.15.112:0/398564790 shutdown_connections 2024-03-16T09:05:31.681 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:31.175+0000 7fc3e56c7700 1 -- 172.21.15.112:0/398564790 wait complete. 2024-03-16T09:05:31.681 INFO:teuthology.orchestra.run.smithi112.stdout:mon is available 2024-03-16T09:05:31.681 INFO:teuthology.orchestra.run.smithi112.stdout:Assimilating anything we can from ceph.conf... 2024-03-16T09:05:32.659 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-03-16T09:05:32.659 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [global] 2024-03-16T09:05:32.659 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout fsid = 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:05:32.659 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.112:3300,v1:172.21.15.112:6789] 2024-03-16T09:05:32.659 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-16T09:05:32.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-16T09:05:32.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-16T09:05:32.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-16T09:05:32.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-03-16T09:05:32.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-16T09:05:32.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-16T09:05:32.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-03-16T09:05:32.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [osd] 2024-03-16T09:05:32.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-16T09:05:32.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-16T09:05:32.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-16T09:05:32.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-16T09:05:32.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-16T09:05:32.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.107+0000 7f7041f74700 1 Processor -- start 2024-03-16T09:05:32.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.107+0000 7f7041f74700 1 -- start start 2024-03-16T09:05:32.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.107+0000 7f7041f74700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c107dc0 0x7f703c1081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:32.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.107+0000 7f7041f74700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f703c1087b0 con 0x7f703c107dc0 2024-03-16T09:05:32.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.107+0000 7f703b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c107dc0 0x7f703c1081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:32.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.107+0000 7f703b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c107dc0 0x7f703c1081e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50788/0 (socket says 172.21.15.112:50788) 2024-03-16T09:05:32.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.107+0000 7f703b7fe700 1 -- 172.21.15.112:0/3242579626 learned_addr learned my addr 172.21.15.112:0/3242579626 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:32.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.108+0000 7f703b7fe700 1 -- 172.21.15.112:0/3242579626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c108f70 con 0x7f703c107dc0 2024-03-16T09:05:32.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.108+0000 7f703b7fe700 1 --2- 172.21.15.112:0/3242579626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c107dc0 0x7f703c1081e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f702c0098a0 tx=0x7f702c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff022a51d2723f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:32.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.108+0000 7f703a7fc700 1 -- 172.21.15.112:0/3242579626 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f702c004160 con 0x7f703c107dc0 2024-03-16T09:05:32.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.108+0000 7f703a7fc700 1 -- 172.21.15.112:0/3242579626 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702c004300 con 0x7f703c107dc0 2024-03-16T09:05:32.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.108+0000 7f703a7fc700 1 -- 172.21.15.112:0/3242579626 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f702c0036a0 con 0x7f703c107dc0 2024-03-16T09:05:32.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.109+0000 7f7041f74700 1 -- 172.21.15.112:0/3242579626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c107dc0 msgr2=0x7f703c1081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:32.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.109+0000 7f7041f74700 1 --2- 172.21.15.112:0/3242579626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c107dc0 0x7f703c1081e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f702c0098a0 tx=0x7f702c009bb0 comp rx=0 tx=0).stop 2024-03-16T09:05:32.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.109+0000 7f7041f74700 1 -- 172.21.15.112:0/3242579626 shutdown_connections 2024-03-16T09:05:32.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.109+0000 7f7041f74700 1 --2- 172.21.15.112:0/3242579626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c107dc0 0x7f703c1081e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:32.662 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.109+0000 7f7041f74700 1 -- 172.21.15.112:0/3242579626 >> 172.21.15.112:0/3242579626 conn(0x7f703c103360 msgr2=0x7f703c1057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:32.662 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.109+0000 7f7041f74700 1 -- 172.21.15.112:0/3242579626 shutdown_connections 2024-03-16T09:05:32.662 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.109+0000 7f7041f74700 1 -- 172.21.15.112:0/3242579626 wait complete. 2024-03-16T09:05:32.662 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.109+0000 7f7041f74700 1 Processor -- start 2024-03-16T09:05:32.662 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.110+0000 7f7041f74700 1 -- start start 2024-03-16T09:05:32.662 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.110+0000 7f7041f74700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c107dc0 0x7f703c07db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:32.662 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.110+0000 7f7041f74700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f703c1094a0 con 0x7f703c107dc0 2024-03-16T09:05:32.662 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.110+0000 7f703b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c107dc0 0x7f703c07db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:32.662 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.110+0000 7f703b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c107dc0 0x7f703c07db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50792/0 (socket says 172.21.15.112:50792) 2024-03-16T09:05:32.662 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.110+0000 7f703b7fe700 1 -- 172.21.15.112:0/1143730855 learned_addr learned my addr 172.21.15.112:0/1143730855 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:32.662 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.110+0000 7f703b7fe700 1 -- 172.21.15.112:0/1143730855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c07e090 con 0x7f703c107dc0 2024-03-16T09:05:32.662 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.110+0000 7f703b7fe700 1 --2- 172.21.15.112:0/1143730855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c107dc0 0x7f703c07db50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f702c000c00 tx=0x7f702c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:32.662 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.110+0000 7f7038ff9700 1 -- 172.21.15.112:0/1143730855 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f702c003db0 con 0x7f703c107dc0 2024-03-16T09:05:32.663 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.110+0000 7f7038ff9700 1 -- 172.21.15.112:0/1143730855 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702c011630 con 0x7f703c107dc0 2024-03-16T09:05:32.663 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.110+0000 7f7041f74700 1 -- 172.21.15.112:0/1143730855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703c07c280 con 0x7f703c107dc0 2024-03-16T09:05:32.663 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.110+0000 7f7038ff9700 1 -- 172.21.15.112:0/1143730855 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f702c011950 con 0x7f703c107dc0 2024-03-16T09:05:32.663 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.110+0000 7f7041f74700 1 -- 172.21.15.112:0/1143730855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f703c07c7c0 con 0x7f703c107dc0 2024-03-16T09:05:32.663 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.111+0000 7f7038ff9700 1 -- 172.21.15.112:0/1143730855 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f702c011af0 con 0x7f703c107dc0 2024-03-16T09:05:32.663 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.111+0000 7f7038ff9700 1 -- 172.21.15.112:0/1143730855 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f702c01c450 con 0x7f703c107dc0 2024-03-16T09:05:32.663 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.111+0000 7f7041f74700 1 -- 172.21.15.112:0/1143730855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f703c063900 con 0x7f703c107dc0 2024-03-16T09:05:32.663 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.113+0000 7f7038ff9700 1 -- 172.21.15.112:0/1143730855 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f702c032020 con 0x7f703c107dc0 2024-03-16T09:05:32.663 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.181+0000 7f7041f74700 1 -- 172.21.15.112:0/1143730855 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f703c194970 con 0x7f703c107dc0 2024-03-16T09:05:32.663 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.184+0000 7f7038ff9700 1 -- 172.21.15.112:0/1143730855 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7f703c194970 con 0x7f703c107dc0 2024-03-16T09:05:32.663 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.185+0000 7f7038ff9700 1 -- 172.21.15.112:0/1143730855 <== mon.0 v2:172.21.15.112:3300/0 8 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f702c004260 con 0x7f703c107dc0 2024-03-16T09:05:32.663 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.185+0000 7f7041f74700 1 -- 172.21.15.112:0/1143730855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c107dc0 msgr2=0x7f703c07db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:32.663 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.185+0000 7f7041f74700 1 --2- 172.21.15.112:0/1143730855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c107dc0 0x7f703c07db50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f702c000c00 tx=0x7f702c004750 comp rx=0 tx=0).stop 2024-03-16T09:05:32.663 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.185+0000 7f7041f74700 1 -- 172.21.15.112:0/1143730855 shutdown_connections 2024-03-16T09:05:32.664 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.185+0000 7f7041f74700 1 --2- 172.21.15.112:0/1143730855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f703c107dc0 0x7f703c07db50 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:32.664 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.185+0000 7f7041f74700 1 -- 172.21.15.112:0/1143730855 >> 172.21.15.112:0/1143730855 conn(0x7f703c103360 msgr2=0x7f703c192020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:32.664 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.186+0000 7f7041f74700 1 -- 172.21.15.112:0/1143730855 shutdown_connections 2024-03-16T09:05:32.664 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:32.186+0000 7f7041f74700 1 -- 172.21.15.112:0/1143730855 wait complete. 2024-03-16T09:05:32.664 INFO:teuthology.orchestra.run.smithi112.stdout:Generating new minimal ceph.conf... 2024-03-16T09:05:33.655 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.055+0000 7f0d9c7ec700 1 Processor -- start 2024-03-16T09:05:33.655 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.055+0000 7f0d9c7ec700 1 -- start start 2024-03-16T09:05:33.655 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.055+0000 7f0d9c7ec700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d9407de50 0x7f0d9407c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:33.655 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.055+0000 7f0d9c7ec700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9407c8d0 con 0x7f0d9407de50 2024-03-16T09:05:33.655 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.055+0000 7f0d9a588700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d9407de50 0x7f0d9407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:33.655 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.055+0000 7f0d9a588700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d9407de50 0x7f0d9407c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50808/0 (socket says 172.21.15.112:50808) 2024-03-16T09:05:33.655 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.055+0000 7f0d9a588700 1 -- 172.21.15.112:0/3179598300 learned_addr learned my addr 172.21.15.112:0/3179598300 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:33.655 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.055+0000 7f0d9a588700 1 -- 172.21.15.112:0/3179598300 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9407ca50 con 0x7f0d9407de50 2024-03-16T09:05:33.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.056+0000 7f0d9a588700 1 --2- 172.21.15.112:0/3179598300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d9407de50 0x7f0d9407c300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0d900098a0 tx=0x7f0d90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07a6a5fa7d46051 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:33.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.056+0000 7f0d99586700 1 -- 172.21.15.112:0/3179598300 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d90004160 con 0x7f0d9407de50 2024-03-16T09:05:33.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.056+0000 7f0d99586700 1 -- 172.21.15.112:0/3179598300 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f0d90004300 con 0x7f0d9407de50 2024-03-16T09:05:33.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.056+0000 7f0d99586700 1 -- 172.21.15.112:0/3179598300 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d90011c00 con 0x7f0d9407de50 2024-03-16T09:05:33.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.056+0000 7f0d9c7ec700 1 -- 172.21.15.112:0/3179598300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d9407de50 msgr2=0x7f0d9407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:33.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.056+0000 7f0d9c7ec700 1 --2- 172.21.15.112:0/3179598300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d9407de50 0x7f0d9407c300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0d900098a0 tx=0x7f0d90009bb0 comp rx=0 tx=0).stop 2024-03-16T09:05:33.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.056+0000 7f0d9c7ec700 1 -- 172.21.15.112:0/3179598300 shutdown_connections 2024-03-16T09:05:33.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.056+0000 7f0d9c7ec700 1 --2- 172.21.15.112:0/3179598300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d9407de50 0x7f0d9407c300 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:33.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.056+0000 7f0d9c7ec700 1 -- 172.21.15.112:0/3179598300 >> 172.21.15.112:0/3179598300 conn(0x7f0d94103360 msgr2=0x7f0d941057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:33.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.056+0000 7f0d9c7ec700 1 -- 172.21.15.112:0/3179598300 shutdown_connections 2024-03-16T09:05:33.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.056+0000 7f0d9c7ec700 1 -- 172.21.15.112:0/3179598300 wait complete. 2024-03-16T09:05:33.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.057+0000 7f0d9c7ec700 1 Processor -- start 2024-03-16T09:05:33.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.057+0000 7f0d9c7ec700 1 -- start start 2024-03-16T09:05:33.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.057+0000 7f0d9c7ec700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d9407de50 0x7f0d9419df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:33.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.057+0000 7f0d9c7ec700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9410e350 con 0x7f0d9407de50 2024-03-16T09:05:33.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.057+0000 7f0d9a588700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d9407de50 0x7f0d9419df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:33.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.057+0000 7f0d9a588700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d9407de50 0x7f0d9419df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50816/0 (socket says 172.21.15.112:50816) 2024-03-16T09:05:33.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.057+0000 7f0d9a588700 1 -- 172.21.15.112:0/3807763264 learned_addr learned my addr 172.21.15.112:0/3807763264 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:33.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.057+0000 7f0d9a588700 1 -- 172.21.15.112:0/3807763264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9419e460 con 0x7f0d9407de50 2024-03-16T09:05:33.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.058+0000 7f0d9a588700 1 --2- 172.21.15.112:0/3807763264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d9407de50 0x7f0d9419df20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0d90011bd0 tx=0x7f0d90011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:33.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.058+0000 7f0d8b7fe700 1 -- 172.21.15.112:0/3807763264 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d90003b20 con 0x7f0d9407de50 2024-03-16T09:05:33.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.058+0000 7f0d8b7fe700 1 -- 172.21.15.112:0/3807763264 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f0d90003cc0 con 0x7f0d9407de50 2024-03-16T09:05:33.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.058+0000 7f0d8b7fe700 1 -- 172.21.15.112:0/3807763264 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d90024930 con 0x7f0d9407de50 2024-03-16T09:05:33.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.058+0000 7f0d9c7ec700 1 -- 172.21.15.112:0/3807763264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9419e6f0 con 0x7f0d9407de50 2024-03-16T09:05:33.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.058+0000 7f0d9c7ec700 1 -- 172.21.15.112:0/3807763264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d941a13e0 con 0x7f0d9407de50 2024-03-16T09:05:33.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.058+0000 7f0d8b7fe700 1 -- 172.21.15.112:0/3807763264 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0d90022020 con 0x7f0d9407de50 2024-03-16T09:05:33.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.058+0000 7f0d8b7fe700 1 -- 172.21.15.112:0/3807763264 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0d900193c0 con 0x7f0d9407de50 2024-03-16T09:05:33.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.059+0000 7f0d9c7ec700 1 -- 172.21.15.112:0/3807763264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d94063900 con 0x7f0d9407de50 2024-03-16T09:05:33.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.061+0000 7f0d8b7fe700 1 -- 172.21.15.112:0/3807763264 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0d9001aca0 con 0x7f0d9407de50 2024-03-16T09:05:33.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.128+0000 7f0d9c7ec700 1 -- 172.21.15.112:0/3807763264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f0d941a1920 con 0x7f0d9407de50 2024-03-16T09:05:33.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.129+0000 7f0d8b7fe700 1 -- 172.21.15.112:0/3807763264 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f0d9002d3f0 con 0x7f0d9407de50 2024-03-16T09:05:33.658 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.130+0000 7f0d9c7ec700 1 -- 172.21.15.112:0/3807763264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d9407de50 msgr2=0x7f0d9419df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:33.658 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.130+0000 7f0d9c7ec700 1 --2- 172.21.15.112:0/3807763264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d9407de50 0x7f0d9419df20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0d90011bd0 tx=0x7f0d90011cb0 comp rx=0 tx=0).stop 2024-03-16T09:05:33.658 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.130+0000 7f0d9c7ec700 1 -- 172.21.15.112:0/3807763264 shutdown_connections 2024-03-16T09:05:33.658 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.130+0000 7f0d9c7ec700 1 --2- 172.21.15.112:0/3807763264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d9407de50 0x7f0d9419df20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:33.658 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.130+0000 7f0d9c7ec700 1 -- 172.21.15.112:0/3807763264 >> 172.21.15.112:0/3807763264 conn(0x7f0d94103360 msgr2=0x7f0d94103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:33.658 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.130+0000 7f0d9c7ec700 1 -- 172.21.15.112:0/3807763264 shutdown_connections 2024-03-16T09:05:33.658 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:33.130+0000 7f0d9c7ec700 1 -- 172.21.15.112:0/3807763264 wait complete. 2024-03-16T09:05:33.658 INFO:teuthology.orchestra.run.smithi112.stdout:Restarting the monitor... 2024-03-16T09:05:33.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:33 smithi112 systemd[1]: Stopping Ceph mon.a for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:05:34.293 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:33 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-a[31918]: 2024-03-16T09:05:33.916+0000 7feaad3f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:05:34.293 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:33 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-a[31918]: 2024-03-16T09:05:33.916+0000 7feaad3f4700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-16T09:05:34.293 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:34 smithi112 podman[32180]: 2024-03-16 09:05:34.090182015 +0000 UTC m=+0.279673851 container died 190c9b9deacc410eebfc25f72068dc2d87539a11f638870dfb17fe1ea5fb7a16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-16T09:05:34.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:34 smithi112 podman[32180]: 2024-03-16 09:05:34.391426286 +0000 UTC m=+0.580918128 container cleanup 190c9b9deacc410eebfc25f72068dc2d87539a11f638870dfb17fe1ea5fb7a16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=) 2024-03-16T09:05:34.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:34 smithi112 bash[32180]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-a 2024-03-16T09:05:34.959 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:34 smithi112 podman[32194]: 2024-03-16 09:05:34.617476133 +0000 UTC m=+0.523993904 container remove 190c9b9deacc410eebfc25f72068dc2d87539a11f638870dfb17fe1ea5fb7a16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-a, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-16T09:05:34.959 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:34 smithi112 systemd[1]: ceph-273be2a4-e374-11ee-95c9-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-16T09:05:34.959 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:34 smithi112 systemd[1]: Stopped Ceph mon.a for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:05:34.959 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:34 smithi112 systemd[1]: Starting Ceph mon.a for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:05:35.214 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:34 smithi112 podman[32288]: 2024-03-16 09:05:34.860321972 +0000 UTC m=+0.017562187 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:05:35.214 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:34 smithi112 podman[32288]: 2024-03-16 09:05:34.959912787 +0000 UTC m=+0.117153037 container create c7d8c6135c176c34c9778c50ff86ef9507cad5b05156eb2e6c3f4aef93534801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-16T09:05:35.214 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 podman[32288]: 2024-03-16 09:05:35.18747912 +0000 UTC m=+0.344719321 container init c7d8c6135c176c34c9778c50ff86ef9507cad5b05156eb2e6c3f4aef93534801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:05:35.214 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 podman[32288]: 2024-03-16 09:05:35.190319363 +0000 UTC m=+0.347559560 container start c7d8c6135c176c34c9778c50ff86ef9507cad5b05156eb2e6c3f4aef93534801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0) 2024-03-16T09:05:35.276 INFO:teuthology.orchestra.run.smithi112.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-16T09:05:35.489 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: set uid:gid to 167:167 (ceph:ceph) 2024-03-16T09:05:35.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev), process ceph-mon, pid 2 2024-03-16T09:05:35.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: pidfile_write: ignore empty --pid-file 2024-03-16T09:05:35.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: load: jerasure load: lrc 2024-03-16T09:05:35.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: RocksDB version: 7.9.2 2024-03-16T09:05:35.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Git sha 0 2024-03-16T09:05:35.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Compile date 2024-03-15 00:33:34 2024-03-16T09:05:35.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: DB SUMMARY 2024-03-16T09:05:35.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: DB Session ID: K141HYQSFFFC7GG0FOI1 2024-03-16T09:05:35.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: CURRENT file: CURRENT 2024-03-16T09:05:35.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: IDENTITY file: IDENTITY 2024-03-16T09:05:35.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-16T09:05:35.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-16T09:05:35.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104872 ; 2024-03-16T09:05:35.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.error_if_exists: 0 2024-03-16T09:05:35.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.create_if_missing: 0 2024-03-16T09:05:35.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.paranoid_checks: 1 2024-03-16T09:05:35.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-16T09:05:35.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-16T09:05:35.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-16T09:05:35.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.env: 0x55c3f2718c60 2024-03-16T09:05:35.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.fs: PosixFileSystem 2024-03-16T09:05:35.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.info_log: 0x55c3f4db1ac0 2024-03-16T09:05:35.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_file_opening_threads: 16 2024-03-16T09:05:35.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.statistics: (nil) 2024-03-16T09:05:35.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.use_fsync: 0 2024-03-16T09:05:35.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_log_file_size: 0 2024-03-16T09:05:35.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-16T09:05:35.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-16T09:05:35.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.keep_log_file_num: 1000 2024-03-16T09:05:35.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.recycle_log_file_num: 0 2024-03-16T09:05:35.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.allow_fallocate: 1 2024-03-16T09:05:35.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.allow_mmap_reads: 0 2024-03-16T09:05:35.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.allow_mmap_writes: 0 2024-03-16T09:05:35.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.use_direct_reads: 0 2024-03-16T09:05:35.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-16T09:05:35.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.create_missing_column_families: 0 2024-03-16T09:05:35.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.db_log_dir: 2024-03-16T09:05:35.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.wal_dir: 2024-03-16T09:05:35.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-16T09:05:35.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-16T09:05:35.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-16T09:05:35.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-16T09:05:35.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-16T09:05:35.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-16T09:05:35.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.advise_random_on_open: 1 2024-03-16T09:05:35.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.db_write_buffer_size: 0 2024-03-16T09:05:35.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.write_buffer_manager: 0x55c3f4d841e0 2024-03-16T09:05:35.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-16T09:05:35.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-16T09:05:35.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-16T09:05:35.494 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.rate_limiter: (nil) 2024-03-16T09:05:35.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-16T09:05:35.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.wal_recovery_mode: 2 2024-03-16T09:05:35.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.enable_thread_tracking: 0 2024-03-16T09:05:35.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.enable_pipelined_write: 0 2024-03-16T09:05:35.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.unordered_write: 0 2024-03-16T09:05:35.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-16T09:05:35.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-16T09:05:35.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-16T09:05:35.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-16T09:05:35.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.row_cache: None 2024-03-16T09:05:35.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.wal_filter: None 2024-03-16T09:05:35.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-16T09:05:35.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.allow_ingest_behind: 0 2024-03-16T09:05:35.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.two_write_queues: 0 2024-03-16T09:05:35.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.manual_wal_flush: 0 2024-03-16T09:05:35.496 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.wal_compression: 0 2024-03-16T09:05:35.496 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.atomic_flush: 0 2024-03-16T09:05:35.496 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-16T09:05:35.496 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-16T09:05:35.496 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-16T09:05:35.496 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.log_readahead_size: 0 2024-03-16T09:05:35.496 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-16T09:05:35.496 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.best_efforts_recovery: 0 2024-03-16T09:05:35.496 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-16T09:05:35.496 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-16T09:05:35.496 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.allow_data_in_errors: 0 2024-03-16T09:05:35.496 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.db_host_id: __hostname__ 2024-03-16T09:05:35.496 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-16T09:05:35.496 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_background_jobs: 2 2024-03-16T09:05:35.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_background_compactions: -1 2024-03-16T09:05:35.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_subcompactions: 1 2024-03-16T09:05:35.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-16T09:05:35.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-16T09:05:35.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-16T09:05:35.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_total_wal_size: 0 2024-03-16T09:05:35.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-16T09:05:35.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-16T09:05:35.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-16T09:05:35.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-16T09:05:35.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_open_files: -1 2024-03-16T09:05:35.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.bytes_per_sync: 0 2024-03-16T09:05:35.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-16T09:05:35.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-16T09:05:35.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compaction_readahead_size: 0 2024-03-16T09:05:35.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_background_flushes: -1 2024-03-16T09:05:35.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Compression algorithms supported: 2024-03-16T09:05:35.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-16T09:05:35.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: kZSTD supported: 0 2024-03-16T09:05:35.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: kXpressCompression supported: 0 2024-03-16T09:05:35.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: kLZ4HCCompression supported: 1 2024-03-16T09:05:35.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: kZlibCompression supported: 1 2024-03-16T09:05:35.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: kSnappyCompression supported: 1 2024-03-16T09:05:35.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: kLZ4Compression supported: 1 2024-03-16T09:05:35.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: kBZip2Compression supported: 0 2024-03-16T09:05:35.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-16T09:05:35.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-16T09:05:35.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-16T09:05:35.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-16T09:05:35.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-16T09:05:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.merge_operator: 2024-03-16T09:05:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compaction_filter: None 2024-03-16T09:05:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compaction_filter_factory: None 2024-03-16T09:05:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.sst_partitioner_factory: None 2024-03-16T09:05:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-16T09:05:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-16T09:05:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c3f4db1bc0) 2024-03-16T09:05:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout: cache_index_and_filter_blocks: 1 2024-03-16T09:05:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-16T09:05:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-16T09:05:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout: pin_top_level_index_and_filter: 1 2024-03-16T09:05:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_type: 0 2024-03-16T09:05:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout: data_block_index_type: 0 2024-03-16T09:05:35.499 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_shortening: 1 2024-03-16T09:05:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-16T09:05:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: checksum: 4 2024-03-16T09:05:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: no_block_cache: 0 2024-03-16T09:05:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache: 0x55c3f4db7090 2024-03-16T09:05:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_name: BinnedLRUCache 2024-03-16T09:05:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_options: 2024-03-16T09:05:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: capacity : 536870912 2024-03-16T09:05:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: num_shard_bits : 4 2024-03-16T09:05:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: strict_capacity_limit : 0 2024-03-16T09:05:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: high_pri_pool_ratio: 0.000 2024-03-16T09:05:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_compressed: (nil) 2024-03-16T09:05:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: persistent_cache: (nil) 2024-03-16T09:05:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_size: 4096 2024-03-16T09:05:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_size_deviation: 10 2024-03-16T09:05:35.500 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_restart_interval: 16 2024-03-16T09:05:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_block_restart_interval: 1 2024-03-16T09:05:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout: metadata_block_size: 4096 2024-03-16T09:05:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout: partition_filters: 0 2024-03-16T09:05:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout: use_delta_encoding: 1 2024-03-16T09:05:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout: filter_policy: bloomfilter 2024-03-16T09:05:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout: whole_key_filtering: 1 2024-03-16T09:05:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout: verify_compression: 0 2024-03-16T09:05:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout: read_amp_bytes_per_bit: 0 2024-03-16T09:05:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout: format_version: 5 2024-03-16T09:05:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout: enable_index_compression: 1 2024-03-16T09:05:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_align: 0 2024-03-16T09:05:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout: max_auto_readahead_size: 262144 2024-03-16T09:05:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout: prepopulate_block_cache: 0 2024-03-16T09:05:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout: initial_auto_readahead_size: 8192 2024-03-16T09:05:35.501 INFO:journalctl@ceph.mon.a.smithi112.stdout: num_file_reads_for_auto_readahead: 2 2024-03-16T09:05:35.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.write_buffer_size: 33554432 2024-03-16T09:05:35.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_write_buffer_number: 2 2024-03-16T09:05:35.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compression: NoCompression 2024-03-16T09:05:35.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.bottommost_compression: Disabled 2024-03-16T09:05:35.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.prefix_extractor: nullptr 2024-03-16T09:05:35.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-16T09:05:35.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.num_levels: 7 2024-03-16T09:05:35.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-16T09:05:35.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-16T09:05:35.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-16T09:05:35.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-16T09:05:35.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-16T09:05:35.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-16T09:05:35.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-16T09:05:35.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-16T09:05:35.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-16T09:05:35.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-16T09:05:35.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-16T09:05:35.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-16T09:05:35.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-16T09:05:35.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compression_opts.level: 32767 2024-03-16T09:05:35.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compression_opts.strategy: 0 2024-03-16T09:05:35.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-16T09:05:35.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-16T09:05:35.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-16T09:05:35.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-16T09:05:35.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compression_opts.enabled: false 2024-03-16T09:05:35.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-16T09:05:35.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-16T09:05:35.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-16T09:05:35.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-16T09:05:35.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.target_file_size_base: 67108864 2024-03-16T09:05:35.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-16T09:05:35.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-16T09:05:35.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-16T09:05:35.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-16T09:05:35.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-16T09:05:35.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-16T09:05:35.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-16T09:05:35.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-16T09:05:35.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-16T09:05:35.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-16T09:05:35.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-16T09:05:35.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-16T09:05:35.505 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-16T09:05:35.505 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-16T09:05:35.505 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.arena_block_size: 1048576 2024-03-16T09:05:35.505 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-16T09:05:35.505 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-16T09:05:35.505 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.disable_auto_compactions: 0 2024-03-16T09:05:35.505 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-16T09:05:35.505 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-16T09:05:35.505 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-16T09:05:35.505 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-16T09:05:35.505 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-16T09:05:35.505 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-16T09:05:35.505 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-16T09:05:35.505 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-16T09:05:35.505 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-16T09:05:35.505 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-16T09:05:35.506 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.table_properties_collectors: 2024-03-16T09:05:35.506 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.inplace_update_support: 0 2024-03-16T09:05:35.506 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-16T09:05:35.506 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-16T09:05:35.506 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-16T09:05:35.506 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-16T09:05:35.506 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.bloom_locality: 0 2024-03-16T09:05:35.506 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.max_successive_merges: 0 2024-03-16T09:05:35.506 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-16T09:05:35.506 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.paranoid_file_checks: 0 2024-03-16T09:05:35.506 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.force_consistency_checks: 1 2024-03-16T09:05:35.506 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.report_bg_io_stats: 0 2024-03-16T09:05:35.506 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.ttl: 2592000 2024-03-16T09:05:35.506 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-16T09:05:35.507 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-16T09:05:35.507 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-16T09:05:35.507 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.enable_blob_files: false 2024-03-16T09:05:35.507 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.min_blob_size: 0 2024-03-16T09:05:35.507 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.blob_file_size: 268435456 2024-03-16T09:05:35.507 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-16T09:05:35.507 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-16T09:05:35.507 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-16T09:05:35.507 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-16T09:05:35.507 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-16T09:05:35.507 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.blob_file_starting_level: 0 2024-03-16T09:05:35.507 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-16T09:05:35.507 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-16T09:05:35.507 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-16T09:05:35.507 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 38f62d2b-24b5-42d7-bdb9-1951e6107e68 2024-03-16T09:05:35.508 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710579935219117, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-16T09:05:35.508 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-16T09:05:35.508 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710579935220093, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100034, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97946, "index_size": 306, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14102, "raw_average_key_size": 49, "raw_value_size": 90265, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710579935, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "38f62d2b-24b5-42d7-bdb9-1951e6107e68", "db_session_id": "K141HYQSFFFC7GG0FOI1", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-16T09:05:35.508 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710579935220174, "job": 1, "event": "recovery_finished"} 2024-03-16T09:05:35.508 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-16T09:05:35.508 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-16T09:05:35.508 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c3f4eaa000 2024-03-16T09:05:35.508 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: DB pointer 0x55c3f4e94000 2024-03-16T09:05:35.508 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-16T09:05:35.508 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-16T09:05:35.508 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** DB Stats ** 2024-03-16T09:05:35.508 INFO:journalctl@ceph.mon.a.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T09:05:35.508 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-16T09:05:35.508 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T09:05:35.508 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T09:05:35.508 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-16T09:05:35.509 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-16T09:05:35.509 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-16T09:05:35.509 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-03-16T09:05:35.509 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** Compaction Stats [default] ** 2024-03-16T09:05:35.509 INFO:journalctl@ceph.mon.a.smithi112.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T09:05:35.509 INFO:journalctl@ceph.mon.a.smithi112.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-16T09:05:35.509 INFO:journalctl@ceph.mon.a.smithi112.stdout: L0 2/0 99.52 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T09:05:35.509 INFO:journalctl@ceph.mon.a.smithi112.stdout: Sum 2/0 99.52 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T09:05:35.509 INFO:journalctl@ceph.mon.a.smithi112.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T09:05:35.509 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-03-16T09:05:35.509 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** Compaction Stats [default] ** 2024-03-16T09:05:35.509 INFO:journalctl@ceph.mon.a.smithi112.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-16T09:05:35.509 INFO:journalctl@ceph.mon.a.smithi112.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-16T09:05:35.509 INFO:journalctl@ceph.mon.a.smithi112.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 150.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-16T09:05:35.509 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-03-16T09:05:35.510 INFO:journalctl@ceph.mon.a.smithi112.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-16T09:05:35.510 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-03-16T09:05:35.510 INFO:journalctl@ceph.mon.a.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-16T09:05:35.510 INFO:journalctl@ceph.mon.a.smithi112.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-16T09:05:35.510 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-16T09:05:35.510 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-16T09:05:35.510 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-16T09:05:35.510 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-16T09:05:35.510 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative compaction: 0.00 GB write, 32.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T09:05:35.510 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval compaction: 0.00 GB write, 32.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-16T09:05:35.510 INFO:journalctl@ceph.mon.a.smithi112.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-16T09:05:35.510 INFO:journalctl@ceph.mon.a.smithi112.stdout: Block cache BinnedLRUCache@0x55c3f4db7090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-16T09:05:35.510 INFO:journalctl@ceph.mon.a.smithi112.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-16T09:05:35.510 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-03-16T09:05:35.511 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-16T09:05:35.511 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: starting mon.a rank 0 at public addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] at bind addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:05:35.511 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: mon.a@-1(???) e1 preinit fsid 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:05:35.511 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: mon.a@-1(???).mds e1 new map 2024-03-16T09:05:35.511 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: mon.a@-1(???).mds e1 print_map 2024-03-16T09:05:35.511 INFO:journalctl@ceph.mon.a.smithi112.stdout: e1 2024-03-16T09:05:35.511 INFO:journalctl@ceph.mon.a.smithi112.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T09:05:35.511 INFO:journalctl@ceph.mon.a.smithi112.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-16T09:05:35.511 INFO:journalctl@ceph.mon.a.smithi112.stdout: legacy client fscid: -1 2024-03-16T09:05:35.511 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-03-16T09:05:35.512 INFO:journalctl@ceph.mon.a.smithi112.stdout: No filesystems configured 2024-03-16T09:05:35.512 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-16T09:05:35.512 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T09:05:35.512 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T09:05:35.512 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T09:05:35.512 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-16T09:05:35.512 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 bash[32288]: c7d8c6135c176c34c9778c50ff86ef9507cad5b05156eb2e6c3f4aef93534801 2024-03-16T09:05:35.512 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 systemd[1]: Started Ceph mon.a for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:05:35.512 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-16T09:05:35.512 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: monmap epoch 1 2024-03-16T09:05:35.512 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: fsid 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:05:35.512 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: last_changed 2024-03-16T09:05:27.608380+0000 2024-03-16T09:05:35.512 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: created 2024-03-16T09:05:27.608380+0000 2024-03-16T09:05:35.512 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: min_mon_release 19 (squid) 2024-03-16T09:05:35.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: election_strategy: 1 2024-03-16T09:05:35.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-03-16T09:05:35.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: fsmap 2024-03-16T09:05:35.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: osdmap e1: 0 total, 0 up, 0 in 2024-03-16T09:05:35.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:35 smithi112 ceph-mon[32307]: mgrmap e1: no daemons active 2024-03-16T09:05:36.302 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.682+0000 7f4bd6780700 1 Processor -- start 2024-03-16T09:05:36.302 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.682+0000 7f4bd6780700 1 -- start start 2024-03-16T09:05:36.303 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.682+0000 7f4bd6780700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0106ca0 0x7f4bd01070c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:36.303 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.682+0000 7f4bd6780700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd0107690 con 0x7f4bd0106ca0 2024-03-16T09:05:36.303 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.683+0000 7f4bcffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0106ca0 0x7f4bd01070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:36.303 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.683+0000 7f4bcffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0106ca0 0x7f4bd01070c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50822/0 (socket says 172.21.15.112:50822) 2024-03-16T09:05:36.303 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.683+0000 7f4bcffff700 1 -- 172.21.15.112:0/2823292786 learned_addr learned my addr 172.21.15.112:0/2823292786 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:36.303 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.683+0000 7f4bcffff700 1 -- 172.21.15.112:0/2823292786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd0107ea0 con 0x7f4bd0106ca0 2024-03-16T09:05:36.303 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.683+0000 7f4bcffff700 1 --2- 172.21.15.112:0/2823292786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0106ca0 0x7f4bd01070c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000bb40 tx=0x7f4bc000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389564d44dc8a044 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:36.303 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.683+0000 7f4bceffd700 1 -- 172.21.15.112:0/2823292786 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bc000dbe0 con 0x7f4bd0106ca0 2024-03-16T09:05:36.303 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.683+0000 7f4bceffd700 1 -- 172.21.15.112:0/2823292786 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f4bc0004510 con 0x7f4bd0106ca0 2024-03-16T09:05:36.303 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.683+0000 7f4bceffd700 1 -- 172.21.15.112:0/2823292786 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bc00104d0 con 0x7f4bd0106ca0 2024-03-16T09:05:36.303 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.684+0000 7f4bd6780700 1 -- 172.21.15.112:0/2823292786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0106ca0 msgr2=0x7f4bd01070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:36.303 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.684+0000 7f4bd6780700 1 --2- 172.21.15.112:0/2823292786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0106ca0 0x7f4bd01070c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000bb40 tx=0x7f4bc000be50 comp rx=0 tx=0).stop 2024-03-16T09:05:36.303 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.684+0000 7f4bd6780700 1 -- 172.21.15.112:0/2823292786 shutdown_connections 2024-03-16T09:05:36.304 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.684+0000 7f4bd6780700 1 --2- 172.21.15.112:0/2823292786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0106ca0 0x7f4bd01070c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:36.304 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.684+0000 7f4bd6780700 1 -- 172.21.15.112:0/2823292786 >> 172.21.15.112:0/2823292786 conn(0x7f4bd0102220 msgr2=0x7f4bd0104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:36.304 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.684+0000 7f4bd6780700 1 -- 172.21.15.112:0/2823292786 shutdown_connections 2024-03-16T09:05:36.304 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.684+0000 7f4bd6780700 1 -- 172.21.15.112:0/2823292786 wait complete. 2024-03-16T09:05:36.304 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.685+0000 7f4bd6780700 1 Processor -- start 2024-03-16T09:05:36.304 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.685+0000 7f4bd6780700 1 -- start start 2024-03-16T09:05:36.304 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.685+0000 7f4bd6780700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0106ca0 0x7f4bd019df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:36.304 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.685+0000 7f4bd6780700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd007afb0 con 0x7f4bd0106ca0 2024-03-16T09:05:36.304 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.685+0000 7f4bcffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0106ca0 0x7f4bd019df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:36.304 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.685+0000 7f4bcffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0106ca0 0x7f4bd019df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50830/0 (socket says 172.21.15.112:50830) 2024-03-16T09:05:36.304 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.685+0000 7f4bcffff700 1 -- 172.21.15.112:0/4186792169 learned_addr learned my addr 172.21.15.112:0/4186792169 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:36.304 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.685+0000 7f4bcffff700 1 -- 172.21.15.112:0/4186792169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd019e490 con 0x7f4bd0106ca0 2024-03-16T09:05:36.304 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.685+0000 7f4bcffff700 1 --2- 172.21.15.112:0/4186792169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0106ca0 0x7f4bd019df50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000b340 tx=0x7f4bc00041b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:36.304 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.686+0000 7f4bcd7fa700 1 -- 172.21.15.112:0/4186792169 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bc000de20 con 0x7f4bd0106ca0 2024-03-16T09:05:36.304 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.686+0000 7f4bd6780700 1 -- 172.21.15.112:0/4186792169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd019e720 con 0x7f4bd0106ca0 2024-03-16T09:05:36.305 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.686+0000 7f4bcd7fa700 1 -- 172.21.15.112:0/4186792169 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f4bc001d070 con 0x7f4bd0106ca0 2024-03-16T09:05:36.305 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.686+0000 7f4bcd7fa700 1 -- 172.21.15.112:0/4186792169 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bc0021d50 con 0x7f4bd0106ca0 2024-03-16T09:05:36.305 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.686+0000 7f4bd6780700 1 -- 172.21.15.112:0/4186792169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd01a1410 con 0x7f4bd0106ca0 2024-03-16T09:05:36.305 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.686+0000 7f4bcd7fa700 1 -- 172.21.15.112:0/4186792169 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4bc001f030 con 0x7f4bd0106ca0 2024-03-16T09:05:36.305 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.686+0000 7f4bcd7fa700 1 -- 172.21.15.112:0/4186792169 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4bc00193f0 con 0x7f4bd0106ca0 2024-03-16T09:05:36.305 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.687+0000 7f4bd6780700 1 -- 172.21.15.112:0/4186792169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd0063900 con 0x7f4bd0106ca0 2024-03-16T09:05:36.305 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.689+0000 7f4bcd7fa700 1 -- 172.21.15.112:0/4186792169 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4bc0015070 con 0x7f4bd0106ca0 2024-03-16T09:05:36.305 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.755+0000 7f4bd6780700 1 -- 172.21.15.112:0/4186792169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f4bd01930a0 con 0x7f4bd0106ca0 2024-03-16T09:05:36.305 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.756+0000 7f4bcd7fa700 1 -- 172.21.15.112:0/4186792169 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f4bd01930a0 con 0x7f4bd0106ca0 2024-03-16T09:05:36.305 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.757+0000 7f4bd6780700 1 -- 172.21.15.112:0/4186792169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0106ca0 msgr2=0x7f4bd019df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:36.305 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.757+0000 7f4bd6780700 1 --2- 172.21.15.112:0/4186792169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0106ca0 0x7f4bd019df50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000b340 tx=0x7f4bc00041b0 comp rx=0 tx=0).stop 2024-03-16T09:05:36.305 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.757+0000 7f4bd6780700 1 -- 172.21.15.112:0/4186792169 shutdown_connections 2024-03-16T09:05:36.305 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.757+0000 7f4bd6780700 1 --2- 172.21.15.112:0/4186792169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd0106ca0 0x7f4bd019df50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:36.305 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.757+0000 7f4bd6780700 1 -- 172.21.15.112:0/4186792169 >> 172.21.15.112:0/4186792169 conn(0x7f4bd0102220 msgr2=0x7f4bd0190180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:36.306 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.757+0000 7f4bd6780700 1 -- 172.21.15.112:0/4186792169 shutdown_connections 2024-03-16T09:05:36.306 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:35.757+0000 7f4bd6780700 1 -- 172.21.15.112:0/4186792169 wait complete. 2024-03-16T09:05:36.306 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-16T09:05:36.328 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-16T09:05:36.328 INFO:teuthology.orchestra.run.smithi112.stdout:Creating mgr... 2024-03-16T09:05:36.328 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-16T09:05:36.329 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-16T09:05:36.581 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-273be2a4-e374-11ee-95c9-87774f69a715@mgr.x 2024-03-16T09:05:36.582 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to reset failed state of unit ceph-273be2a4-e374-11ee-95c9-87774f69a715@mgr.x.service: Unit ceph-273be2a4-e374-11ee-95c9-87774f69a715@mgr.x.service not loaded. 2024-03-16T09:05:36.724 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-273be2a4-e374-11ee-95c9-87774f69a715.target.wants/ceph-273be2a4-e374-11ee-95c9-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-273be2a4-e374-11ee-95c9-87774f69a715@.service. 2024-03-16T09:05:36.841 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:36 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/4186792169' entity='client.admin' 2024-03-16T09:05:37.250 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:37 smithi112 podman[32526]: 2024-03-16 09:05:37.159559486 +0000 UTC m=+0.318224337 container start f6fd1d82bafdeaf2bbfa913aee3cf7cc7b944a6e83373aa97b34c3af951b8d8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-03-16T09:05:37.250 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:37 smithi112 bash[32526]: f6fd1d82bafdeaf2bbfa913aee3cf7cc7b944a6e83373aa97b34c3af951b8d8f 2024-03-16T09:05:37.263 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T09:05:37.263 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-03-16T09:05:37.270 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T09:05:37.270 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-03-16T09:05:37.271 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-03-16T09:05:37.271 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to enable service . firewalld.service is not available 2024-03-16T09:05:37.279 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T09:05:37.280 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-03-16T09:05:37.287 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T09:05:37.288 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-03-16T09:05:37.288 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-03-16T09:05:37.288 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-16T09:05:37.288 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr to start... 2024-03-16T09:05:37.288 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr... 2024-03-16T09:05:37.533 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:37 smithi112 systemd[1]: Started Ceph mgr.x for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:05:37.808 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:37 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:37.712+0000 7fd0ebd52200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T09:05:38.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:37 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3014340634' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-16T09:05:38.211 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:37 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:37.811+0000 7fd0ebd52200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T09:05:38.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-03-16T09:05:38.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-03-16T09:05:38.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "273be2a4-e374-11ee-95c9-87774f69a715", 2024-03-16T09:05:38.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-03-16T09:05:38.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-16T09:05:38.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-16T09:05:38.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-16T09:05:38.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:38.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-16T09:05:38.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-16T09:05:38.353 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-03-16T09:05:38.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:05:38.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-16T09:05:38.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-03-16T09:05:38.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:05:38.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-16T09:05:38.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-16T09:05:38.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:05:38.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-16T09:05:38.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-16T09:05:38.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:38.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-16T09:05:38.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:05:38.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-16T09:05:38.354 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-16T09:05:38.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-16T09:05:38.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-16T09:05:38.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-16T09:05:38.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-16T09:05:38.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:38.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-16T09:05:38.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-16T09:05:38.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-16T09:05:38.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-16T09:05:38.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-16T09:05:38.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-16T09:05:38.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-16T09:05:38.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-16T09:05:38.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-16T09:05:38.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:38.355 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-16T09:05:38.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:05:38.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-16T09:05:38.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-16T09:05:38.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:38.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-16T09:05:38.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-16T09:05:38.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-16T09:05:38.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-16T09:05:38.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-16T09:05:38.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-16T09:05:38.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-03-16T09:05:38.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:05:38.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T09:05:38.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:38.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-16T09:05:38.356 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:05:38.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-03-16T09:05:30.616443+0000", 2024-03-16T09:05:38.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T09:05:38.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:38.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-16T09:05:38.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-03-16T09:05:38.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.726+0000 7f93a8f0e700 1 Processor -- start 2024-03-16T09:05:38.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.726+0000 7f93a8f0e700 1 -- start start 2024-03-16T09:05:38.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.726+0000 7f93a8f0e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41087e0 0x7f93a410abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:38.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.726+0000 7f93a8f0e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f93a407be90 con 0x7f93a41087e0 2024-03-16T09:05:38.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.727+0000 7f93a259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41087e0 0x7f93a410abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:38.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.727+0000 7f93a259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41087e0 0x7f93a410abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50850/0 (socket says 172.21.15.112:50850) 2024-03-16T09:05:38.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.727+0000 7f93a259c700 1 -- 172.21.15.112:0/3716813453 learned_addr learned my addr 172.21.15.112:0/3716813453 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:38.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.727+0000 7f93a259c700 1 -- 172.21.15.112:0/3716813453 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a410b110 con 0x7f93a41087e0 2024-03-16T09:05:38.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.727+0000 7f93a259c700 1 --2- 172.21.15.112:0/3716813453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41087e0 0x7f93a410abd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f93940098a0 tx=0x7f9394009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9653342f87c3c9f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:38.357 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.728+0000 7f93a159a700 1 -- 172.21.15.112:0/3716813453 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9394004160 con 0x7f93a41087e0 2024-03-16T09:05:38.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.728+0000 7f93a159a700 1 -- 172.21.15.112:0/3716813453 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f9394004300 con 0x7f93a41087e0 2024-03-16T09:05:38.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.728+0000 7f93a159a700 1 -- 172.21.15.112:0/3716813453 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9394011c00 con 0x7f93a41087e0 2024-03-16T09:05:38.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.728+0000 7f93a8f0e700 1 -- 172.21.15.112:0/3716813453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41087e0 msgr2=0x7f93a410abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:38.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.728+0000 7f93a8f0e700 1 --2- 172.21.15.112:0/3716813453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41087e0 0x7f93a410abd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f93940098a0 tx=0x7f9394009bb0 comp rx=0 tx=0).stop 2024-03-16T09:05:38.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.728+0000 7f93a8f0e700 1 -- 172.21.15.112:0/3716813453 shutdown_connections 2024-03-16T09:05:38.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.728+0000 7f93a8f0e700 1 --2- 172.21.15.112:0/3716813453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41087e0 0x7f93a410abd0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:38.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.728+0000 7f93a8f0e700 1 -- 172.21.15.112:0/3716813453 >> 172.21.15.112:0/3716813453 conn(0x7f93a4102230 msgr2=0x7f93a4104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:38.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.728+0000 7f93a8f0e700 1 -- 172.21.15.112:0/3716813453 shutdown_connections 2024-03-16T09:05:38.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.728+0000 7f93a8f0e700 1 -- 172.21.15.112:0/3716813453 wait complete. 2024-03-16T09:05:38.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.729+0000 7f93a8f0e700 1 Processor -- start 2024-03-16T09:05:38.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.729+0000 7f93a8f0e700 1 -- start start 2024-03-16T09:05:38.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.729+0000 7f93a8f0e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41087e0 0x7f93a419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:38.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.729+0000 7f93a8f0e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f93a410c130 con 0x7f93a41087e0 2024-03-16T09:05:38.358 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.729+0000 7f93a259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41087e0 0x7f93a419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:38.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.729+0000 7f93a259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41087e0 0x7f93a419bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50854/0 (socket says 172.21.15.112:50854) 2024-03-16T09:05:38.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.729+0000 7f93a259c700 1 -- 172.21.15.112:0/3014340634 learned_addr learned my addr 172.21.15.112:0/3014340634 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:38.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.730+0000 7f93a259c700 1 -- 172.21.15.112:0/3014340634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a419c270 con 0x7f93a41087e0 2024-03-16T09:05:38.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.730+0000 7f93a259c700 1 --2- 172.21.15.112:0/3014340634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41087e0 0x7f93a419bd30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9394003fb0 tx=0x7f9394011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:38.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.730+0000 7f93937fe700 1 -- 172.21.15.112:0/3014340634 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93940039c0 con 0x7f93a41087e0 2024-03-16T09:05:38.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.730+0000 7f93a8f0e700 1 -- 172.21.15.112:0/3014340634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a419c500 con 0x7f93a41087e0 2024-03-16T09:05:38.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.730+0000 7f93937fe700 1 -- 172.21.15.112:0/3014340634 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f9394003b60 con 0x7f93a41087e0 2024-03-16T09:05:38.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.730+0000 7f93937fe700 1 -- 172.21.15.112:0/3014340634 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f939401a6b0 con 0x7f93a41087e0 2024-03-16T09:05:38.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.730+0000 7f93a8f0e700 1 -- 172.21.15.112:0/3014340634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a419f1f0 con 0x7f93a41087e0 2024-03-16T09:05:38.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.730+0000 7f93937fe700 1 -- 172.21.15.112:0/3014340634 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f939402d3f0 con 0x7f93a41087e0 2024-03-16T09:05:38.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.730+0000 7f93937fe700 1 -- 172.21.15.112:0/3014340634 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9394019740 con 0x7f93a41087e0 2024-03-16T09:05:38.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.731+0000 7f93a8f0e700 1 -- 172.21.15.112:0/3014340634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93a4063900 con 0x7f93a41087e0 2024-03-16T09:05:38.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.735+0000 7f93937fe700 1 -- 172.21.15.112:0/3014340634 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9394003d10 con 0x7f93a41087e0 2024-03-16T09:05:38.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.807+0000 7f93a8f0e700 1 -- 172.21.15.112:0/3014340634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f93a419f820 con 0x7f93a41087e0 2024-03-16T09:05:38.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.807+0000 7f93937fe700 1 -- 172.21.15.112:0/3014340634 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f93940249a0 con 0x7f93a41087e0 2024-03-16T09:05:38.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.808+0000 7f93a8f0e700 1 -- 172.21.15.112:0/3014340634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41087e0 msgr2=0x7f93a419bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:38.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.808+0000 7f93a8f0e700 1 --2- 172.21.15.112:0/3014340634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41087e0 0x7f93a419bd30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9394003fb0 tx=0x7f9394011bd0 comp rx=0 tx=0).stop 2024-03-16T09:05:38.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.808+0000 7f93a8f0e700 1 -- 172.21.15.112:0/3014340634 shutdown_connections 2024-03-16T09:05:38.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.808+0000 7f93a8f0e700 1 --2- 172.21.15.112:0/3014340634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93a41087e0 0x7f93a419bd30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:38.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.809+0000 7f93a8f0e700 1 -- 172.21.15.112:0/3014340634 >> 172.21.15.112:0/3014340634 conn(0x7f93a4102230 msgr2=0x7f93a4102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:38.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.809+0000 7f93a8f0e700 1 -- 172.21.15.112:0/3014340634 shutdown_connections 2024-03-16T09:05:38.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:37.809+0000 7f93a8f0e700 1 -- 172.21.15.112:0/3014340634 wait complete. 2024-03-16T09:05:38.360 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (1/15)... 2024-03-16T09:05:38.594 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:38 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:38.459+0000 7fd0ebd52200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T09:05:38.594 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:38 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:38.549+0000 7fd0ebd52200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T09:05:39.092 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:38 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:38.686+0000 7fd0ebd52200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T09:05:39.092 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:38 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:38.778+0000 7fd0ebd52200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T09:05:39.092 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:38 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:38.858+0000 7fd0ebd52200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T09:05:39.396 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:39 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:39.114+0000 7fd0ebd52200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T09:05:39.396 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:39 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:39.220+0000 7fd0ebd52200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T09:05:39.396 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:39 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:39.303+0000 7fd0ebd52200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T09:05:39.396 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:39 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:39.394+0000 7fd0ebd52200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T09:05:39.842 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:39 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:39.565+0000 7fd0ebd52200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T09:05:40.727 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:40 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:40.513+0000 7fd0ebd52200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T09:05:40.727 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:40 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:40.643+0000 7fd0ebd52200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T09:05:40.989 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:40 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/996382743' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-16T09:05:41.273 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:40 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:40.986+0000 7fd0ebd52200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T09:05:41.274 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:41 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:41.075+0000 7fd0ebd52200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T09:05:41.274 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:41 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:41.271+0000 7fd0ebd52200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:05:41.591 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:41 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:41.459+0000 7fd0ebd52200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T09:05:41.601 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-03-16T09:05:41.602 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-03-16T09:05:41.602 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "273be2a4-e374-11ee-95c9-87774f69a715", 2024-03-16T09:05:41.602 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-03-16T09:05:41.602 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-16T09:05:41.602 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-16T09:05:41.602 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-16T09:05:41.602 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:41.602 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-16T09:05:41.602 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-16T09:05:41.602 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-03-16T09:05:41.602 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:05:41.602 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-16T09:05:41.602 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-03-16T09:05:41.602 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:05:41.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-16T09:05:41.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-16T09:05:41.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:05:41.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-16T09:05:41.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-16T09:05:41.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:41.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-16T09:05:41.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:05:41.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-16T09:05:41.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-16T09:05:41.603 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-16T09:05:41.611 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-16T09:05:41.611 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-16T09:05:41.611 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-16T09:05:41.611 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:41.611 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-16T09:05:41.611 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-16T09:05:41.611 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-16T09:05:41.611 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-16T09:05:41.611 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-16T09:05:41.611 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-16T09:05:41.611 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-16T09:05:41.611 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-16T09:05:41.611 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-16T09:05:41.611 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:41.612 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-16T09:05:41.612 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:05:41.612 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-16T09:05:41.612 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-16T09:05:41.612 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:41.612 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-16T09:05:41.612 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-16T09:05:41.612 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-16T09:05:41.612 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-16T09:05:41.612 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-16T09:05:41.612 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-16T09:05:41.612 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-03-16T09:05:41.612 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:05:41.612 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T09:05:41.612 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:41.612 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-16T09:05:41.612 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:05:41.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-03-16T09:05:30.616443+0000", 2024-03-16T09:05:41.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T09:05:41.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:41.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-16T09:05:41.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-03-16T09:05:41.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.755+0000 7f32a3562700 1 Processor -- start 2024-03-16T09:05:41.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.755+0000 7f32a3562700 1 -- start start 2024-03-16T09:05:41.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.755+0000 7f32a3562700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c106650 0x7f329c106a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:41.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.755+0000 7f32a3562700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f329c07be10 con 0x7f329c106650 2024-03-16T09:05:41.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.755+0000 7f32a12fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c106650 0x7f329c106a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:41.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.755+0000 7f32a12fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c106650 0x7f329c106a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50870/0 (socket says 172.21.15.112:50870) 2024-03-16T09:05:41.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.755+0000 7f32a12fe700 1 -- 172.21.15.112:0/1598533980 learned_addr learned my addr 172.21.15.112:0/1598533980 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:41.613 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.756+0000 7f32a12fe700 1 -- 172.21.15.112:0/1598533980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329c106fb0 con 0x7f329c106650 2024-03-16T09:05:41.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.756+0000 7f32a12fe700 1 --2- 172.21.15.112:0/1598533980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c106650 0x7f329c106a70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f32980098a0 tx=0x7f3298009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ea48a796e818b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:41.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.756+0000 7f3293fff700 1 -- 172.21.15.112:0/1598533980 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3298004160 con 0x7f329c106650 2024-03-16T09:05:41.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.756+0000 7f3293fff700 1 -- 172.21.15.112:0/1598533980 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f3298004300 con 0x7f329c106650 2024-03-16T09:05:41.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.756+0000 7f3293fff700 1 -- 172.21.15.112:0/1598533980 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3298011c00 con 0x7f329c106650 2024-03-16T09:05:41.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.757+0000 7f32a3562700 1 -- 172.21.15.112:0/1598533980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c106650 msgr2=0x7f329c106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:41.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.757+0000 7f32a3562700 1 --2- 172.21.15.112:0/1598533980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c106650 0x7f329c106a70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f32980098a0 tx=0x7f3298009bb0 comp rx=0 tx=0).stop 2024-03-16T09:05:41.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.757+0000 7f32a3562700 1 -- 172.21.15.112:0/1598533980 shutdown_connections 2024-03-16T09:05:41.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.757+0000 7f32a3562700 1 --2- 172.21.15.112:0/1598533980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c106650 0x7f329c106a70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:41.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.757+0000 7f32a3562700 1 -- 172.21.15.112:0/1598533980 >> 172.21.15.112:0/1598533980 conn(0x7f329c1021d0 msgr2=0x7f329c104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:41.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.757+0000 7f32a3562700 1 -- 172.21.15.112:0/1598533980 shutdown_connections 2024-03-16T09:05:41.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.757+0000 7f32a3562700 1 -- 172.21.15.112:0/1598533980 wait complete. 2024-03-16T09:05:41.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.757+0000 7f32a3562700 1 Processor -- start 2024-03-16T09:05:41.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.757+0000 7f32a3562700 1 -- start start 2024-03-16T09:05:41.614 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.757+0000 7f32a3562700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c106650 0x7f329c19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:41.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.758+0000 7f32a3562700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f329c108030 con 0x7f329c106650 2024-03-16T09:05:41.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.758+0000 7f32a12fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c106650 0x7f329c19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:41.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.758+0000 7f32a12fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c106650 0x7f329c19df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50886/0 (socket says 172.21.15.112:50886) 2024-03-16T09:05:41.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.758+0000 7f32a12fe700 1 -- 172.21.15.112:0/996382743 learned_addr learned my addr 172.21.15.112:0/996382743 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:41.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.758+0000 7f32a12fe700 1 -- 172.21.15.112:0/996382743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f329c19e450 con 0x7f329c106650 2024-03-16T09:05:41.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.758+0000 7f32a12fe700 1 --2- 172.21.15.112:0/996382743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c106650 0x7f329c19df10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3298003fb0 tx=0x7f3298011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:41.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.758+0000 7f32927fc700 1 -- 172.21.15.112:0/996382743 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32980039c0 con 0x7f329c106650 2024-03-16T09:05:41.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.758+0000 7f32a3562700 1 -- 172.21.15.112:0/996382743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f329c19e6e0 con 0x7f329c106650 2024-03-16T09:05:41.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.758+0000 7f32927fc700 1 -- 172.21.15.112:0/996382743 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f3298003b60 con 0x7f329c106650 2024-03-16T09:05:41.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.758+0000 7f32927fc700 1 -- 172.21.15.112:0/996382743 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f329801a6b0 con 0x7f329c106650 2024-03-16T09:05:41.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.758+0000 7f32a3562700 1 -- 172.21.15.112:0/996382743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f329c1a13d0 con 0x7f329c106650 2024-03-16T09:05:41.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.758+0000 7f32927fc700 1 -- 172.21.15.112:0/996382743 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f329802d3f0 con 0x7f329c106650 2024-03-16T09:05:41.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.759+0000 7f32927fc700 1 -- 172.21.15.112:0/996382743 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3298019740 con 0x7f329c106650 2024-03-16T09:05:41.615 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.759+0000 7f32a3562700 1 -- 172.21.15.112:0/996382743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f329c063900 con 0x7f329c106650 2024-03-16T09:05:41.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.761+0000 7f32927fc700 1 -- 172.21.15.112:0/996382743 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3298003d10 con 0x7f329c106650 2024-03-16T09:05:41.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.834+0000 7f32a3562700 1 -- 172.21.15.112:0/996382743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f329c02dd70 con 0x7f329c106650 2024-03-16T09:05:41.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.835+0000 7f32927fc700 1 -- 172.21.15.112:0/996382743 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f32980249a0 con 0x7f329c106650 2024-03-16T09:05:41.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.836+0000 7f32a3562700 1 -- 172.21.15.112:0/996382743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c106650 msgr2=0x7f329c19df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:41.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.836+0000 7f32a3562700 1 --2- 172.21.15.112:0/996382743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c106650 0x7f329c19df10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3298003fb0 tx=0x7f3298011bd0 comp rx=0 tx=0).stop 2024-03-16T09:05:41.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.836+0000 7f32a3562700 1 -- 172.21.15.112:0/996382743 shutdown_connections 2024-03-16T09:05:41.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.836+0000 7f32a3562700 1 --2- 172.21.15.112:0/996382743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f329c106650 0x7f329c19df10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:41.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.836+0000 7f32a3562700 1 -- 172.21.15.112:0/996382743 >> 172.21.15.112:0/996382743 conn(0x7f329c1021d0 msgr2=0x7f329c194bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:41.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.836+0000 7f32a3562700 1 -- 172.21.15.112:0/996382743 shutdown_connections 2024-03-16T09:05:41.616 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:40.836+0000 7f32a3562700 1 -- 172.21.15.112:0/996382743 wait complete. 2024-03-16T09:05:41.616 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (2/15)... 2024-03-16T09:05:41.994 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:41 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:41.738+0000 7fd0ebd52200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:05:41.995 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:41 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:41.991+0000 7fd0ebd52200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T09:05:42.342 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:42 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:42.222+0000 7fd0ebd52200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T09:05:43.342 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:43 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:43.012+0000 7fd0ebd52200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T09:05:43.842 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:43 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:43.574+0000 7fd0ebd52200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T09:05:43.842 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:43 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:43.674+0000 7fd0ebd52200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T09:05:44.342 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:43 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:43.892+0000 7fd0ebd52200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T09:05:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:43 smithi112 ceph-mon[32307]: Activating manager daemon x 2024-03-16T09:05:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:43 smithi112 ceph-mon[32307]: mgrmap e2: x(active, starting, since 0.00319887s) 2024-03-16T09:05:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:43 smithi112 ceph-mon[32307]: from='mgr.14100 172.21.15.112:0/494693397' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T09:05:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:43 smithi112 ceph-mon[32307]: from='mgr.14100 172.21.15.112:0/494693397' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T09:05:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:43 smithi112 ceph-mon[32307]: from='mgr.14100 172.21.15.112:0/494693397' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T09:05:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:43 smithi112 ceph-mon[32307]: from='mgr.14100 172.21.15.112:0/494693397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:05:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:43 smithi112 ceph-mon[32307]: from='mgr.14100 172.21.15.112:0/494693397' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-16T09:05:44.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:43 smithi112 ceph-mon[32307]: Manager daemon x is now available 2024-03-16T09:05:44.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:43 smithi112 ceph-mon[32307]: from='mgr.14100 172.21.15.112:0/494693397' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T09:05:44.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:43 smithi112 ceph-mon[32307]: from='mgr.14100 172.21.15.112:0/494693397' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T09:05:44.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:43 smithi112 ceph-mon[32307]: from='mgr.14100 172.21.15.112:0/494693397' entity='mgr.x' 2024-03-16T09:05:44.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:43 smithi112 ceph-mon[32307]: from='mgr.14100 172.21.15.112:0/494693397' entity='mgr.x' 2024-03-16T09:05:44.767 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-03-16T09:05:44.767 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-03-16T09:05:44.767 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "273be2a4-e374-11ee-95c9-87774f69a715", 2024-03-16T09:05:44.767 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-03-16T09:05:44.768 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-16T09:05:44.768 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-16T09:05:44.768 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-16T09:05:44.768 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:44.768 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-16T09:05:44.768 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-16T09:05:44.768 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-03-16T09:05:44.768 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:05:44.768 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-16T09:05:44.768 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-03-16T09:05:44.768 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:05:44.768 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-16T09:05:44.768 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-16T09:05:44.768 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:05:44.768 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-16T09:05:44.768 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-16T09:05:44.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:44.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-16T09:05:44.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:05:44.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-16T09:05:44.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-16T09:05:44.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-16T09:05:44.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-16T09:05:44.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-16T09:05:44.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-16T09:05:44.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:44.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-16T09:05:44.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-16T09:05:44.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-16T09:05:44.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-16T09:05:44.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-16T09:05:44.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-16T09:05:44.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-16T09:05:44.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-16T09:05:44.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-16T09:05:44.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:44.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-16T09:05:44.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:05:44.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-16T09:05:44.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-16T09:05:44.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:44.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-16T09:05:44.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-16T09:05:44.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-16T09:05:44.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-16T09:05:44.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-16T09:05:44.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-16T09:05:44.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-03-16T09:05:44.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:05:44.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T09:05:44.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:44.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-16T09:05:44.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:05:44.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-03-16T09:05:30.616443+0000", 2024-03-16T09:05:44.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T09:05:44.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:44.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-16T09:05:44.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-03-16T09:05:44.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.030+0000 7fa42c2ed700 1 Processor -- start 2024-03-16T09:05:44.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.030+0000 7fa42c2ed700 1 -- start start 2024-03-16T09:05:44.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.030+0000 7fa42c2ed700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424106630 0x7fa424106a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:44.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.030+0000 7fa42c2ed700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa42407be30 con 0x7fa424106630 2024-03-16T09:05:44.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.031+0000 7fa42a089700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424106630 0x7fa424106a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:44.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.031+0000 7fa42a089700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424106630 0x7fa424106a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40456/0 (socket says 172.21.15.112:40456) 2024-03-16T09:05:44.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.031+0000 7fa42a089700 1 -- 172.21.15.112:0/2620793813 learned_addr learned my addr 172.21.15.112:0/2620793813 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:44.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.031+0000 7fa42a089700 1 -- 172.21.15.112:0/2620793813 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa424106f90 con 0x7fa424106630 2024-03-16T09:05:44.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.031+0000 7fa42a089700 1 --2- 172.21.15.112:0/2620793813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424106630 0x7fa424106a50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0098a0 tx=0x7fa41c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28475cf62970761 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:44.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.031+0000 7fa429087700 1 -- 172.21.15.112:0/2620793813 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa41c004160 con 0x7fa424106630 2024-03-16T09:05:44.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.031+0000 7fa429087700 1 -- 172.21.15.112:0/2620793813 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fa41c004300 con 0x7fa424106630 2024-03-16T09:05:44.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.031+0000 7fa429087700 1 -- 172.21.15.112:0/2620793813 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa41c011c00 con 0x7fa424106630 2024-03-16T09:05:44.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.032+0000 7fa42c2ed700 1 -- 172.21.15.112:0/2620793813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424106630 msgr2=0x7fa424106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:44.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.032+0000 7fa42c2ed700 1 --2- 172.21.15.112:0/2620793813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424106630 0x7fa424106a50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0098a0 tx=0x7fa41c009bb0 comp rx=0 tx=0).stop 2024-03-16T09:05:44.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.032+0000 7fa42c2ed700 1 -- 172.21.15.112:0/2620793813 shutdown_connections 2024-03-16T09:05:44.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.032+0000 7fa42c2ed700 1 --2- 172.21.15.112:0/2620793813 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424106630 0x7fa424106a50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:44.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.032+0000 7fa42c2ed700 1 -- 172.21.15.112:0/2620793813 >> 172.21.15.112:0/2620793813 conn(0x7fa424102230 msgr2=0x7fa424104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:44.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.032+0000 7fa42c2ed700 1 -- 172.21.15.112:0/2620793813 shutdown_connections 2024-03-16T09:05:44.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.032+0000 7fa42c2ed700 1 -- 172.21.15.112:0/2620793813 wait complete. 2024-03-16T09:05:44.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.033+0000 7fa42c2ed700 1 Processor -- start 2024-03-16T09:05:44.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.033+0000 7fa42c2ed700 1 -- start start 2024-03-16T09:05:44.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.033+0000 7fa42c2ed700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424106630 0x7fa42419df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:44.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.033+0000 7fa42c2ed700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa424108010 con 0x7fa424106630 2024-03-16T09:05:44.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.033+0000 7fa42a089700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424106630 0x7fa42419df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:44.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.033+0000 7fa42a089700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424106630 0x7fa42419df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40466/0 (socket says 172.21.15.112:40466) 2024-03-16T09:05:44.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.033+0000 7fa42a089700 1 -- 172.21.15.112:0/2602317890 learned_addr learned my addr 172.21.15.112:0/2602317890 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:44.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.033+0000 7fa42a089700 1 -- 172.21.15.112:0/2602317890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42419e470 con 0x7fa424106630 2024-03-16T09:05:44.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.033+0000 7fa42a089700 1 --2- 172.21.15.112:0/2602317890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424106630 0x7fa42419df30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa41c003fb0 tx=0x7fa41c011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:44.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.033+0000 7fa4237fe700 1 -- 172.21.15.112:0/2602317890 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa41c0039c0 con 0x7fa424106630 2024-03-16T09:05:44.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.033+0000 7fa42c2ed700 1 -- 172.21.15.112:0/2602317890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42419e700 con 0x7fa424106630 2024-03-16T09:05:44.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.033+0000 7fa4237fe700 1 -- 172.21.15.112:0/2602317890 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fa41c003b60 con 0x7fa424106630 2024-03-16T09:05:44.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.033+0000 7fa4237fe700 1 -- 172.21.15.112:0/2602317890 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa41c01a6b0 con 0x7fa424106630 2024-03-16T09:05:44.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.033+0000 7fa42c2ed700 1 -- 172.21.15.112:0/2602317890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4241a13f0 con 0x7fa424106630 2024-03-16T09:05:44.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.034+0000 7fa42c2ed700 1 -- 172.21.15.112:0/2602317890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa424063900 con 0x7fa424106630 2024-03-16T09:05:44.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.034+0000 7fa4237fe700 1 -- 172.21.15.112:0/2602317890 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 2) ==== 45862+0+0 (secure 0 0 0) 0x7fa41c024430 con 0x7fa424106630 2024-03-16T09:05:44.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.035+0000 7fa4237fe700 1 -- 172.21.15.112:0/2602317890 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa41c04ce20 con 0x7fa424106630 2024-03-16T09:05:44.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.037+0000 7fa4237fe700 1 -- 172.21.15.112:0/2602317890 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa41c003d10 con 0x7fa424106630 2024-03-16T09:05:44.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.107+0000 7fa42c2ed700 1 -- 172.21.15.112:0/2602317890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa4240008d0 con 0x7fa424106630 2024-03-16T09:05:44.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.107+0000 7fa4237fe700 1 -- 172.21.15.112:0/2602317890 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa41c024720 con 0x7fa424106630 2024-03-16T09:05:44.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.108+0000 7fa42c2ed700 1 -- 172.21.15.112:0/2602317890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424106630 msgr2=0x7fa42419df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:44.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.108+0000 7fa42c2ed700 1 --2- 172.21.15.112:0/2602317890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424106630 0x7fa42419df30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa41c003fb0 tx=0x7fa41c011bd0 comp rx=0 tx=0).stop 2024-03-16T09:05:44.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.108+0000 7fa42c2ed700 1 -- 172.21.15.112:0/2602317890 shutdown_connections 2024-03-16T09:05:44.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.108+0000 7fa42c2ed700 1 --2- 172.21.15.112:0/2602317890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424106630 0x7fa42419df30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:44.775 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.108+0000 7fa42c2ed700 1 -- 172.21.15.112:0/2602317890 >> 172.21.15.112:0/2602317890 conn(0x7fa424102230 msgr2=0x7fa424194b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:44.775 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.108+0000 7fa42c2ed700 1 -- 172.21.15.112:0/2602317890 shutdown_connections 2024-03-16T09:05:44.775 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:44.108+0000 7fa42c2ed700 1 -- 172.21.15.112:0/2602317890 wait complete. 2024-03-16T09:05:44.775 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (3/15)... 2024-03-16T09:05:45.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:44 smithi112 ceph-mon[32307]: from='mgr.14100 172.21.15.112:0/494693397' entity='mgr.x' 2024-03-16T09:05:45.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:44 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2602317890' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-16T09:05:45.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:44 smithi112 ceph-mon[32307]: mgrmap e3: x(active, since 1.00667s) 2024-03-16T09:05:47.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:46 smithi112 ceph-mon[32307]: mgrmap e4: x(active, since 2s) 2024-03-16T09:05:50.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:49 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1562521591' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-16T09:05:55.493 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-03-16T09:05:55.493 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-03-16T09:05:55.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "273be2a4-e374-11ee-95c9-87774f69a715", 2024-03-16T09:05:55.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-03-16T09:05:55.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-16T09:05:55.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-16T09:05:55.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-16T09:05:55.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:55.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-16T09:05:55.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-16T09:05:55.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-03-16T09:05:55.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:05:55.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-16T09:05:55.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-03-16T09:05:55.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:05:55.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-16T09:05:55.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-16T09:05:55.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:05:55.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-16T09:05:55.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-16T09:05:55.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:55.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-16T09:05:55.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:05:55.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-16T09:05:55.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-16T09:05:55.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-16T09:05:55.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-16T09:05:55.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-16T09:05:55.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-16T09:05:55.495 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:55.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-16T09:05:55.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-16T09:05:55.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-16T09:05:55.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-16T09:05:55.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-16T09:05:55.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-16T09:05:55.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-16T09:05:55.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-16T09:05:55.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-16T09:05:55.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:55.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-16T09:05:55.496 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:05:55.497 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-16T09:05:55.497 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-16T09:05:55.497 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:55.497 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-16T09:05:55.497 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-16T09:05:55.497 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-16T09:05:55.497 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-16T09:05:55.497 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-16T09:05:55.497 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-16T09:05:55.497 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-03-16T09:05:55.497 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-03-16T09:05:55.497 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T09:05:55.498 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:55.498 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-16T09:05:55.498 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-16T09:05:55.498 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-03-16T09:05:30.616443+0000", 2024-03-16T09:05:55.498 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-16T09:05:55.498 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-03-16T09:05:55.498 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-16T09:05:55.498 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-03-16T09:05:55.498 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.475+0000 7fb91f1ad700 1 Processor -- start 2024-03-16T09:05:55.498 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.475+0000 7fb91f1ad700 1 -- start start 2024-03-16T09:05:55.498 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.475+0000 7fb91f1ad700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918108820 0x7fb91810ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:55.498 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.475+0000 7fb91f1ad700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb91807be50 con 0x7fb918108820 2024-03-16T09:05:55.498 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.475+0000 7fb91cf49700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918108820 0x7fb91810ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:55.498 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.475+0000 7fb91cf49700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918108820 0x7fb91810ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40468/0 (socket says 172.21.15.112:40468) 2024-03-16T09:05:55.499 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.475+0000 7fb91cf49700 1 -- 172.21.15.112:0/1218222484 learned_addr learned my addr 172.21.15.112:0/1218222484 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:55.499 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.475+0000 7fb91cf49700 1 -- 172.21.15.112:0/1218222484 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91810b150 con 0x7fb918108820 2024-03-16T09:05:55.499 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.475+0000 7fb91cf49700 1 --2- 172.21.15.112:0/1218222484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918108820 0x7fb91810ac10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0098a0 tx=0x7fb90c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa73301d5728c8a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:55.499 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.476+0000 7fb9177fe700 1 -- 172.21.15.112:0/1218222484 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb90c004160 con 0x7fb918108820 2024-03-16T09:05:55.499 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.476+0000 7fb9177fe700 1 -- 172.21.15.112:0/1218222484 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fb90c004300 con 0x7fb918108820 2024-03-16T09:05:55.499 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.476+0000 7fb9177fe700 1 -- 172.21.15.112:0/1218222484 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb90c011c00 con 0x7fb918108820 2024-03-16T09:05:55.499 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.476+0000 7fb91f1ad700 1 -- 172.21.15.112:0/1218222484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918108820 msgr2=0x7fb91810ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:55.499 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.476+0000 7fb91f1ad700 1 --2- 172.21.15.112:0/1218222484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918108820 0x7fb91810ac10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0098a0 tx=0x7fb90c009bb0 comp rx=0 tx=0).stop 2024-03-16T09:05:55.499 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.476+0000 7fb91f1ad700 1 -- 172.21.15.112:0/1218222484 shutdown_connections 2024-03-16T09:05:55.499 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.476+0000 7fb91f1ad700 1 --2- 172.21.15.112:0/1218222484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918108820 0x7fb91810ac10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:55.499 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.476+0000 7fb91f1ad700 1 -- 172.21.15.112:0/1218222484 >> 172.21.15.112:0/1218222484 conn(0x7fb918102230 msgr2=0x7fb918104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:55.499 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.476+0000 7fb91f1ad700 1 -- 172.21.15.112:0/1218222484 shutdown_connections 2024-03-16T09:05:55.499 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.476+0000 7fb91f1ad700 1 -- 172.21.15.112:0/1218222484 wait complete. 2024-03-16T09:05:55.499 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.477+0000 7fb91f1ad700 1 Processor -- start 2024-03-16T09:05:55.500 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.477+0000 7fb91f1ad700 1 -- start start 2024-03-16T09:05:55.500 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.477+0000 7fb91f1ad700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918108820 0x7fb91819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:55.500 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.477+0000 7fb91f1ad700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb91810c160 con 0x7fb918108820 2024-03-16T09:05:55.500 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.477+0000 7fb91cf49700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918108820 0x7fb91819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:55.500 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.477+0000 7fb91cf49700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918108820 0x7fb91819bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40474/0 (socket says 172.21.15.112:40474) 2024-03-16T09:05:55.500 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.477+0000 7fb91cf49700 1 -- 172.21.15.112:0/1562521591 learned_addr learned my addr 172.21.15.112:0/1562521591 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:55.500 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.477+0000 7fb91cf49700 1 -- 172.21.15.112:0/1562521591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91819c270 con 0x7fb918108820 2024-03-16T09:05:55.500 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.477+0000 7fb91cf49700 1 --2- 172.21.15.112:0/1562521591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918108820 0x7fb91819bd30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb90c003fb0 tx=0x7fb90c011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:55.500 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.478+0000 7fb915ffb700 1 -- 172.21.15.112:0/1562521591 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb90c0039c0 con 0x7fb918108820 2024-03-16T09:05:55.500 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.478+0000 7fb915ffb700 1 -- 172.21.15.112:0/1562521591 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fb90c003b60 con 0x7fb918108820 2024-03-16T09:05:55.500 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.478+0000 7fb91f1ad700 1 -- 172.21.15.112:0/1562521591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91819c500 con 0x7fb918108820 2024-03-16T09:05:55.500 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.478+0000 7fb915ffb700 1 -- 172.21.15.112:0/1562521591 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb90c01a6b0 con 0x7fb918108820 2024-03-16T09:05:55.500 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.478+0000 7fb91f1ad700 1 -- 172.21.15.112:0/1562521591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb91819f1f0 con 0x7fb918108820 2024-03-16T09:05:55.500 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.479+0000 7fb915ffb700 1 -- 172.21.15.112:0/1562521591 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7fb90c01a960 con 0x7fb918108820 2024-03-16T09:05:55.501 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.479+0000 7fb91f1ad700 1 -- 172.21.15.112:0/1562521591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb918063900 con 0x7fb918108820 2024-03-16T09:05:55.501 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.479+0000 7fb915ffb700 1 --2- 172.21.15.112:0/1562521591 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7fb908038ed0 0x7fb90803b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:55.501 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.479+0000 7fb915ffb700 1 -- 172.21.15.112:0/1562521591 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb90c04d280 con 0x7fb918108820 2024-03-16T09:05:55.501 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.479+0000 7fb917fff700 1 --2- 172.21.15.112:0/1562521591 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7fb908038ed0 0x7fb90803b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:55.501 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.479+0000 7fb917fff700 1 --2- 172.21.15.112:0/1562521591 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7fb908038ed0 0x7fb90803b390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb904006fd0 tx=0x7fb904006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:55.501 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.484+0000 7fb915ffb700 1 -- 172.21.15.112:0/1562521591 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb90c019c70 con 0x7fb918108820 2024-03-16T09:05:55.501 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.760+0000 7fb91f1ad700 1 -- 172.21.15.112:0/1562521591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb91819f3c0 con 0x7fb918108820 2024-03-16T09:05:55.501 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.760+0000 7fb915ffb700 1 -- 172.21.15.112:0/1562521591 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb91819f3c0 con 0x7fb918108820 2024-03-16T09:05:55.501 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.763+0000 7fb91f1ad700 1 -- 172.21.15.112:0/1562521591 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7fb908038ed0 msgr2=0x7fb90803b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:55.501 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.763+0000 7fb91f1ad700 1 --2- 172.21.15.112:0/1562521591 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7fb908038ed0 0x7fb90803b390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb904006fd0 tx=0x7fb904006e40 comp rx=0 tx=0).stop 2024-03-16T09:05:55.501 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.763+0000 7fb91f1ad700 1 -- 172.21.15.112:0/1562521591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918108820 msgr2=0x7fb91819bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:55.501 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.763+0000 7fb91f1ad700 1 --2- 172.21.15.112:0/1562521591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918108820 0x7fb91819bd30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb90c003fb0 tx=0x7fb90c011bd0 comp rx=0 tx=0).stop 2024-03-16T09:05:55.501 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.763+0000 7fb91f1ad700 1 -- 172.21.15.112:0/1562521591 shutdown_connections 2024-03-16T09:05:55.502 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.763+0000 7fb91f1ad700 1 --2- 172.21.15.112:0/1562521591 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7fb908038ed0 0x7fb90803b390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:55.502 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.763+0000 7fb91f1ad700 1 --2- 172.21.15.112:0/1562521591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb918108820 0x7fb91819bd30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:55.502 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.763+0000 7fb91f1ad700 1 -- 172.21.15.112:0/1562521591 >> 172.21.15.112:0/1562521591 conn(0x7fb918102230 msgr2=0x7fb918102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:55.502 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.763+0000 7fb91f1ad700 1 -- 172.21.15.112:0/1562521591 shutdown_connections 2024-03-16T09:05:55.502 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:49.763+0000 7fb91f1ad700 1 -- 172.21.15.112:0/1562521591 wait complete. 2024-03-16T09:05:55.502 INFO:teuthology.orchestra.run.smithi112.stdout:mgr is available 2024-03-16T09:05:56.410 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:56 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2513209880' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-16T09:05:56.711 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-03-16T09:05:56.712 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [global] 2024-03-16T09:05:56.712 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout fsid = 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:05:56.712 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-16T09:05:56.712 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-16T09:05:56.712 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-16T09:05:56.712 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-16T09:05:56.712 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-03-16T09:05:56.712 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-16T09:05:56.712 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-16T09:05:56.712 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-03-16T09:05:56.712 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [osd] 2024-03-16T09:05:56.712 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-16T09:05:56.712 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-16T09:05:56.712 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-16T09:05:56.713 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-16T09:05:56.713 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-16T09:05:56.713 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.899+0000 7f5e77ca4700 1 Processor -- start 2024-03-16T09:05:56.713 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.899+0000 7f5e77ca4700 1 -- start start 2024-03-16T09:05:56.713 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.899+0000 7f5e77ca4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e70108820 0x7f5e7010ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:56.713 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.899+0000 7f5e77ca4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7007be50 con 0x7f5e70108820 2024-03-16T09:05:56.713 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.899+0000 7f5e75a40700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e70108820 0x7f5e7010ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:56.713 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.899+0000 7f5e75a40700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e70108820 0x7f5e7010ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59408/0 (socket says 172.21.15.112:59408) 2024-03-16T09:05:56.713 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.899+0000 7f5e75a40700 1 -- 172.21.15.112:0/4002546805 learned_addr learned my addr 172.21.15.112:0/4002546805 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:56.713 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.900+0000 7f5e75a40700 1 -- 172.21.15.112:0/4002546805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7010b150 con 0x7f5e70108820 2024-03-16T09:05:56.713 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.900+0000 7f5e75a40700 1 --2- 172.21.15.112:0/4002546805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e70108820 0x7f5e7010ac10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0098a0 tx=0x7f5e6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57acf2ecd84ee00a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:56.713 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.900+0000 7f5e74a3e700 1 -- 172.21.15.112:0/4002546805 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e6c004160 con 0x7f5e70108820 2024-03-16T09:05:56.713 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.900+0000 7f5e74a3e700 1 -- 172.21.15.112:0/4002546805 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f5e6c004300 con 0x7f5e70108820 2024-03-16T09:05:56.713 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.900+0000 7f5e74a3e700 1 -- 172.21.15.112:0/4002546805 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e6c011c00 con 0x7f5e70108820 2024-03-16T09:05:56.714 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.900+0000 7f5e77ca4700 1 -- 172.21.15.112:0/4002546805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e70108820 msgr2=0x7f5e7010ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:56.714 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.900+0000 7f5e77ca4700 1 --2- 172.21.15.112:0/4002546805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e70108820 0x7f5e7010ac10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0098a0 tx=0x7f5e6c009bb0 comp rx=0 tx=0).stop 2024-03-16T09:05:56.714 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.900+0000 7f5e77ca4700 1 -- 172.21.15.112:0/4002546805 shutdown_connections 2024-03-16T09:05:56.714 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.900+0000 7f5e77ca4700 1 --2- 172.21.15.112:0/4002546805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e70108820 0x7f5e7010ac10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:56.714 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.900+0000 7f5e77ca4700 1 -- 172.21.15.112:0/4002546805 >> 172.21.15.112:0/4002546805 conn(0x7f5e701021d0 msgr2=0x7f5e70104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:56.714 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.901+0000 7f5e77ca4700 1 -- 172.21.15.112:0/4002546805 shutdown_connections 2024-03-16T09:05:56.714 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.901+0000 7f5e77ca4700 1 -- 172.21.15.112:0/4002546805 wait complete. 2024-03-16T09:05:56.714 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.901+0000 7f5e77ca4700 1 Processor -- start 2024-03-16T09:05:56.714 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.901+0000 7f5e77ca4700 1 -- start start 2024-03-16T09:05:56.714 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.901+0000 7f5e77ca4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e70108820 0x7f5e7019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:56.714 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.901+0000 7f5e77ca4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7010c160 con 0x7f5e70108820 2024-03-16T09:05:56.714 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.901+0000 7f5e75a40700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e70108820 0x7f5e7019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:56.714 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.901+0000 7f5e75a40700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e70108820 0x7f5e7019bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59414/0 (socket says 172.21.15.112:59414) 2024-03-16T09:05:56.715 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.901+0000 7f5e75a40700 1 -- 172.21.15.112:0/2513209880 learned_addr learned my addr 172.21.15.112:0/2513209880 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:56.715 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.902+0000 7f5e75a40700 1 -- 172.21.15.112:0/2513209880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7019c210 con 0x7f5e70108820 2024-03-16T09:05:56.715 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.902+0000 7f5e75a40700 1 --2- 172.21.15.112:0/2513209880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e70108820 0x7f5e7019bcd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c003fb0 tx=0x7f5e6c011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:56.715 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.902+0000 7f5e66ffd700 1 -- 172.21.15.112:0/2513209880 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e6c0039c0 con 0x7f5e70108820 2024-03-16T09:05:56.715 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.902+0000 7f5e66ffd700 1 -- 172.21.15.112:0/2513209880 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f5e6c003b60 con 0x7f5e70108820 2024-03-16T09:05:56.715 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.902+0000 7f5e66ffd700 1 -- 172.21.15.112:0/2513209880 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e6c01a6b0 con 0x7f5e70108820 2024-03-16T09:05:56.715 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.902+0000 7f5e77ca4700 1 -- 172.21.15.112:0/2513209880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e7019c4a0 con 0x7f5e70108820 2024-03-16T09:05:56.715 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.902+0000 7f5e77ca4700 1 -- 172.21.15.112:0/2513209880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e7019f180 con 0x7f5e70108820 2024-03-16T09:05:56.715 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.903+0000 7f5e66ffd700 1 -- 172.21.15.112:0/2513209880 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7f5e6c024430 con 0x7f5e70108820 2024-03-16T09:05:56.715 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.903+0000 7f5e77ca4700 1 -- 172.21.15.112:0/2513209880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e70063900 con 0x7f5e70108820 2024-03-16T09:05:56.715 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.903+0000 7f5e66ffd700 1 --2- 172.21.15.112:0/2513209880 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7f5e5c038e80 0x7f5e5c03b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:56.715 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.903+0000 7f5e66ffd700 1 -- 172.21.15.112:0/2513209880 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5e6c04d150 con 0x7f5e70108820 2024-03-16T09:05:56.715 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.903+0000 7f5e7523f700 1 --2- 172.21.15.112:0/2513209880 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7f5e5c038e80 0x7f5e5c03b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:56.715 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.904+0000 7f5e7523f700 1 --2- 172.21.15.112:0/2513209880 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7f5e5c038e80 0x7f5e5c03b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5e60006fd0 tx=0x7f5e60006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:55.908+0000 7f5e66ffd700 1 -- 172.21.15.112:0/2513209880 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5e6c003d10 con 0x7f5e70108820 2024-03-16T09:05:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:56.103+0000 7f5e77ca4700 1 -- 172.21.15.112:0/2513209880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f5e7019f350 con 0x7f5e70108820 2024-03-16T09:05:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:56.104+0000 7f5e66ffd700 1 -- 172.21.15.112:0/2513209880 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f5e7019f350 con 0x7f5e70108820 2024-03-16T09:05:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:56.106+0000 7f5e77ca4700 1 -- 172.21.15.112:0/2513209880 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7f5e5c038e80 msgr2=0x7f5e5c03b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:56.106+0000 7f5e77ca4700 1 --2- 172.21.15.112:0/2513209880 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7f5e5c038e80 0x7f5e5c03b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5e60006fd0 tx=0x7f5e60006e40 comp rx=0 tx=0).stop 2024-03-16T09:05:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:56.106+0000 7f5e77ca4700 1 -- 172.21.15.112:0/2513209880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e70108820 msgr2=0x7f5e7019bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:56.106+0000 7f5e77ca4700 1 --2- 172.21.15.112:0/2513209880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e70108820 0x7f5e7019bcd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c003fb0 tx=0x7f5e6c011bd0 comp rx=0 tx=0).stop 2024-03-16T09:05:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:56.106+0000 7f5e77ca4700 1 -- 172.21.15.112:0/2513209880 shutdown_connections 2024-03-16T09:05:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:56.106+0000 7f5e77ca4700 1 --2- 172.21.15.112:0/2513209880 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7f5e5c038e80 0x7f5e5c03b340 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:56.106+0000 7f5e77ca4700 1 --2- 172.21.15.112:0/2513209880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e70108820 0x7f5e7019bcd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:56.106+0000 7f5e77ca4700 1 -- 172.21.15.112:0/2513209880 >> 172.21.15.112:0/2513209880 conn(0x7f5e701021d0 msgr2=0x7f5e70102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:56.106+0000 7f5e77ca4700 1 -- 172.21.15.112:0/2513209880 shutdown_connections 2024-03-16T09:05:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:56.106+0000 7f5e77ca4700 1 -- 172.21.15.112:0/2513209880 wait complete. 2024-03-16T09:05:56.716 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling cephadm module... 2024-03-16T09:05:57.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:57 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/4200950057' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-16T09:05:57.842 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:57 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: ignoring --setuser ceph since I am not root 2024-03-16T09:05:57.842 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:57 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: ignoring --setgroup ceph since I am not root 2024-03-16T09:05:58.342 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:57 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:57.949+0000 7f6798c2e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T09:05:58.342 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:58 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:58.040+0000 7f6798c2e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T09:05:58.676 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:58 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/4200950057' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-16T09:05:58.676 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:58 smithi112 ceph-mon[32307]: mgrmap e5: x(active, since 13s) 2024-03-16T09:05:58.726 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.087+0000 7fd6fef75700 1 Processor -- start 2024-03-16T09:05:58.726 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.087+0000 7fd6fef75700 1 -- start start 2024-03-16T09:05:58.726 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.087+0000 7fd6fef75700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8106ca0 0x7fd6f81070c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:58.726 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.087+0000 7fd6fef75700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f8107690 con 0x7fd6f8106ca0 2024-03-16T09:05:58.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.088+0000 7fd6fcd11700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8106ca0 0x7fd6f81070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:58.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.088+0000 7fd6fcd11700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8106ca0 0x7fd6f81070c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59424/0 (socket says 172.21.15.112:59424) 2024-03-16T09:05:58.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.088+0000 7fd6fcd11700 1 -- 172.21.15.112:0/4079330225 learned_addr learned my addr 172.21.15.112:0/4079330225 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:58.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.088+0000 7fd6fcd11700 1 -- 172.21.15.112:0/4079330225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f8107ea0 con 0x7fd6f8106ca0 2024-03-16T09:05:58.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.088+0000 7fd6fcd11700 1 --2- 172.21.15.112:0/4079330225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8106ca0 0x7fd6f81070c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800d180 tx=0x7fd6e800d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139c5776abf3c4e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:58.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.088+0000 7fd6f77fe700 1 -- 172.21.15.112:0/4079330225 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6e8019070 con 0x7fd6f8106ca0 2024-03-16T09:05:58.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.088+0000 7fd6f77fe700 1 -- 172.21.15.112:0/4079330225 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fd6e800de20 con 0x7fd6f8106ca0 2024-03-16T09:05:58.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.088+0000 7fd6f77fe700 1 -- 172.21.15.112:0/4079330225 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6e8009c30 con 0x7fd6f8106ca0 2024-03-16T09:05:58.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.089+0000 7fd6fef75700 1 -- 172.21.15.112:0/4079330225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8106ca0 msgr2=0x7fd6f81070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:58.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.089+0000 7fd6fef75700 1 --2- 172.21.15.112:0/4079330225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8106ca0 0x7fd6f81070c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800d180 tx=0x7fd6e800d490 comp rx=0 tx=0).stop 2024-03-16T09:05:58.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.089+0000 7fd6fef75700 1 -- 172.21.15.112:0/4079330225 shutdown_connections 2024-03-16T09:05:58.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.089+0000 7fd6fef75700 1 --2- 172.21.15.112:0/4079330225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8106ca0 0x7fd6f81070c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:58.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.089+0000 7fd6fef75700 1 -- 172.21.15.112:0/4079330225 >> 172.21.15.112:0/4079330225 conn(0x7fd6f8102220 msgr2=0x7fd6f8104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:58.727 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.089+0000 7fd6fef75700 1 -- 172.21.15.112:0/4079330225 shutdown_connections 2024-03-16T09:05:58.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.089+0000 7fd6fef75700 1 -- 172.21.15.112:0/4079330225 wait complete. 2024-03-16T09:05:58.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.089+0000 7fd6fef75700 1 Processor -- start 2024-03-16T09:05:58.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.089+0000 7fd6fef75700 1 -- start start 2024-03-16T09:05:58.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.090+0000 7fd6fef75700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8106ca0 0x7fd6f8199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:58.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.090+0000 7fd6fef75700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f807afb0 con 0x7fd6f8106ca0 2024-03-16T09:05:58.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.090+0000 7fd6fcd11700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8106ca0 0x7fd6f8199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:58.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.090+0000 7fd6fcd11700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8106ca0 0x7fd6f8199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59428/0 (socket says 172.21.15.112:59428) 2024-03-16T09:05:58.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.090+0000 7fd6fcd11700 1 -- 172.21.15.112:0/4200950057 learned_addr learned my addr 172.21.15.112:0/4200950057 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:05:58.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.090+0000 7fd6fcd11700 1 -- 172.21.15.112:0/4200950057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f819a080 con 0x7fd6f8106ca0 2024-03-16T09:05:58.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.090+0000 7fd6fcd11700 1 --2- 172.21.15.112:0/4200950057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8106ca0 0x7fd6f8199b40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009640 tx=0x7fd6e80070b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:58.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.090+0000 7fd6f5ffb700 1 -- 172.21.15.112:0/4200950057 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6e8019070 con 0x7fd6f8106ca0 2024-03-16T09:05:58.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.090+0000 7fd6f5ffb700 1 -- 172.21.15.112:0/4200950057 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fd6e801a040 con 0x7fd6f8106ca0 2024-03-16T09:05:58.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.090+0000 7fd6fef75700 1 -- 172.21.15.112:0/4200950057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f819a310 con 0x7fd6f8106ca0 2024-03-16T09:05:58.728 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.090+0000 7fd6f5ffb700 1 -- 172.21.15.112:0/4200950057 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6e8004470 con 0x7fd6f8106ca0 2024-03-16T09:05:58.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.091+0000 7fd6fef75700 1 -- 172.21.15.112:0/4200950057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f819d000 con 0x7fd6f8106ca0 2024-03-16T09:05:58.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.091+0000 7fd6f5ffb700 1 -- 172.21.15.112:0/4200950057 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7fd6e8004720 con 0x7fd6f8106ca0 2024-03-16T09:05:58.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.091+0000 7fd6fef75700 1 -- 172.21.15.112:0/4200950057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f8063900 con 0x7fd6f8106ca0 2024-03-16T09:05:58.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.092+0000 7fd6f5ffb700 1 --2- 172.21.15.112:0/4200950057 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7fd6e0038ed0 0x7fd6e003b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:05:58.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.092+0000 7fd6f5ffb700 1 -- 172.21.15.112:0/4200950057 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd6e8011050 con 0x7fd6f8106ca0 2024-03-16T09:05:58.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.092+0000 7fd6f7fff700 1 --2- 172.21.15.112:0/4200950057 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7fd6e0038ed0 0x7fd6e003b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:05:58.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.092+0000 7fd6f7fff700 1 --2- 172.21.15.112:0/4200950057 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7fd6e0038ed0 0x7fd6e003b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec006fd0 tx=0x7fd6ec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:05:58.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.096+0000 7fd6f5ffb700 1 -- 172.21.15.112:0/4200950057 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd6e801d360 con 0x7fd6f8106ca0 2024-03-16T09:05:58.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.348+0000 7fd6fef75700 1 -- 172.21.15.112:0/4200950057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fd6f802df00 con 0x7fd6f8106ca0 2024-03-16T09:05:58.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.400+0000 7fd6f5ffb700 1 -- 172.21.15.112:0/4200950057 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fd6e804eea0 con 0x7fd6f8106ca0 2024-03-16T09:05:58.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.402+0000 7fd6f5ffb700 1 -- 172.21.15.112:0/4200950057 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7fd6e8027ca0 con 0x7fd6f8106ca0 2024-03-16T09:05:58.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.403+0000 7fd6fef75700 1 -- 172.21.15.112:0/4200950057 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7fd6e0038ed0 msgr2=0x7fd6e003b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:58.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.403+0000 7fd6fef75700 1 --2- 172.21.15.112:0/4200950057 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7fd6e0038ed0 0x7fd6e003b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec006fd0 tx=0x7fd6ec006e40 comp rx=0 tx=0).stop 2024-03-16T09:05:58.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.403+0000 7fd6fef75700 1 -- 172.21.15.112:0/4200950057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8106ca0 msgr2=0x7fd6f8199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:05:58.729 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.403+0000 7fd6fef75700 1 --2- 172.21.15.112:0/4200950057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8106ca0 0x7fd6f8199b40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009640 tx=0x7fd6e80070b0 comp rx=0 tx=0).stop 2024-03-16T09:05:58.730 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.403+0000 7fd6fef75700 1 -- 172.21.15.112:0/4200950057 shutdown_connections 2024-03-16T09:05:58.730 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.403+0000 7fd6fef75700 1 --2- 172.21.15.112:0/4200950057 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7fd6e0038ed0 0x7fd6e003b390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:58.730 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.403+0000 7fd6fef75700 1 --2- 172.21.15.112:0/4200950057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f8106ca0 0x7fd6f8199b40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:05:58.730 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.403+0000 7fd6fef75700 1 -- 172.21.15.112:0/4200950057 >> 172.21.15.112:0/4200950057 conn(0x7fd6f8102220 msgr2=0x7fd6f8190670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:05:58.730 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.404+0000 7fd6fef75700 1 -- 172.21.15.112:0/4200950057 shutdown_connections 2024-03-16T09:05:58.730 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:57.404+0000 7fd6fef75700 1 -- 172.21.15.112:0/4200950057 wait complete. 2024-03-16T09:05:58.962 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:58 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:58.674+0000 7f6798c2e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T09:05:58.962 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:58 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:58.773+0000 7f6798c2e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T09:05:58.962 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:58 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:58.914+0000 7f6798c2e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T09:05:59.342 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:59 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:59.013+0000 7f6798c2e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T09:05:59.342 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:59 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:59.095+0000 7f6798c2e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T09:05:59.672 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:05:59 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/334177485' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-16T09:05:59.672 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:59 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:59.368+0000 7f6798c2e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T09:05:59.672 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:59 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:59.489+0000 7f6798c2e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T09:05:59.672 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:59 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:59.577+0000 7f6798c2e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T09:05:59.672 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:59 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:59.670+0000 7f6798c2e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T09:06:00.033 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:05:59 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:05:59.846+0000 7f6798c2e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T09:06:00.041 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-03-16T09:06:00.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-16T09:06:00.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-16T09:06:00.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-16T09:06:00.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-16T09:06:00.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-03-16T09:06:00.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.184+0000 7f4a4059e700 1 Processor -- start 2024-03-16T09:06:00.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.184+0000 7f4a4059e700 1 -- start start 2024-03-16T09:06:00.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.184+0000 7f4a4059e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a38106cc0 0x7f4a381070e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:00.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.184+0000 7f4a4059e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a381076b0 con 0x7f4a38106cc0 2024-03-16T09:06:00.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.185+0000 7f4a3e33a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a38106cc0 0x7f4a381070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:00.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.185+0000 7f4a3e33a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a38106cc0 0x7f4a381070e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59452/0 (socket says 172.21.15.112:59452) 2024-03-16T09:06:00.042 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.185+0000 7f4a3e33a700 1 -- 172.21.15.112:0/1491714156 learned_addr learned my addr 172.21.15.112:0/1491714156 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:00.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.185+0000 7f4a3e33a700 1 -- 172.21.15.112:0/1491714156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a38107ec0 con 0x7f4a38106cc0 2024-03-16T09:06:00.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.185+0000 7f4a3e33a700 1 --2- 172.21.15.112:0/1491714156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a38106cc0 0x7f4a381070e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800bb40 tx=0x7f4a2800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923acd4c5c0fdf27 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:00.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.185+0000 7f4a3d338700 1 -- 172.21.15.112:0/1491714156 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a2800dbe0 con 0x7f4a38106cc0 2024-03-16T09:06:00.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.185+0000 7f4a3d338700 1 -- 172.21.15.112:0/1491714156 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f4a28004510 con 0x7f4a38106cc0 2024-03-16T09:06:00.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.185+0000 7f4a3d338700 1 -- 172.21.15.112:0/1491714156 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a280104d0 con 0x7f4a38106cc0 2024-03-16T09:06:00.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.186+0000 7f4a4059e700 1 -- 172.21.15.112:0/1491714156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a38106cc0 msgr2=0x7f4a381070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:00.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.186+0000 7f4a4059e700 1 --2- 172.21.15.112:0/1491714156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a38106cc0 0x7f4a381070e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800bb40 tx=0x7f4a2800be50 comp rx=0 tx=0).stop 2024-03-16T09:06:00.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.186+0000 7f4a4059e700 1 -- 172.21.15.112:0/1491714156 shutdown_connections 2024-03-16T09:06:00.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.186+0000 7f4a4059e700 1 --2- 172.21.15.112:0/1491714156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a38106cc0 0x7f4a381070e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:00.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.186+0000 7f4a4059e700 1 -- 172.21.15.112:0/1491714156 >> 172.21.15.112:0/1491714156 conn(0x7f4a38102260 msgr2=0x7f4a381046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:00.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.186+0000 7f4a4059e700 1 -- 172.21.15.112:0/1491714156 shutdown_connections 2024-03-16T09:06:00.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.186+0000 7f4a4059e700 1 -- 172.21.15.112:0/1491714156 wait complete. 2024-03-16T09:06:00.043 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.187+0000 7f4a4059e700 1 Processor -- start 2024-03-16T09:06:00.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.187+0000 7f4a4059e700 1 -- start start 2024-03-16T09:06:00.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.187+0000 7f4a4059e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a38106cc0 0x7f4a3819df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:00.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.187+0000 7f4a4059e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3807afb0 con 0x7f4a38106cc0 2024-03-16T09:06:00.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.187+0000 7f4a3e33a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a38106cc0 0x7f4a3819df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:00.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.187+0000 7f4a3e33a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a38106cc0 0x7f4a3819df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59456/0 (socket says 172.21.15.112:59456) 2024-03-16T09:06:00.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.187+0000 7f4a3e33a700 1 -- 172.21.15.112:0/334177485 learned_addr learned my addr 172.21.15.112:0/334177485 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:00.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.187+0000 7f4a3e33a700 1 -- 172.21.15.112:0/334177485 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3819e480 con 0x7f4a38106cc0 2024-03-16T09:06:00.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.187+0000 7f4a3e33a700 1 --2- 172.21.15.112:0/334177485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a38106cc0 0x7f4a3819df40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800b340 tx=0x7f4a280041b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:00.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.187+0000 7f4a2f7fe700 1 -- 172.21.15.112:0/334177485 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a2800de20 con 0x7f4a38106cc0 2024-03-16T09:06:00.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.187+0000 7f4a4059e700 1 -- 172.21.15.112:0/334177485 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3819e710 con 0x7f4a38106cc0 2024-03-16T09:06:00.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.187+0000 7f4a2f7fe700 1 -- 172.21.15.112:0/334177485 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f4a2801d070 con 0x7f4a38106cc0 2024-03-16T09:06:00.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.187+0000 7f4a2f7fe700 1 -- 172.21.15.112:0/334177485 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a28021d50 con 0x7f4a38106cc0 2024-03-16T09:06:00.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.188+0000 7f4a4059e700 1 -- 172.21.15.112:0/334177485 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a381a1400 con 0x7f4a38106cc0 2024-03-16T09:06:00.044 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.188+0000 7f4a2f7fe700 1 -- 172.21.15.112:0/334177485 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7f4a2801f030 con 0x7f4a38106cc0 2024-03-16T09:06:00.045 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.188+0000 7f4a4059e700 1 -- 172.21.15.112:0/334177485 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a38063900 con 0x7f4a38106cc0 2024-03-16T09:06:00.045 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.189+0000 7f4a2f7fe700 1 --2- 172.21.15.112:0/334177485 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7f4a240390c0 0x7f4a2403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:00.045 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.189+0000 7f4a2f7fe700 1 -- 172.21.15.112:0/334177485 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4a2804d0f0 con 0x7f4a38106cc0 2024-03-16T09:06:00.045 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.189+0000 7f4a3db39700 1 -- 172.21.15.112:0/334177485 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7f4a240390c0 msgr2=0x7f4a2403b580 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1790808883 2024-03-16T09:06:00.045 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.189+0000 7f4a3db39700 1 --2- 172.21.15.112:0/334177485 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7f4a240390c0 0x7f4a2403b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:06:00.045 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.198+0000 7f4a2f7fe700 1 -- 172.21.15.112:0/334177485 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4a28015070 con 0x7f4a38106cc0 2024-03-16T09:06:00.045 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.390+0000 7f4a3db39700 1 -- 172.21.15.112:0/334177485 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7f4a240390c0 msgr2=0x7f4a2403b580 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1790808883 2024-03-16T09:06:00.045 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.390+0000 7f4a3db39700 1 --2- 172.21.15.112:0/334177485 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7f4a240390c0 0x7f4a2403b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T09:06:00.045 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.453+0000 7f4a4059e700 1 -- 172.21.15.112:0/334177485 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f4a3802df40 con 0x7f4a38106cc0 2024-03-16T09:06:00.045 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.454+0000 7f4a2f7fe700 1 -- 172.21.15.112:0/334177485 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f4a28026210 con 0x7f4a38106cc0 2024-03-16T09:06:00.045 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.456+0000 7f4a4059e700 1 -- 172.21.15.112:0/334177485 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7f4a240390c0 msgr2=0x7f4a2403b580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T09:06:00.045 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.456+0000 7f4a4059e700 1 --2- 172.21.15.112:0/334177485 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7f4a240390c0 0x7f4a2403b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:00.045 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.456+0000 7f4a4059e700 1 -- 172.21.15.112:0/334177485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a38106cc0 msgr2=0x7f4a3819df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:00.045 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.456+0000 7f4a4059e700 1 --2- 172.21.15.112:0/334177485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a38106cc0 0x7f4a3819df40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800b340 tx=0x7f4a280041b0 comp rx=0 tx=0).stop 2024-03-16T09:06:00.046 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.456+0000 7f4a4059e700 1 -- 172.21.15.112:0/334177485 shutdown_connections 2024-03-16T09:06:00.046 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.456+0000 7f4a4059e700 1 --2- 172.21.15.112:0/334177485 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7f4a240390c0 0x7f4a2403b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:00.046 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.456+0000 7f4a4059e700 1 --2- 172.21.15.112:0/334177485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a38106cc0 0x7f4a3819df40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:00.046 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.456+0000 7f4a4059e700 1 -- 172.21.15.112:0/334177485 >> 172.21.15.112:0/334177485 conn(0x7f4a38102260 msgr2=0x7f4a38190690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:00.046 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.456+0000 7f4a4059e700 1 -- 172.21.15.112:0/334177485 shutdown_connections 2024-03-16T09:06:00.046 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:05:59.456+0000 7f4a4059e700 1 -- 172.21.15.112:0/334177485 wait complete. 2024-03-16T09:06:00.046 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for the mgr to restart... 2024-03-16T09:06:00.046 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr epoch 5... 2024-03-16T09:06:00.947 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:00 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:00.817+0000 7f6798c2e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T09:06:01.273 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:00 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:00.945+0000 7f6798c2e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T09:06:01.273 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:01 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:01.271+0000 7f6798c2e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T09:06:01.553 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:01 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:01.358+0000 7f6798c2e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T09:06:01.553 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:01 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:01.551+0000 7f6798c2e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:06:01.842 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:01 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:01.733+0000 7f6798c2e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T09:06:02.271 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:02 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:02.015+0000 7f6798c2e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:06:02.271 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:02 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:02.269+0000 7f6798c2e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T09:06:02.592 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:02 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:02.507+0000 7f6798c2e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T09:06:03.592 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:03 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:03.297+0000 7f6798c2e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T09:06:04.174 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:03 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:03.859+0000 7f6798c2e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T09:06:04.175 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:03 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:03.957+0000 7f6798c2e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T09:06:04.175 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:04 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:04.172+0000 7f6798c2e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T09:06:04.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:04 smithi112 ceph-mon[32307]: Active manager daemon x restarted 2024-03-16T09:06:04.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:04 smithi112 ceph-mon[32307]: Activating manager daemon x 2024-03-16T09:06:04.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:04 smithi112 ceph-mon[32307]: osdmap e2: 0 total, 0 up, 0 in 2024-03-16T09:06:04.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:04 smithi112 ceph-mon[32307]: mgrmap e6: x(active, starting, since 0.00330281s) 2024-03-16T09:06:04.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:04 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:06:04.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:04 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-16T09:06:04.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:04 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T09:06:04.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:04 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T09:06:04.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:04 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T09:06:04.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:04 smithi112 ceph-mon[32307]: Manager daemon x is now available 2024-03-16T09:06:04.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:04 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' 2024-03-16T09:06:04.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:04 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' 2024-03-16T09:06:04.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:04 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:04.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:04 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:04.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:04 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:05.489 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:05 smithi112 ceph-mon[32307]: Found migration_current of "None". Setting to last migration. 2024-03-16T09:06:05.489 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:05 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T09:06:05.489 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:05 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T09:06:05.489 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:05 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' 2024-03-16T09:06:05.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:05 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' 2024-03-16T09:06:05.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:05 smithi112 ceph-mon[32307]: mgrmap e7: x(active, since 1.00776s) 2024-03-16T09:06:05.783 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-03-16T09:06:05.783 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-16T09:06:05.783 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-16T09:06:05.783 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-03-16T09:06:05.783 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.462+0000 7ff42786e700 1 Processor -- start 2024-03-16T09:06:05.783 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.462+0000 7ff42786e700 1 -- start start 2024-03-16T09:06:05.783 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.462+0000 7ff42786e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42007ddc0 0x7ff42007c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:05.784 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.462+0000 7ff42786e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff42007c800 con 0x7ff42007ddc0 2024-03-16T09:06:05.784 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.462+0000 7ff42560a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42007ddc0 0x7ff42007c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:05.784 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.463+0000 7ff42560a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42007ddc0 0x7ff42007c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59458/0 (socket says 172.21.15.112:59458) 2024-03-16T09:06:05.784 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.463+0000 7ff42560a700 1 -- 172.21.15.112:0/1811388510 learned_addr learned my addr 172.21.15.112:0/1811388510 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:05.784 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.463+0000 7ff42560a700 1 -- 172.21.15.112:0/1811388510 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42007c980 con 0x7ff42007ddc0 2024-03-16T09:06:05.784 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.463+0000 7ff42560a700 1 --2- 172.21.15.112:0/1811388510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42007ddc0 0x7ff42007c2c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff4100098a0 tx=0x7ff410009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cbec959f049046 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:05.784 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.463+0000 7ff417fff700 1 -- 172.21.15.112:0/1811388510 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff410004160 con 0x7ff42007ddc0 2024-03-16T09:06:05.784 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.463+0000 7ff417fff700 1 -- 172.21.15.112:0/1811388510 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7ff410004300 con 0x7ff42007ddc0 2024-03-16T09:06:05.784 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.463+0000 7ff417fff700 1 -- 172.21.15.112:0/1811388510 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff410011c00 con 0x7ff42007ddc0 2024-03-16T09:06:05.784 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.464+0000 7ff42786e700 1 -- 172.21.15.112:0/1811388510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42007ddc0 msgr2=0x7ff42007c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:05.784 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.464+0000 7ff42786e700 1 --2- 172.21.15.112:0/1811388510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42007ddc0 0x7ff42007c2c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff4100098a0 tx=0x7ff410009bb0 comp rx=0 tx=0).stop 2024-03-16T09:06:05.784 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.464+0000 7ff42786e700 1 -- 172.21.15.112:0/1811388510 shutdown_connections 2024-03-16T09:06:05.784 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.464+0000 7ff42786e700 1 --2- 172.21.15.112:0/1811388510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42007ddc0 0x7ff42007c2c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:05.785 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.464+0000 7ff42786e700 1 -- 172.21.15.112:0/1811388510 >> 172.21.15.112:0/1811388510 conn(0x7ff4201033c0 msgr2=0x7ff4201057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:05.785 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.464+0000 7ff42786e700 1 -- 172.21.15.112:0/1811388510 shutdown_connections 2024-03-16T09:06:05.785 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.464+0000 7ff42786e700 1 -- 172.21.15.112:0/1811388510 wait complete. 2024-03-16T09:06:05.785 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.464+0000 7ff42786e700 1 Processor -- start 2024-03-16T09:06:05.785 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.464+0000 7ff42786e700 1 -- start start 2024-03-16T09:06:05.785 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.465+0000 7ff42786e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42007ddc0 0x7ff42019def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:05.785 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.465+0000 7ff42786e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff42010e310 con 0x7ff42007ddc0 2024-03-16T09:06:05.785 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.465+0000 7ff42560a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42007ddc0 0x7ff42019def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:05.785 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.465+0000 7ff42560a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42007ddc0 0x7ff42019def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59464/0 (socket says 172.21.15.112:59464) 2024-03-16T09:06:05.785 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.465+0000 7ff42560a700 1 -- 172.21.15.112:0/2497261384 learned_addr learned my addr 172.21.15.112:0/2497261384 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:05.785 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.465+0000 7ff42560a700 1 -- 172.21.15.112:0/2497261384 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42019e430 con 0x7ff42007ddc0 2024-03-16T09:06:05.785 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.465+0000 7ff42560a700 1 --2- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42007ddc0 0x7ff42019def0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff410011bd0 tx=0x7ff410011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:05.785 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.465+0000 7ff4167fc700 1 -- 172.21.15.112:0/2497261384 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff410003b20 con 0x7ff42007ddc0 2024-03-16T09:06:05.785 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.465+0000 7ff4167fc700 1 -- 172.21.15.112:0/2497261384 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7ff410003cc0 con 0x7ff42007ddc0 2024-03-16T09:06:05.785 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.465+0000 7ff42786e700 1 -- 172.21.15.112:0/2497261384 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42019e6c0 con 0x7ff42007ddc0 2024-03-16T09:06:05.786 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.465+0000 7ff4167fc700 1 -- 172.21.15.112:0/2497261384 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff410024930 con 0x7ff42007ddc0 2024-03-16T09:06:05.786 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.465+0000 7ff42786e700 1 -- 172.21.15.112:0/2497261384 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4201a13b0 con 0x7ff42007ddc0 2024-03-16T09:06:05.786 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.466+0000 7ff4167fc700 1 -- 172.21.15.112:0/2497261384 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7ff410022020 con 0x7ff42007ddc0 2024-03-16T09:06:05.786 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.466+0000 7ff4167fc700 1 --2- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7ff40c038f70 0x7ff40c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:05.786 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.466+0000 7ff42786e700 1 -- 172.21.15.112:0/2497261384 --> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff420063900 con 0x7ff40c038f70 2024-03-16T09:06:05.786 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.466+0000 7ff4167fc700 1 -- 172.21.15.112:0/2497261384 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff41004cb40 con 0x7ff42007ddc0 2024-03-16T09:06:05.786 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.466+0000 7ff424e09700 1 -- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7ff40c038f70 msgr2=0x7ff40c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1790808883 2024-03-16T09:06:05.786 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.466+0000 7ff424e09700 1 --2- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7ff40c038f70 0x7ff40c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:06:05.786 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.667+0000 7ff424e09700 1 -- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7ff40c038f70 msgr2=0x7ff40c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1790808883 2024-03-16T09:06:05.786 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:00.667+0000 7ff424e09700 1 --2- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7ff40c038f70 0x7ff40c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T09:06:05.786 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:01.068+0000 7ff424e09700 1 -- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7ff40c038f70 msgr2=0x7ff40c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1790808883 2024-03-16T09:06:05.786 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:01.068+0000 7ff424e09700 1 --2- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7ff40c038f70 0x7ff40c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-16T09:06:05.786 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:01.869+0000 7ff424e09700 1 -- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7ff40c038f70 msgr2=0x7ff40c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1790808883 2024-03-16T09:06:05.787 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:01.869+0000 7ff424e09700 1 --2- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7ff40c038f70 0x7ff40c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-16T09:06:05.787 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:03.470+0000 7ff424e09700 1 -- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7ff40c038f70 msgr2=0x7ff40c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1790808883 2024-03-16T09:06:05.787 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:03.470+0000 7ff424e09700 1 --2- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7ff40c038f70 0x7ff40c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-16T09:06:05.787 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:04.188+0000 7ff4167fc700 1 -- 172.21.15.112:0/2497261384 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mgrmap(e 6) ==== 45873+0+0 (secure 0 0 0) 0x7ff41001aca0 con 0x7ff42007ddc0 2024-03-16T09:06:05.787 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:04.188+0000 7ff4167fc700 1 -- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7ff40c038f70 msgr2=0x7ff40c03b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T09:06:05.787 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:04.188+0000 7ff4167fc700 1 --2- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7ff40c038f70 0x7ff40c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:05.787 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:05.191+0000 7ff4167fc700 1 -- 172.21.15.112:0/2497261384 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 7) ==== 46000+0+0 (secure 0 0 0) 0x7ff41004e6f0 con 0x7ff42007ddc0 2024-03-16T09:06:05.787 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:05.192+0000 7ff4167fc700 1 --2- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7ff40c03c670 0x7ff40c03ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:05.787 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:05.192+0000 7ff4167fc700 1 -- 172.21.15.112:0/2497261384 --> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff410001d30 con 0x7ff40c03c670 2024-03-16T09:06:05.787 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:05.192+0000 7ff424e09700 1 --2- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7ff40c03c670 0x7ff40c03ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:05.787 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:05.192+0000 7ff424e09700 1 --2- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7ff40c03c670 0x7ff40c03ea60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff41c003a10 tx=0x7ff41c0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:05.787 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:05.193+0000 7ff4167fc700 1 -- 172.21.15.112:0/2497261384 <== mgr.14120 v2:172.21.15.112:6800/2629104617 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7ff410001d30 con 0x7ff40c03c670 2024-03-16T09:06:05.787 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:05.199+0000 7ff42786e700 1 -- 172.21.15.112:0/2497261384 --> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff420104660 con 0x7ff40c03c670 2024-03-16T09:06:05.787 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:05.200+0000 7ff4167fc700 1 -- 172.21.15.112:0/2497261384 <== mgr.14120 v2:172.21.15.112:6800/2629104617 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7ff420104660 con 0x7ff40c03c670 2024-03-16T09:06:05.788 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:05.200+0000 7ff42786e700 1 -- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7ff40c03c670 msgr2=0x7ff40c03ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:05.788 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:05.200+0000 7ff42786e700 1 --2- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7ff40c03c670 0x7ff40c03ea60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff41c003a10 tx=0x7ff41c0092b0 comp rx=0 tx=0).stop 2024-03-16T09:06:05.788 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:05.200+0000 7ff42786e700 1 -- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42007ddc0 msgr2=0x7ff42019def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:05.788 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:05.200+0000 7ff42786e700 1 --2- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42007ddc0 0x7ff42019def0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff410011bd0 tx=0x7ff410011cb0 comp rx=0 tx=0).stop 2024-03-16T09:06:05.788 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:05.200+0000 7ff42786e700 1 -- 172.21.15.112:0/2497261384 shutdown_connections 2024-03-16T09:06:05.788 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:05.200+0000 7ff42786e700 1 --2- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7ff40c03c670 0x7ff40c03ea60 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:05.788 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:05.200+0000 7ff42786e700 1 --2- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42007ddc0 0x7ff42019def0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:05.788 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:05.200+0000 7ff42786e700 1 --2- 172.21.15.112:0/2497261384 >> [v2:172.21.15.112:6800/1790808883,v1:172.21.15.112:6801/1790808883] conn(0x7ff40c038f70 0x7ff40c03b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:05.788 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:05.200+0000 7ff42786e700 1 -- 172.21.15.112:0/2497261384 >> 172.21.15.112:0/2497261384 conn(0x7ff4201033c0 msgr2=0x7ff420103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:05.788 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:05.200+0000 7ff42786e700 1 -- 172.21.15.112:0/2497261384 shutdown_connections 2024-03-16T09:06:05.788 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:05.200+0000 7ff42786e700 1 -- 172.21.15.112:0/2497261384 wait complete. 2024-03-16T09:06:05.788 INFO:teuthology.orchestra.run.smithi112.stdout:mgr epoch 5 is available 2024-03-16T09:06:05.788 INFO:teuthology.orchestra.run.smithi112.stdout:Setting orchestrator backend to cephadm... 2024-03-16T09:06:06.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:06 smithi112 ceph-mon[32307]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-16T09:06:06.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:06 smithi112 ceph-mon[32307]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-16T09:06:06.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:06 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:07.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.248+0000 7fdc84231700 1 Processor -- start 2024-03-16T09:06:07.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.249+0000 7fdc84231700 1 -- start start 2024-03-16T09:06:07.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.249+0000 7fdc84231700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c108870 0x7fdc7c10ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:07.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.249+0000 7fdc84231700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c07be50 con 0x7fdc7c108870 2024-03-16T09:06:07.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.249+0000 7fdc81fcd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c108870 0x7fdc7c10ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:07.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.249+0000 7fdc81fcd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c108870 0x7fdc7c10ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38668/0 (socket says 172.21.15.112:38668) 2024-03-16T09:06:07.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.249+0000 7fdc81fcd700 1 -- 172.21.15.112:0/1378596165 learned_addr learned my addr 172.21.15.112:0/1378596165 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:07.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.249+0000 7fdc81fcd700 1 -- 172.21.15.112:0/1378596165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7c10b1a0 con 0x7fdc7c108870 2024-03-16T09:06:07.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.249+0000 7fdc81fcd700 1 --2- 172.21.15.112:0/1378596165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c108870 0x7fdc7c10ac60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdc780098a0 tx=0x7fdc78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799780f4acca6aca server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:07.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.252+0000 7fdc80fcb700 1 -- 172.21.15.112:0/1378596165 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc78004160 con 0x7fdc7c108870 2024-03-16T09:06:07.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.252+0000 7fdc80fcb700 1 -- 172.21.15.112:0/1378596165 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fdc78004300 con 0x7fdc7c108870 2024-03-16T09:06:07.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.252+0000 7fdc80fcb700 1 -- 172.21.15.112:0/1378596165 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc78011c00 con 0x7fdc7c108870 2024-03-16T09:06:07.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.252+0000 7fdc84231700 1 -- 172.21.15.112:0/1378596165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c108870 msgr2=0x7fdc7c10ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:07.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.252+0000 7fdc84231700 1 --2- 172.21.15.112:0/1378596165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c108870 0x7fdc7c10ac60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdc780098a0 tx=0x7fdc78009bb0 comp rx=0 tx=0).stop 2024-03-16T09:06:07.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.252+0000 7fdc84231700 1 -- 172.21.15.112:0/1378596165 shutdown_connections 2024-03-16T09:06:07.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.252+0000 7fdc84231700 1 --2- 172.21.15.112:0/1378596165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c108870 0x7fdc7c10ac60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:07.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.252+0000 7fdc84231700 1 -- 172.21.15.112:0/1378596165 >> 172.21.15.112:0/1378596165 conn(0x7fdc7c102260 msgr2=0x7fdc7c1046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:07.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.252+0000 7fdc84231700 1 -- 172.21.15.112:0/1378596165 shutdown_connections 2024-03-16T09:06:07.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.252+0000 7fdc84231700 1 -- 172.21.15.112:0/1378596165 wait complete. 2024-03-16T09:06:07.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.253+0000 7fdc84231700 1 Processor -- start 2024-03-16T09:06:07.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.253+0000 7fdc84231700 1 -- start start 2024-03-16T09:06:07.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.253+0000 7fdc84231700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c108870 0x7fdc7c19bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:07.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.253+0000 7fdc84231700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c10c1b0 con 0x7fdc7c108870 2024-03-16T09:06:07.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.253+0000 7fdc81fcd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c108870 0x7fdc7c19bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:07.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.253+0000 7fdc81fcd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c108870 0x7fdc7c19bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38670/0 (socket says 172.21.15.112:38670) 2024-03-16T09:06:07.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.253+0000 7fdc81fcd700 1 -- 172.21.15.112:0/1580031067 learned_addr learned my addr 172.21.15.112:0/1580031067 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:07.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.253+0000 7fdc81fcd700 1 -- 172.21.15.112:0/1580031067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7c19c2a0 con 0x7fdc7c108870 2024-03-16T09:06:07.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.254+0000 7fdc81fcd700 1 --2- 172.21.15.112:0/1580031067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c108870 0x7fdc7c19bd60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdc78011bd0 tx=0x7fdc78011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:07.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.254+0000 7fdc72ffd700 1 -- 172.21.15.112:0/1580031067 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc78003b20 con 0x7fdc7c108870 2024-03-16T09:06:07.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.254+0000 7fdc72ffd700 1 -- 172.21.15.112:0/1580031067 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fdc78003cc0 con 0x7fdc7c108870 2024-03-16T09:06:07.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.254+0000 7fdc84231700 1 -- 172.21.15.112:0/1580031067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7c19c530 con 0x7fdc7c108870 2024-03-16T09:06:07.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.254+0000 7fdc72ffd700 1 -- 172.21.15.112:0/1580031067 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc78024930 con 0x7fdc7c108870 2024-03-16T09:06:07.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.254+0000 7fdc84231700 1 -- 172.21.15.112:0/1580031067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7c19f220 con 0x7fdc7c108870 2024-03-16T09:06:07.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.255+0000 7fdc72ffd700 1 -- 172.21.15.112:0/1580031067 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fdc78022020 con 0x7fdc7c108870 2024-03-16T09:06:07.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.255+0000 7fdc84231700 1 -- 172.21.15.112:0/1580031067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc7c063900 con 0x7fdc7c108870 2024-03-16T09:06:07.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.255+0000 7fdc72ffd700 1 --2- 172.21.15.112:0/1580031067 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7fdc68038f70 0x7fdc6803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:07.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.255+0000 7fdc72ffd700 1 -- 172.21.15.112:0/1580031067 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdc7804ce90 con 0x7fdc7c108870 2024-03-16T09:06:07.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.255+0000 7fdc817cc700 1 --2- 172.21.15.112:0/1580031067 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7fdc68038f70 0x7fdc6803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:07.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.255+0000 7fdc817cc700 1 --2- 172.21.15.112:0/1580031067 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7fdc68038f70 0x7fdc6803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c006fd0 tx=0x7fdc6c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:07.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.260+0000 7fdc72ffd700 1 -- 172.21.15.112:0/1580031067 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdc780172b0 con 0x7fdc7c108870 2024-03-16T09:06:07.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.472+0000 7fdc84231700 1 -- 172.21.15.112:0/1580031067 --> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fdc7c1034b0 con 0x7fdc68038f70 2024-03-16T09:06:07.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.480+0000 7fdc72ffd700 1 -- 172.21.15.112:0/1580031067 <== mgr.14120 v2:172.21.15.112:6800/2629104617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdc7c1034b0 con 0x7fdc68038f70 2024-03-16T09:06:07.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.483+0000 7fdc84231700 1 -- 172.21.15.112:0/1580031067 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7fdc68038f70 msgr2=0x7fdc6803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:07.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.483+0000 7fdc84231700 1 --2- 172.21.15.112:0/1580031067 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7fdc68038f70 0x7fdc6803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c006fd0 tx=0x7fdc6c006e40 comp rx=0 tx=0).stop 2024-03-16T09:06:07.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.483+0000 7fdc84231700 1 -- 172.21.15.112:0/1580031067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c108870 msgr2=0x7fdc7c19bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:07.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.483+0000 7fdc84231700 1 --2- 172.21.15.112:0/1580031067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c108870 0x7fdc7c19bd60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdc78011bd0 tx=0x7fdc78011cb0 comp rx=0 tx=0).stop 2024-03-16T09:06:07.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.483+0000 7fdc84231700 1 -- 172.21.15.112:0/1580031067 shutdown_connections 2024-03-16T09:06:07.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.483+0000 7fdc84231700 1 --2- 172.21.15.112:0/1580031067 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7fdc68038f70 0x7fdc6803b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:07.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.483+0000 7fdc84231700 1 --2- 172.21.15.112:0/1580031067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc7c108870 0x7fdc7c19bd60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:07.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.483+0000 7fdc84231700 1 -- 172.21.15.112:0/1580031067 >> 172.21.15.112:0/1580031067 conn(0x7fdc7c102260 msgr2=0x7fdc7c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:07.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.483+0000 7fdc84231700 1 -- 172.21.15.112:0/1580031067 shutdown_connections 2024-03-16T09:06:07.060 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:06.483+0000 7fdc84231700 1 -- 172.21.15.112:0/1580031067 wait complete. 2024-03-16T09:06:07.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:07 smithi112 ceph-mon[32307]: [16/Mar/2024:09:06:05] ENGINE Bus STARTING 2024-03-16T09:06:07.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:07 smithi112 ceph-mon[32307]: [16/Mar/2024:09:06:05] ENGINE Serving on https://172.21.15.112:7150 2024-03-16T09:06:07.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:07 smithi112 ceph-mon[32307]: [16/Mar/2024:09:06:06] ENGINE Serving on http://172.21.15.112:8765 2024-03-16T09:06:07.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:07 smithi112 ceph-mon[32307]: [16/Mar/2024:09:06:06] ENGINE Bus STARTED 2024-03-16T09:06:07.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:07 smithi112 ceph-mon[32307]: mgrmap e8: x(active, since 2s) 2024-03-16T09:06:07.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:07 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' 2024-03-16T09:06:07.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:07 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:08.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-16T09:06:08.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.475+0000 7f9b60f4c700 1 Processor -- start 2024-03-16T09:06:08.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.475+0000 7f9b60f4c700 1 -- start start 2024-03-16T09:06:08.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.476+0000 7f9b60f4c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b5c106c70 0x7f9b5c107090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:08.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.476+0000 7f9b60f4c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c107660 con 0x7f9b5c106c70 2024-03-16T09:06:08.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.476+0000 7f9b5a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b5c106c70 0x7f9b5c107090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:08.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.476+0000 7f9b5a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b5c106c70 0x7f9b5c107090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38676/0 (socket says 172.21.15.112:38676) 2024-03-16T09:06:08.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.476+0000 7f9b5a59c700 1 -- 172.21.15.112:0/569782005 learned_addr learned my addr 172.21.15.112:0/569782005 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:08.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.476+0000 7f9b5a59c700 1 -- 172.21.15.112:0/569782005 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5c107e70 con 0x7f9b5c106c70 2024-03-16T09:06:08.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.476+0000 7f9b5a59c700 1 --2- 172.21.15.112:0/569782005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b5c106c70 0x7f9b5c107090 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9b500098a0 tx=0x7f9b50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9daad6e67f5c7f66 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:08.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.476+0000 7f9b5959a700 1 -- 172.21.15.112:0/569782005 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b50004160 con 0x7f9b5c106c70 2024-03-16T09:06:08.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.476+0000 7f9b5959a700 1 -- 172.21.15.112:0/569782005 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f9b50004300 con 0x7f9b5c106c70 2024-03-16T09:06:08.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.476+0000 7f9b5959a700 1 -- 172.21.15.112:0/569782005 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b50011c00 con 0x7f9b5c106c70 2024-03-16T09:06:08.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.477+0000 7f9b60f4c700 1 -- 172.21.15.112:0/569782005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b5c106c70 msgr2=0x7f9b5c107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:08.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.477+0000 7f9b60f4c700 1 --2- 172.21.15.112:0/569782005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b5c106c70 0x7f9b5c107090 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9b500098a0 tx=0x7f9b50009bb0 comp rx=0 tx=0).stop 2024-03-16T09:06:08.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.477+0000 7f9b60f4c700 1 -- 172.21.15.112:0/569782005 shutdown_connections 2024-03-16T09:06:08.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.477+0000 7f9b60f4c700 1 --2- 172.21.15.112:0/569782005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b5c106c70 0x7f9b5c107090 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:08.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.477+0000 7f9b60f4c700 1 -- 172.21.15.112:0/569782005 >> 172.21.15.112:0/569782005 conn(0x7f9b5c102230 msgr2=0x7f9b5c104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:08.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.477+0000 7f9b60f4c700 1 -- 172.21.15.112:0/569782005 shutdown_connections 2024-03-16T09:06:08.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.477+0000 7f9b60f4c700 1 -- 172.21.15.112:0/569782005 wait complete. 2024-03-16T09:06:08.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.478+0000 7f9b60f4c700 1 Processor -- start 2024-03-16T09:06:08.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.478+0000 7f9b60f4c700 1 -- start start 2024-03-16T09:06:08.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.478+0000 7f9b60f4c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b5c106c70 0x7f9b5c19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:08.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.478+0000 7f9b60f4c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c10b220 con 0x7f9b5c106c70 2024-03-16T09:06:08.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.478+0000 7f9b5a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b5c106c70 0x7f9b5c19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:08.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.478+0000 7f9b5a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b5c106c70 0x7f9b5c19ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38690/0 (socket says 172.21.15.112:38690) 2024-03-16T09:06:08.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.478+0000 7f9b5a59c700 1 -- 172.21.15.112:0/2056744715 learned_addr learned my addr 172.21.15.112:0/2056744715 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:08.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.478+0000 7f9b5a59c700 1 -- 172.21.15.112:0/2056744715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5c19b1a0 con 0x7f9b5c106c70 2024-03-16T09:06:08.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.478+0000 7f9b5a59c700 1 --2- 172.21.15.112:0/2056744715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b5c106c70 0x7f9b5c19ac60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9b50000c00 tx=0x7f9b50011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:08.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.479+0000 7f9b4b7fe700 1 -- 172.21.15.112:0/2056744715 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b50003b20 con 0x7f9b5c106c70 2024-03-16T09:06:08.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.479+0000 7f9b4b7fe700 1 -- 172.21.15.112:0/2056744715 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f9b50003cc0 con 0x7f9b5c106c70 2024-03-16T09:06:08.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.479+0000 7f9b4b7fe700 1 -- 172.21.15.112:0/2056744715 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b500249a0 con 0x7f9b5c106c70 2024-03-16T09:06:08.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.479+0000 7f9b60f4c700 1 -- 172.21.15.112:0/2056744715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5c19b430 con 0x7f9b5c106c70 2024-03-16T09:06:08.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.479+0000 7f9b60f4c700 1 -- 172.21.15.112:0/2056744715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b5c19e120 con 0x7f9b5c106c70 2024-03-16T09:06:08.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.479+0000 7f9b4b7fe700 1 -- 172.21.15.112:0/2056744715 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f9b50022020 con 0x7f9b5c106c70 2024-03-16T09:06:08.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.480+0000 7f9b60f4c700 1 -- 172.21.15.112:0/2056744715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b5c063900 con 0x7f9b5c106c70 2024-03-16T09:06:08.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.480+0000 7f9b4b7fe700 1 --2- 172.21.15.112:0/2056744715 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f9b44038f70 0x7f9b4403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:08.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.480+0000 7f9b4b7fe700 1 -- 172.21.15.112:0/2056744715 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9b5004d080 con 0x7f9b5c106c70 2024-03-16T09:06:08.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.480+0000 7f9b59d9b700 1 --2- 172.21.15.112:0/2056744715 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f9b44038f70 0x7f9b4403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:08.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.480+0000 7f9b59d9b700 1 --2- 172.21.15.112:0/2056744715 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f9b44038f70 0x7f9b4403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c006fd0 tx=0x7f9b4c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:08.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.485+0000 7f9b4b7fe700 1 -- 172.21.15.112:0/2056744715 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9b5000b3e0 con 0x7f9b5c106c70 2024-03-16T09:06:08.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.681+0000 7f9b60f4c700 1 -- 172.21.15.112:0/2056744715 --> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f9b5c103480 con 0x7f9b44038f70 2024-03-16T09:06:08.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.682+0000 7f9b4b7fe700 1 -- 172.21.15.112:0/2056744715 <== mgr.14120 v2:172.21.15.112:6800/2629104617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f9b5c103480 con 0x7f9b44038f70 2024-03-16T09:06:08.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.684+0000 7f9b60f4c700 1 -- 172.21.15.112:0/2056744715 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f9b44038f70 msgr2=0x7f9b4403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:08.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.684+0000 7f9b60f4c700 1 --2- 172.21.15.112:0/2056744715 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f9b44038f70 0x7f9b4403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c006fd0 tx=0x7f9b4c006e40 comp rx=0 tx=0).stop 2024-03-16T09:06:08.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.684+0000 7f9b60f4c700 1 -- 172.21.15.112:0/2056744715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b5c106c70 msgr2=0x7f9b5c19ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:08.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.684+0000 7f9b60f4c700 1 --2- 172.21.15.112:0/2056744715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b5c106c70 0x7f9b5c19ac60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9b50000c00 tx=0x7f9b50011c80 comp rx=0 tx=0).stop 2024-03-16T09:06:08.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.684+0000 7f9b60f4c700 1 -- 172.21.15.112:0/2056744715 shutdown_connections 2024-03-16T09:06:08.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.684+0000 7f9b60f4c700 1 --2- 172.21.15.112:0/2056744715 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f9b44038f70 0x7f9b4403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:08.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.684+0000 7f9b60f4c700 1 --2- 172.21.15.112:0/2056744715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b5c106c70 0x7f9b5c19ac60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:08.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.684+0000 7f9b60f4c700 1 -- 172.21.15.112:0/2056744715 >> 172.21.15.112:0/2056744715 conn(0x7f9b5c102230 msgr2=0x7f9b5c102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:08.223 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.684+0000 7f9b60f4c700 1 -- 172.21.15.112:0/2056744715 shutdown_connections 2024-03-16T09:06:08.224 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:07.684+0000 7f9b60f4c700 1 -- 172.21.15.112:0/2056744715 wait complete. 2024-03-16T09:06:08.224 INFO:teuthology.orchestra.run.smithi112.stdout:Generating ssh key... 2024-03-16T09:06:08.548 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:08 smithi112 ceph-mon[32307]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:09.251 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: Generating public/private rsa key pair. 2024-03-16T09:06:09.251 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: Your identification has been saved in /tmp/tmpdvdj9d4a/key. 2024-03-16T09:06:09.251 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: Your public key has been saved in /tmp/tmpdvdj9d4a/key.pub. 2024-03-16T09:06:09.251 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: The key fingerprint is: 2024-03-16T09:06:09.251 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: SHA256:RdNBVttscZlU1m1bUlYV81/ACk53UDma3QSVogtNd58 ceph-273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:06:09.251 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: The key's randomart image is: 2024-03-16T09:06:09.251 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: +---[RSA 3072]----+ 2024-03-16T09:06:09.252 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: | oo=**X^| 2024-03-16T09:06:09.252 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: | .o+oo*%@| 2024-03-16T09:06:09.252 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: | o+oo*+*@| 2024-03-16T09:06:09.252 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: | o.o+ .E=| 2024-03-16T09:06:09.252 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: | S . . .| 2024-03-16T09:06:09.252 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: | . | 2024-03-16T09:06:09.252 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: | | 2024-03-16T09:06:09.252 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: | | 2024-03-16T09:06:09.252 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: | | 2024-03-16T09:06:09.252 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: +----[SHA256]-----+ 2024-03-16T09:06:09.544 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:09 smithi112 ceph-mon[32307]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:09.544 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:09 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' 2024-03-16T09:06:09.544 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:09 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' 2024-03-16T09:06:09.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.643+0000 7f299a17d700 1 Processor -- start 2024-03-16T09:06:09.551 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.644+0000 7f299a17d700 1 -- start start 2024-03-16T09:06:09.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.644+0000 7f299a17d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2994107dc0 0x7f29941081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:09.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.644+0000 7f299a17d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29941087b0 con 0x7f2994107dc0 2024-03-16T09:06:09.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.644+0000 7f29937fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2994107dc0 0x7f29941081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:09.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.644+0000 7f29937fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2994107dc0 0x7f29941081e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38696/0 (socket says 172.21.15.112:38696) 2024-03-16T09:06:09.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.644+0000 7f29937fe700 1 -- 172.21.15.112:0/2333886820 learned_addr learned my addr 172.21.15.112:0/2333886820 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:09.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.644+0000 7f29937fe700 1 -- 172.21.15.112:0/2333886820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2994108f70 con 0x7f2994107dc0 2024-03-16T09:06:09.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.644+0000 7f29937fe700 1 --2- 172.21.15.112:0/2333886820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2994107dc0 0x7f29941081e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f29840098a0 tx=0x7f2984009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43464b086ab813ea server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:09.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.644+0000 7f2992ffd700 1 -- 172.21.15.112:0/2333886820 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2984004160 con 0x7f2994107dc0 2024-03-16T09:06:09.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.644+0000 7f2992ffd700 1 -- 172.21.15.112:0/2333886820 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f2984004300 con 0x7f2994107dc0 2024-03-16T09:06:09.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.644+0000 7f2992ffd700 1 -- 172.21.15.112:0/2333886820 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2984011c00 con 0x7f2994107dc0 2024-03-16T09:06:09.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.645+0000 7f299a17d700 1 -- 172.21.15.112:0/2333886820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2994107dc0 msgr2=0x7f29941081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:09.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.645+0000 7f299a17d700 1 --2- 172.21.15.112:0/2333886820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2994107dc0 0x7f29941081e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f29840098a0 tx=0x7f2984009bb0 comp rx=0 tx=0).stop 2024-03-16T09:06:09.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.645+0000 7f299a17d700 1 -- 172.21.15.112:0/2333886820 shutdown_connections 2024-03-16T09:06:09.552 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.645+0000 7f299a17d700 1 --2- 172.21.15.112:0/2333886820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2994107dc0 0x7f29941081e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:09.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.645+0000 7f299a17d700 1 -- 172.21.15.112:0/2333886820 >> 172.21.15.112:0/2333886820 conn(0x7f2994103360 msgr2=0x7f29941057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:09.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.645+0000 7f299a17d700 1 -- 172.21.15.112:0/2333886820 shutdown_connections 2024-03-16T09:06:09.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.645+0000 7f299a17d700 1 -- 172.21.15.112:0/2333886820 wait complete. 2024-03-16T09:06:09.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.646+0000 7f299a17d700 1 Processor -- start 2024-03-16T09:06:09.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.646+0000 7f299a17d700 1 -- start start 2024-03-16T09:06:09.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.646+0000 7f299a17d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2994107dc0 0x7f299419bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:09.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.646+0000 7f299a17d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f299410c320 con 0x7f2994107dc0 2024-03-16T09:06:09.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.646+0000 7f29937fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2994107dc0 0x7f299419bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:09.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.646+0000 7f29937fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2994107dc0 0x7f299419bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38710/0 (socket says 172.21.15.112:38710) 2024-03-16T09:06:09.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.646+0000 7f29937fe700 1 -- 172.21.15.112:0/2847618199 learned_addr learned my addr 172.21.15.112:0/2847618199 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:09.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.646+0000 7f29937fe700 1 -- 172.21.15.112:0/2847618199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299419c240 con 0x7f2994107dc0 2024-03-16T09:06:09.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.646+0000 7f29937fe700 1 --2- 172.21.15.112:0/2847618199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2994107dc0 0x7f299419bd00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2984000c00 tx=0x7f2984011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:09.553 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.646+0000 7f29917fa700 1 -- 172.21.15.112:0/2847618199 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2984003b20 con 0x7f2994107dc0 2024-03-16T09:06:09.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.646+0000 7f29917fa700 1 -- 172.21.15.112:0/2847618199 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f2984003cc0 con 0x7f2994107dc0 2024-03-16T09:06:09.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.646+0000 7f299a17d700 1 -- 172.21.15.112:0/2847618199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299419c4d0 con 0x7f2994107dc0 2024-03-16T09:06:09.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.647+0000 7f29917fa700 1 -- 172.21.15.112:0/2847618199 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29840249a0 con 0x7f2994107dc0 2024-03-16T09:06:09.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.647+0000 7f299a17d700 1 -- 172.21.15.112:0/2847618199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299419f1b0 con 0x7f2994107dc0 2024-03-16T09:06:09.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.647+0000 7f299a17d700 1 -- 172.21.15.112:0/2847618199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2994063900 con 0x7f2994107dc0 2024-03-16T09:06:09.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.647+0000 7f29917fa700 1 -- 172.21.15.112:0/2847618199 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f2984022020 con 0x7f2994107dc0 2024-03-16T09:06:09.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.648+0000 7f29917fa700 1 --2- 172.21.15.112:0/2847618199 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f2980038f70 0x7f298003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:09.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.648+0000 7f29917fa700 1 -- 172.21.15.112:0/2847618199 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f298404d080 con 0x7f2994107dc0 2024-03-16T09:06:09.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.648+0000 7f298affd700 1 --2- 172.21.15.112:0/2847618199 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f2980038f70 0x7f298003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:09.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.648+0000 7f298affd700 1 --2- 172.21.15.112:0/2847618199 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f2980038f70 0x7f298003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f297c006fd0 tx=0x7f297c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:09.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.652+0000 7f29917fa700 1 -- 172.21.15.112:0/2847618199 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f298400b3e0 con 0x7f2994107dc0 2024-03-16T09:06:09.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.847+0000 7f299a17d700 1 -- 172.21.15.112:0/2847618199 --> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f29941045b0 con 0x7f2980038f70 2024-03-16T09:06:09.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.948+0000 7f29917fa700 1 -- 172.21.15.112:0/2847618199 <== mgr.14120 v2:172.21.15.112:6800/2629104617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f29941045b0 con 0x7f2980038f70 2024-03-16T09:06:09.554 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.950+0000 7f299a17d700 1 -- 172.21.15.112:0/2847618199 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f2980038f70 msgr2=0x7f298003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:09.555 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.950+0000 7f299a17d700 1 --2- 172.21.15.112:0/2847618199 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f2980038f70 0x7f298003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f297c006fd0 tx=0x7f297c006e40 comp rx=0 tx=0).stop 2024-03-16T09:06:09.555 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.950+0000 7f299a17d700 1 -- 172.21.15.112:0/2847618199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2994107dc0 msgr2=0x7f299419bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:09.555 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.950+0000 7f299a17d700 1 --2- 172.21.15.112:0/2847618199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2994107dc0 0x7f299419bd00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2984000c00 tx=0x7f2984011c80 comp rx=0 tx=0).stop 2024-03-16T09:06:09.555 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.951+0000 7f299a17d700 1 -- 172.21.15.112:0/2847618199 shutdown_connections 2024-03-16T09:06:09.555 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.951+0000 7f299a17d700 1 --2- 172.21.15.112:0/2847618199 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f2980038f70 0x7f298003b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:09.555 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.951+0000 7f299a17d700 1 --2- 172.21.15.112:0/2847618199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2994107dc0 0x7f299419bd00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:09.555 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.951+0000 7f299a17d700 1 -- 172.21.15.112:0/2847618199 >> 172.21.15.112:0/2847618199 conn(0x7f2994103360 msgr2=0x7f2994103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:09.555 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.951+0000 7f299a17d700 1 -- 172.21.15.112:0/2847618199 shutdown_connections 2024-03-16T09:06:09.555 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:08.951+0000 7f299a17d700 1 -- 172.21.15.112:0/2847618199 wait complete. 2024-03-16T09:06:10.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:10 smithi112 ceph-mon[32307]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:10.492 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:10 smithi112 ceph-mon[32307]: Generating ssh key... 2024-03-16T09:06:10.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDFcOmJO9hp3V8Ms/aru4Pan1LEQ5rNjV78XjDDJiH1DoIHboQXmLXbZ5TRFi1cq2QREwty1rPpPRCFnxUXhgU0F1jKlxOQMNgmk7oL1YQ2We/B/ErzxwaMJ7/3joa8Ovw6K/Xps/2sMXuUk+uaxPl+ucZ8Z5P6dHwAkflC2+vkwcg71nYS2eoRqrpZv1SZRqlacY8ky6FVI9owXeE8AiEM9x3S1gEqCGRDHL/EBHXGx67Zs2Aq5cPnoVtla8Njw5VDVbqUgVS+Z8Nis0qBokVC8tAx70EPjCAnWsfh5cm1u7ug9g7E48oQpFnEaSaed6atuF+QZq11pYboVhGt7vZLILkRleRvC9k9bIOmh/rCF49csDhvsJ5jgwuxWfPc2MvAAXhbRs2V+D2xRr0ugfmbIlmWqzQ0yqhDbrn+szcrAn6e13030Oev95yz8aq6hQZlZW8Cgn6nsn+HVxOOvtgoNPdu/iIPtf17h4qPpnh/ZJsD1np8ckzDwOpKMClBEM= ceph-273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:06:10.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.925+0000 7f1baeb0a700 1 Processor -- start 2024-03-16T09:06:10.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.925+0000 7f1baeb0a700 1 -- start start 2024-03-16T09:06:10.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.925+0000 7f1baeb0a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba8107da0 0x7f1ba81081c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:10.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.925+0000 7f1baeb0a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba8108790 con 0x7f1ba8107da0 2024-03-16T09:06:10.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.926+0000 7f1bac8a6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba8107da0 0x7f1ba81081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:10.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.926+0000 7f1bac8a6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba8107da0 0x7f1ba81081c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38722/0 (socket says 172.21.15.112:38722) 2024-03-16T09:06:10.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.926+0000 7f1bac8a6700 1 -- 172.21.15.112:0/1513290203 learned_addr learned my addr 172.21.15.112:0/1513290203 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:10.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.926+0000 7f1bac8a6700 1 -- 172.21.15.112:0/1513290203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba8108f50 con 0x7f1ba8107da0 2024-03-16T09:06:10.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.926+0000 7f1bac8a6700 1 --2- 172.21.15.112:0/1513290203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba8107da0 0x7f1ba81081c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009e80 tx=0x7f1ba000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40516ed1c34085a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:10.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.926+0000 7f1ba77fe700 1 -- 172.21.15.112:0/1513290203 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ba0017070 con 0x7f1ba8107da0 2024-03-16T09:06:10.792 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.926+0000 7f1ba77fe700 1 -- 172.21.15.112:0/1513290203 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f1ba000b8c0 con 0x7f1ba8107da0 2024-03-16T09:06:10.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.926+0000 7f1ba77fe700 1 -- 172.21.15.112:0/1513290203 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ba0004620 con 0x7f1ba8107da0 2024-03-16T09:06:10.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.927+0000 7f1baeb0a700 1 -- 172.21.15.112:0/1513290203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba8107da0 msgr2=0x7f1ba81081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:10.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.927+0000 7f1baeb0a700 1 --2- 172.21.15.112:0/1513290203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba8107da0 0x7f1ba81081c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009e80 tx=0x7f1ba000b330 comp rx=0 tx=0).stop 2024-03-16T09:06:10.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.927+0000 7f1baeb0a700 1 -- 172.21.15.112:0/1513290203 shutdown_connections 2024-03-16T09:06:10.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.927+0000 7f1baeb0a700 1 --2- 172.21.15.112:0/1513290203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba8107da0 0x7f1ba81081c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:10.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.927+0000 7f1baeb0a700 1 -- 172.21.15.112:0/1513290203 >> 172.21.15.112:0/1513290203 conn(0x7f1ba8103360 msgr2=0x7f1ba8105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:10.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.927+0000 7f1baeb0a700 1 -- 172.21.15.112:0/1513290203 shutdown_connections 2024-03-16T09:06:10.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.927+0000 7f1baeb0a700 1 -- 172.21.15.112:0/1513290203 wait complete. 2024-03-16T09:06:10.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.927+0000 7f1baeb0a700 1 Processor -- start 2024-03-16T09:06:10.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.927+0000 7f1baeb0a700 1 -- start start 2024-03-16T09:06:10.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.928+0000 7f1baeb0a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba8107da0 0x7f1ba807db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:10.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.928+0000 7f1baeb0a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba810c300 con 0x7f1ba8107da0 2024-03-16T09:06:10.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.928+0000 7f1bac8a6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba8107da0 0x7f1ba807db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:10.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.928+0000 7f1bac8a6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba8107da0 0x7f1ba807db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38732/0 (socket says 172.21.15.112:38732) 2024-03-16T09:06:10.793 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.928+0000 7f1bac8a6700 1 -- 172.21.15.112:0/4082777861 learned_addr learned my addr 172.21.15.112:0/4082777861 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:10.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.928+0000 7f1bac8a6700 1 -- 172.21.15.112:0/4082777861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba807e090 con 0x7f1ba8107da0 2024-03-16T09:06:10.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.928+0000 7f1bac8a6700 1 --2- 172.21.15.112:0/4082777861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba8107da0 0x7f1ba807db50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000d010 tx=0x7f1ba0003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:10.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.928+0000 7f1ba5ffb700 1 -- 172.21.15.112:0/4082777861 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ba0021020 con 0x7f1ba8107da0 2024-03-16T09:06:10.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.928+0000 7f1ba5ffb700 1 -- 172.21.15.112:0/4082777861 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f1ba00094d0 con 0x7f1ba8107da0 2024-03-16T09:06:10.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.928+0000 7f1baeb0a700 1 -- 172.21.15.112:0/4082777861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba807c280 con 0x7f1ba8107da0 2024-03-16T09:06:10.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.928+0000 7f1ba5ffb700 1 -- 172.21.15.112:0/4082777861 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ba0017040 con 0x7f1ba8107da0 2024-03-16T09:06:10.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.928+0000 7f1baeb0a700 1 -- 172.21.15.112:0/4082777861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba807c7c0 con 0x7f1ba8107da0 2024-03-16T09:06:10.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.929+0000 7f1ba5ffb700 1 -- 172.21.15.112:0/4082777861 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f1ba0024020 con 0x7f1ba8107da0 2024-03-16T09:06:10.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.929+0000 7f1ba5ffb700 1 --2- 172.21.15.112:0/4082777861 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f1b90038ed0 0x7f1b9003b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:10.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.929+0000 7f1baeb0a700 1 -- 172.21.15.112:0/4082777861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba8063900 con 0x7f1ba8107da0 2024-03-16T09:06:10.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.929+0000 7f1ba5ffb700 1 -- 172.21.15.112:0/4082777861 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1ba000f050 con 0x7f1ba8107da0 2024-03-16T09:06:10.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.930+0000 7f1ba7fff700 1 --2- 172.21.15.112:0/4082777861 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f1b90038ed0 0x7f1b9003b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:10.794 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.930+0000 7f1ba7fff700 1 --2- 172.21.15.112:0/4082777861 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f1b90038ed0 0x7f1b9003b390 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c006fd0 tx=0x7f1b9c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:10.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:09.934+0000 7f1ba5ffb700 1 -- 172.21.15.112:0/4082777861 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1ba002b020 con 0x7f1ba8107da0 2024-03-16T09:06:10.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:10.130+0000 7f1baeb0a700 1 -- 172.21.15.112:0/4082777861 --> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f1ba8104530 con 0x7f1b90038ed0 2024-03-16T09:06:10.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:10.130+0000 7f1ba5ffb700 1 -- 172.21.15.112:0/4082777861 <== mgr.14120 v2:172.21.15.112:6800/2629104617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f1ba8104530 con 0x7f1b90038ed0 2024-03-16T09:06:10.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:10.133+0000 7f1baeb0a700 1 -- 172.21.15.112:0/4082777861 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f1b90038ed0 msgr2=0x7f1b9003b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:10.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:10.133+0000 7f1baeb0a700 1 --2- 172.21.15.112:0/4082777861 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f1b90038ed0 0x7f1b9003b390 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c006fd0 tx=0x7f1b9c006e40 comp rx=0 tx=0).stop 2024-03-16T09:06:10.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:10.133+0000 7f1baeb0a700 1 -- 172.21.15.112:0/4082777861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba8107da0 msgr2=0x7f1ba807db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:10.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:10.133+0000 7f1baeb0a700 1 --2- 172.21.15.112:0/4082777861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba8107da0 0x7f1ba807db50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000d010 tx=0x7f1ba0003d00 comp rx=0 tx=0).stop 2024-03-16T09:06:10.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:10.133+0000 7f1baeb0a700 1 -- 172.21.15.112:0/4082777861 shutdown_connections 2024-03-16T09:06:10.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:10.133+0000 7f1baeb0a700 1 --2- 172.21.15.112:0/4082777861 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f1b90038ed0 0x7f1b9003b390 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:10.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:10.133+0000 7f1baeb0a700 1 --2- 172.21.15.112:0/4082777861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ba8107da0 0x7f1ba807db50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:10.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:10.133+0000 7f1baeb0a700 1 -- 172.21.15.112:0/4082777861 >> 172.21.15.112:0/4082777861 conn(0x7f1ba8103360 msgr2=0x7f1ba8103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:10.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:10.133+0000 7f1baeb0a700 1 -- 172.21.15.112:0/4082777861 shutdown_connections 2024-03-16T09:06:10.795 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:10.133+0000 7f1baeb0a700 1 -- 172.21.15.112:0/4082777861 wait complete. 2024-03-16T09:06:10.795 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-16T09:06:10.795 INFO:teuthology.orchestra.run.smithi112.stdout:Adding key to root@localhost authorized_keys... 2024-03-16T09:06:10.795 INFO:teuthology.orchestra.run.smithi112.stdout:Adding host smithi112... 2024-03-16T09:06:11.341 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:11 smithi112 ceph-mon[32307]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:12.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:12 smithi112 ceph-mon[32307]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi112", "addr": "172.21.15.112", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:13.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:13 smithi112 ceph-mon[32307]: Deploying cephadm binary to smithi112 2024-03-16T09:06:16.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Added host 'smithi112' with addr '172.21.15.112' 2024-03-16T09:06:16.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.185+0000 7fdbb3a3a700 1 Processor -- start 2024-03-16T09:06:16.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.185+0000 7fdbb3a3a700 1 -- start start 2024-03-16T09:06:16.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.185+0000 7fdbb3a3a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbac106640 0x7fdbac106a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:16.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.185+0000 7fdbb3a3a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac07be50 con 0x7fdbac106640 2024-03-16T09:06:16.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.186+0000 7fdbb17d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbac106640 0x7fdbac106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:16.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.186+0000 7fdbb17d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbac106640 0x7fdbac106a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38742/0 (socket says 172.21.15.112:38742) 2024-03-16T09:06:16.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.186+0000 7fdbb17d6700 1 -- 172.21.15.112:0/1430419941 learned_addr learned my addr 172.21.15.112:0/1430419941 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:16.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.186+0000 7fdbb17d6700 1 -- 172.21.15.112:0/1430419941 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbac106fa0 con 0x7fdbac106640 2024-03-16T09:06:16.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.186+0000 7fdbb17d6700 1 --2- 172.21.15.112:0/1430419941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbac106640 0x7fdbac106a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdba80098a0 tx=0x7fdba8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43aa057d0ff5d923 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:16.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.186+0000 7fdba3fff700 1 -- 172.21.15.112:0/1430419941 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba8004160 con 0x7fdbac106640 2024-03-16T09:06:16.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.186+0000 7fdba3fff700 1 -- 172.21.15.112:0/1430419941 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fdba8004300 con 0x7fdbac106640 2024-03-16T09:06:16.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.186+0000 7fdba3fff700 1 -- 172.21.15.112:0/1430419941 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba8011c00 con 0x7fdbac106640 2024-03-16T09:06:16.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.187+0000 7fdbb3a3a700 1 -- 172.21.15.112:0/1430419941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbac106640 msgr2=0x7fdbac106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:16.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.187+0000 7fdbb3a3a700 1 --2- 172.21.15.112:0/1430419941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbac106640 0x7fdbac106a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdba80098a0 tx=0x7fdba8009bb0 comp rx=0 tx=0).stop 2024-03-16T09:06:16.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.187+0000 7fdbb3a3a700 1 -- 172.21.15.112:0/1430419941 shutdown_connections 2024-03-16T09:06:16.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.187+0000 7fdbb3a3a700 1 --2- 172.21.15.112:0/1430419941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbac106640 0x7fdbac106a60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:16.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.187+0000 7fdbb3a3a700 1 -- 172.21.15.112:0/1430419941 >> 172.21.15.112:0/1430419941 conn(0x7fdbac102220 msgr2=0x7fdbac104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:16.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.187+0000 7fdbb3a3a700 1 -- 172.21.15.112:0/1430419941 shutdown_connections 2024-03-16T09:06:16.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.187+0000 7fdbb3a3a700 1 -- 172.21.15.112:0/1430419941 wait complete. 2024-03-16T09:06:16.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.187+0000 7fdbb3a3a700 1 Processor -- start 2024-03-16T09:06:16.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.187+0000 7fdbb3a3a700 1 -- start start 2024-03-16T09:06:16.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.188+0000 7fdbb3a3a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbac106640 0x7fdbac19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:16.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.188+0000 7fdbb3a3a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac108020 con 0x7fdbac106640 2024-03-16T09:06:16.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.188+0000 7fdbb17d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbac106640 0x7fdbac19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:16.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.188+0000 7fdbb17d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbac106640 0x7fdbac19df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38748/0 (socket says 172.21.15.112:38748) 2024-03-16T09:06:16.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.188+0000 7fdbb17d6700 1 -- 172.21.15.112:0/3897749203 learned_addr learned my addr 172.21.15.112:0/3897749203 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:16.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.188+0000 7fdbb17d6700 1 -- 172.21.15.112:0/3897749203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbac19e450 con 0x7fdbac106640 2024-03-16T09:06:16.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.188+0000 7fdbb17d6700 1 --2- 172.21.15.112:0/3897749203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbac106640 0x7fdbac19df10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdba8011bd0 tx=0x7fdba8011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:16.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.188+0000 7fdba27fc700 1 -- 172.21.15.112:0/3897749203 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba8003b20 con 0x7fdbac106640 2024-03-16T09:06:16.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.188+0000 7fdba27fc700 1 -- 172.21.15.112:0/3897749203 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fdba8003cc0 con 0x7fdbac106640 2024-03-16T09:06:16.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.188+0000 7fdbb3a3a700 1 -- 172.21.15.112:0/3897749203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbac19e6e0 con 0x7fdbac106640 2024-03-16T09:06:16.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.188+0000 7fdba27fc700 1 -- 172.21.15.112:0/3897749203 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdba8024930 con 0x7fdbac106640 2024-03-16T09:06:16.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.188+0000 7fdbb3a3a700 1 -- 172.21.15.112:0/3897749203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbac1a13d0 con 0x7fdbac106640 2024-03-16T09:06:16.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.189+0000 7fdba27fc700 1 -- 172.21.15.112:0/3897749203 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fdba8022020 con 0x7fdbac106640 2024-03-16T09:06:16.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.189+0000 7fdbb3a3a700 1 -- 172.21.15.112:0/3897749203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbac063900 con 0x7fdbac106640 2024-03-16T09:06:16.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.189+0000 7fdba27fc700 1 --2- 172.21.15.112:0/3897749203 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7fdb98038f20 0x7fdb9803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:16.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.189+0000 7fdba27fc700 1 -- 172.21.15.112:0/3897749203 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdba804cf30 con 0x7fdbac106640 2024-03-16T09:06:16.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.189+0000 7fdbb0fd5700 1 --2- 172.21.15.112:0/3897749203 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7fdb98038f20 0x7fdb9803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:16.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.190+0000 7fdbb0fd5700 1 --2- 172.21.15.112:0/3897749203 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7fdb98038f20 0x7fdb9803b3e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c006fd0 tx=0x7fdb9c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:16.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.194+0000 7fdba27fc700 1 -- 172.21.15.112:0/3897749203 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdba802d450 con 0x7fdbac106640 2024-03-16T09:06:16.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:11.396+0000 7fdbb3a3a700 1 -- 172.21.15.112:0/3897749203 --> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi112", "addr": "172.21.15.112", "target": ["mon-mgr", ""]}) -- 0x7fdbac0008d0 con 0x7fdb98038f20 2024-03-16T09:06:16.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:15.656+0000 7fdba27fc700 1 -- 172.21.15.112:0/3897749203 <== mgr.14120 v2:172.21.15.112:6800/2629104617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fdbac0008d0 con 0x7fdb98038f20 2024-03-16T09:06:16.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:15.659+0000 7fdbb3a3a700 1 -- 172.21.15.112:0/3897749203 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7fdb98038f20 msgr2=0x7fdb9803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:16.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:15.659+0000 7fdbb3a3a700 1 --2- 172.21.15.112:0/3897749203 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7fdb98038f20 0x7fdb9803b3e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c006fd0 tx=0x7fdb9c006e40 comp rx=0 tx=0).stop 2024-03-16T09:06:16.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:15.659+0000 7fdbb3a3a700 1 -- 172.21.15.112:0/3897749203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbac106640 msgr2=0x7fdbac19df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:16.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:15.659+0000 7fdbb3a3a700 1 --2- 172.21.15.112:0/3897749203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbac106640 0x7fdbac19df10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdba8011bd0 tx=0x7fdba8011cb0 comp rx=0 tx=0).stop 2024-03-16T09:06:16.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:15.659+0000 7fdbb3a3a700 1 -- 172.21.15.112:0/3897749203 shutdown_connections 2024-03-16T09:06:16.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:15.659+0000 7fdbb3a3a700 1 --2- 172.21.15.112:0/3897749203 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7fdb98038f20 0x7fdb9803b3e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:16.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:15.659+0000 7fdbb3a3a700 1 --2- 172.21.15.112:0/3897749203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbac106640 0x7fdbac19df10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:16.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:15.659+0000 7fdbb3a3a700 1 -- 172.21.15.112:0/3897749203 >> 172.21.15.112:0/3897749203 conn(0x7fdbac102220 msgr2=0x7fdbac194b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:16.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:15.659+0000 7fdbb3a3a700 1 -- 172.21.15.112:0/3897749203 shutdown_connections 2024-03-16T09:06:16.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:15.659+0000 7fdbb3a3a700 1 -- 172.21.15.112:0/3897749203 wait complete. 2024-03-16T09:06:16.287 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying unmanaged mon service... 2024-03-16T09:06:16.899 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:16 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' 2024-03-16T09:06:16.899 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:16 smithi112 ceph-mon[32307]: Added host smithi112 2024-03-16T09:06:16.899 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:16 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:18.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-16T09:06:18.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.739+0000 7efc86eb2700 1 Processor -- start 2024-03-16T09:06:18.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.739+0000 7efc86eb2700 1 -- start start 2024-03-16T09:06:18.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.739+0000 7efc86eb2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc8007de30 0x7efc8007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:18.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.739+0000 7efc86eb2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efc8007c8b0 con 0x7efc8007de30 2024-03-16T09:06:18.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.739+0000 7efc84c4e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc8007de30 0x7efc8007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:18.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.740+0000 7efc84c4e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc8007de30 0x7efc8007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54378/0 (socket says 172.21.15.112:54378) 2024-03-16T09:06:18.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.740+0000 7efc84c4e700 1 -- 172.21.15.112:0/1972262207 learned_addr learned my addr 172.21.15.112:0/1972262207 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:18.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.740+0000 7efc84c4e700 1 -- 172.21.15.112:0/1972262207 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8007ca30 con 0x7efc8007de30 2024-03-16T09:06:18.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.740+0000 7efc84c4e700 1 --2- 172.21.15.112:0/1972262207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc8007de30 0x7efc8007c2e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efc740098a0 tx=0x7efc74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7c5abbb26131bd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:18.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.740+0000 7efc7f7fe700 1 -- 172.21.15.112:0/1972262207 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc74004160 con 0x7efc8007de30 2024-03-16T09:06:18.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.740+0000 7efc7f7fe700 1 -- 172.21.15.112:0/1972262207 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7efc74004300 con 0x7efc8007de30 2024-03-16T09:06:18.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.740+0000 7efc7f7fe700 1 -- 172.21.15.112:0/1972262207 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc74011c00 con 0x7efc8007de30 2024-03-16T09:06:18.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.741+0000 7efc86eb2700 1 -- 172.21.15.112:0/1972262207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc8007de30 msgr2=0x7efc8007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:18.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.741+0000 7efc86eb2700 1 --2- 172.21.15.112:0/1972262207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc8007de30 0x7efc8007c2e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efc740098a0 tx=0x7efc74009bb0 comp rx=0 tx=0).stop 2024-03-16T09:06:18.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.741+0000 7efc86eb2700 1 -- 172.21.15.112:0/1972262207 shutdown_connections 2024-03-16T09:06:18.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.741+0000 7efc86eb2700 1 --2- 172.21.15.112:0/1972262207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc8007de30 0x7efc8007c2e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:18.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.741+0000 7efc86eb2700 1 -- 172.21.15.112:0/1972262207 >> 172.21.15.112:0/1972262207 conn(0x7efc80103360 msgr2=0x7efc801057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:18.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.741+0000 7efc86eb2700 1 -- 172.21.15.112:0/1972262207 shutdown_connections 2024-03-16T09:06:18.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.741+0000 7efc86eb2700 1 -- 172.21.15.112:0/1972262207 wait complete. 2024-03-16T09:06:18.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.741+0000 7efc86eb2700 1 Processor -- start 2024-03-16T09:06:18.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.741+0000 7efc86eb2700 1 -- start start 2024-03-16T09:06:18.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.741+0000 7efc86eb2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc8007de30 0x7efc8019dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:18.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.741+0000 7efc86eb2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efc8010e370 con 0x7efc8007de30 2024-03-16T09:06:18.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.742+0000 7efc84c4e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc8007de30 0x7efc8019dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:18.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.742+0000 7efc84c4e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc8007de30 0x7efc8019dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54386/0 (socket says 172.21.15.112:54386) 2024-03-16T09:06:18.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.742+0000 7efc84c4e700 1 -- 172.21.15.112:0/84272595 learned_addr learned my addr 172.21.15.112:0/84272595 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:18.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.742+0000 7efc84c4e700 1 -- 172.21.15.112:0/84272595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8019e420 con 0x7efc8007de30 2024-03-16T09:06:18.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.742+0000 7efc84c4e700 1 --2- 172.21.15.112:0/84272595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc8007de30 0x7efc8019dee0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efc74011bd0 tx=0x7efc74011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:18.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.742+0000 7efc7dffb700 1 -- 172.21.15.112:0/84272595 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc74003b20 con 0x7efc8007de30 2024-03-16T09:06:18.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.742+0000 7efc7dffb700 1 -- 172.21.15.112:0/84272595 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7efc74003cc0 con 0x7efc8007de30 2024-03-16T09:06:18.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.742+0000 7efc86eb2700 1 -- 172.21.15.112:0/84272595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc8019e6b0 con 0x7efc8007de30 2024-03-16T09:06:18.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.742+0000 7efc7dffb700 1 -- 172.21.15.112:0/84272595 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc74024930 con 0x7efc8007de30 2024-03-16T09:06:18.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.742+0000 7efc86eb2700 1 -- 172.21.15.112:0/84272595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc801a13a0 con 0x7efc8007de30 2024-03-16T09:06:18.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.743+0000 7efc7dffb700 1 -- 172.21.15.112:0/84272595 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7efc74022020 con 0x7efc8007de30 2024-03-16T09:06:18.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.743+0000 7efc7dffb700 1 --2- 172.21.15.112:0/84272595 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7efc70038f70 0x7efc7003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:18.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.743+0000 7efc86eb2700 1 -- 172.21.15.112:0/84272595 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc80063900 con 0x7efc8007de30 2024-03-16T09:06:18.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.743+0000 7efc7dffb700 1 -- 172.21.15.112:0/84272595 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efc7404d120 con 0x7efc8007de30 2024-03-16T09:06:18.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.743+0000 7efc7ffff700 1 --2- 172.21.15.112:0/84272595 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7efc70038f70 0x7efc7003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:18.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.744+0000 7efc7ffff700 1 --2- 172.21.15.112:0/84272595 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7efc70038f70 0x7efc7003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efc6c006fd0 tx=0x7efc6c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:18.134 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.748+0000 7efc7dffb700 1 -- 172.21.15.112:0/84272595 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efc7402cdf0 con 0x7efc8007de30 2024-03-16T09:06:18.134 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.964+0000 7efc86eb2700 1 -- 172.21.15.112:0/84272595 --> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7efc80104600 con 0x7efc70038f70 2024-03-16T09:06:18.134 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.971+0000 7efc7dffb700 1 -- 172.21.15.112:0/84272595 <== mgr.14120 v2:172.21.15.112:6800/2629104617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7efc80104600 con 0x7efc70038f70 2024-03-16T09:06:18.134 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.974+0000 7efc86eb2700 1 -- 172.21.15.112:0/84272595 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7efc70038f70 msgr2=0x7efc7003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:18.134 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.974+0000 7efc86eb2700 1 --2- 172.21.15.112:0/84272595 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7efc70038f70 0x7efc7003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efc6c006fd0 tx=0x7efc6c006e40 comp rx=0 tx=0).stop 2024-03-16T09:06:18.134 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.974+0000 7efc86eb2700 1 -- 172.21.15.112:0/84272595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc8007de30 msgr2=0x7efc8019dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:18.134 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.974+0000 7efc86eb2700 1 --2- 172.21.15.112:0/84272595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc8007de30 0x7efc8019dee0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efc74011bd0 tx=0x7efc74011cb0 comp rx=0 tx=0).stop 2024-03-16T09:06:18.134 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.974+0000 7efc86eb2700 1 -- 172.21.15.112:0/84272595 shutdown_connections 2024-03-16T09:06:18.134 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.974+0000 7efc86eb2700 1 --2- 172.21.15.112:0/84272595 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7efc70038f70 0x7efc7003b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:18.134 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.974+0000 7efc86eb2700 1 --2- 172.21.15.112:0/84272595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc8007de30 0x7efc8019dee0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:18.134 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.974+0000 7efc86eb2700 1 -- 172.21.15.112:0/84272595 >> 172.21.15.112:0/84272595 conn(0x7efc80103360 msgr2=0x7efc80103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:18.134 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.974+0000 7efc86eb2700 1 -- 172.21.15.112:0/84272595 shutdown_connections 2024-03-16T09:06:18.134 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:16.974+0000 7efc86eb2700 1 -- 172.21.15.112:0/84272595 wait complete. 2024-03-16T09:06:18.134 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying unmanaged mgr service... 2024-03-16T09:06:18.248 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:17 smithi112 ceph-mon[32307]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:18.249 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:17 smithi112 ceph-mon[32307]: Saving service mon spec with placement count:5 2024-03-16T09:06:18.249 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:17 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' 2024-03-16T09:06:19.757 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-16T09:06:19.757 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.915+0000 7fcd6ec2e700 1 Processor -- start 2024-03-16T09:06:19.757 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.915+0000 7fcd6ec2e700 1 -- start start 2024-03-16T09:06:19.757 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.915+0000 7fcd6ec2e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd681066a0 0x7fcd68106ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:19.757 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.916+0000 7fcd6ec2e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6807bdf0 con 0x7fcd681066a0 2024-03-16T09:06:19.757 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.916+0000 7fcd6c9ca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd681066a0 0x7fcd68106ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:19.757 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.916+0000 7fcd6c9ca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd681066a0 0x7fcd68106ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54388/0 (socket says 172.21.15.112:54388) 2024-03-16T09:06:19.758 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.916+0000 7fcd6c9ca700 1 -- 172.21.15.112:0/1356163518 learned_addr learned my addr 172.21.15.112:0/1356163518 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:19.758 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.916+0000 7fcd6c9ca700 1 -- 172.21.15.112:0/1356163518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd68107000 con 0x7fcd681066a0 2024-03-16T09:06:19.758 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.916+0000 7fcd6c9ca700 1 --2- 172.21.15.112:0/1356163518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd681066a0 0x7fcd68106ac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0098a0 tx=0x7fcd5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579a98c56ededbc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:19.758 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.916+0000 7fcd677fe700 1 -- 172.21.15.112:0/1356163518 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd5c004160 con 0x7fcd681066a0 2024-03-16T09:06:19.758 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.916+0000 7fcd677fe700 1 -- 172.21.15.112:0/1356163518 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcd5c004300 con 0x7fcd681066a0 2024-03-16T09:06:19.758 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.916+0000 7fcd677fe700 1 -- 172.21.15.112:0/1356163518 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd5c011bb0 con 0x7fcd681066a0 2024-03-16T09:06:19.758 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.917+0000 7fcd6ec2e700 1 -- 172.21.15.112:0/1356163518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd681066a0 msgr2=0x7fcd68106ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:19.758 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.917+0000 7fcd6ec2e700 1 --2- 172.21.15.112:0/1356163518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd681066a0 0x7fcd68106ac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0098a0 tx=0x7fcd5c009bb0 comp rx=0 tx=0).stop 2024-03-16T09:06:19.758 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.917+0000 7fcd6ec2e700 1 -- 172.21.15.112:0/1356163518 shutdown_connections 2024-03-16T09:06:19.758 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.917+0000 7fcd6ec2e700 1 --2- 172.21.15.112:0/1356163518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd681066a0 0x7fcd68106ac0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:19.759 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.917+0000 7fcd6ec2e700 1 -- 172.21.15.112:0/1356163518 >> 172.21.15.112:0/1356163518 conn(0x7fcd68102220 msgr2=0x7fcd68104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:19.759 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.917+0000 7fcd6ec2e700 1 -- 172.21.15.112:0/1356163518 shutdown_connections 2024-03-16T09:06:19.759 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.917+0000 7fcd6ec2e700 1 -- 172.21.15.112:0/1356163518 wait complete. 2024-03-16T09:06:19.759 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.918+0000 7fcd6ec2e700 1 Processor -- start 2024-03-16T09:06:19.759 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.918+0000 7fcd6ec2e700 1 -- start start 2024-03-16T09:06:19.759 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.918+0000 7fcd6ec2e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd681066a0 0x7fcd6819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:19.759 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.918+0000 7fcd6ec2e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcd68108130 con 0x7fcd681066a0 2024-03-16T09:06:19.759 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.918+0000 7fcd6c9ca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd681066a0 0x7fcd6819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:19.759 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.918+0000 7fcd6c9ca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd681066a0 0x7fcd6819ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54398/0 (socket says 172.21.15.112:54398) 2024-03-16T09:06:19.759 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.918+0000 7fcd6c9ca700 1 -- 172.21.15.112:0/883660578 learned_addr learned my addr 172.21.15.112:0/883660578 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:19.759 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.918+0000 7fcd6c9ca700 1 -- 172.21.15.112:0/883660578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6819e410 con 0x7fcd681066a0 2024-03-16T09:06:19.759 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.919+0000 7fcd6c9ca700 1 --2- 172.21.15.112:0/883660578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd681066a0 0x7fcd6819ded0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c011870 tx=0x7fcd5c011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:19.759 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.919+0000 7fcd65ffb700 1 -- 172.21.15.112:0/883660578 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd5c003af0 con 0x7fcd681066a0 2024-03-16T09:06:19.759 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.919+0000 7fcd65ffb700 1 -- 172.21.15.112:0/883660578 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcd5c003c90 con 0x7fcd681066a0 2024-03-16T09:06:19.759 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.919+0000 7fcd6ec2e700 1 -- 172.21.15.112:0/883660578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6819e6a0 con 0x7fcd681066a0 2024-03-16T09:06:19.760 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.919+0000 7fcd65ffb700 1 -- 172.21.15.112:0/883660578 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd5c019960 con 0x7fcd681066a0 2024-03-16T09:06:19.760 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.919+0000 7fcd6ec2e700 1 -- 172.21.15.112:0/883660578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd6819eb00 con 0x7fcd681066a0 2024-03-16T09:06:19.760 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.920+0000 7fcd65ffb700 1 -- 172.21.15.112:0/883660578 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fcd5c019c10 con 0x7fcd681066a0 2024-03-16T09:06:19.760 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.920+0000 7fcd65ffb700 1 --2- 172.21.15.112:0/883660578 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7fcd58038f20 0x7fcd5803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:19.760 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.920+0000 7fcd65ffb700 1 -- 172.21.15.112:0/883660578 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcd5c04e400 con 0x7fcd681066a0 2024-03-16T09:06:19.760 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.920+0000 7fcd67fff700 1 --2- 172.21.15.112:0/883660578 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7fcd58038f20 0x7fcd5803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:19.760 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.920+0000 7fcd6ec2e700 1 -- 172.21.15.112:0/883660578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd68063900 con 0x7fcd681066a0 2024-03-16T09:06:19.760 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.921+0000 7fcd67fff700 1 --2- 172.21.15.112:0/883660578 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7fcd58038f20 0x7fcd5803b3e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcd54006fd0 tx=0x7fcd54006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:19.760 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:18.925+0000 7fcd65ffb700 1 -- 172.21.15.112:0/883660578 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcd5c01ad10 con 0x7fcd681066a0 2024-03-16T09:06:19.760 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:19.141+0000 7fcd6ec2e700 1 -- 172.21.15.112:0/883660578 --> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fcd68062710 con 0x7fcd58038f20 2024-03-16T09:06:19.760 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:19.148+0000 7fcd65ffb700 1 -- 172.21.15.112:0/883660578 <== mgr.14120 v2:172.21.15.112:6800/2629104617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcd68062710 con 0x7fcd58038f20 2024-03-16T09:06:19.760 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:19.151+0000 7fcd6ec2e700 1 -- 172.21.15.112:0/883660578 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7fcd58038f20 msgr2=0x7fcd5803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:19.760 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:19.151+0000 7fcd6ec2e700 1 --2- 172.21.15.112:0/883660578 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7fcd58038f20 0x7fcd5803b3e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcd54006fd0 tx=0x7fcd54006e40 comp rx=0 tx=0).stop 2024-03-16T09:06:19.761 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:19.151+0000 7fcd6ec2e700 1 -- 172.21.15.112:0/883660578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd681066a0 msgr2=0x7fcd6819ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:19.761 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:19.151+0000 7fcd6ec2e700 1 --2- 172.21.15.112:0/883660578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd681066a0 0x7fcd6819ded0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c011870 tx=0x7fcd5c011d70 comp rx=0 tx=0).stop 2024-03-16T09:06:19.761 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:19.151+0000 7fcd6ec2e700 1 -- 172.21.15.112:0/883660578 shutdown_connections 2024-03-16T09:06:19.761 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:19.151+0000 7fcd6ec2e700 1 --2- 172.21.15.112:0/883660578 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7fcd58038f20 0x7fcd5803b3e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:19.761 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:19.151+0000 7fcd6ec2e700 1 --2- 172.21.15.112:0/883660578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd681066a0 0x7fcd6819ded0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:19.761 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:19.151+0000 7fcd6ec2e700 1 -- 172.21.15.112:0/883660578 >> 172.21.15.112:0/883660578 conn(0x7fcd68102220 msgr2=0x7fcd68194b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:19.761 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:19.151+0000 7fcd6ec2e700 1 -- 172.21.15.112:0/883660578 shutdown_connections 2024-03-16T09:06:19.761 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:19.151+0000 7fcd6ec2e700 1 -- 172.21.15.112:0/883660578 wait complete. 2024-03-16T09:06:20.076 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:19 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' 2024-03-16T09:06:20.076 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:19 smithi112 ceph-mon[32307]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:20.077 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:19 smithi112 ceph-mon[32307]: Saving service mgr spec with placement count:2 2024-03-16T09:06:20.077 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:19 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' 2024-03-16T09:06:20.077 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:19 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' 2024-03-16T09:06:21.165 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.182+0000 7f1b8ae5d700 1 Processor -- start 2024-03-16T09:06:21.165 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.182+0000 7f1b8ae5d700 1 -- start start 2024-03-16T09:06:21.165 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.182+0000 7f1b8ae5d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84108820 0x7f1b8410ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:21.165 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.183+0000 7f1b8ae5d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8407be50 con 0x7f1b84108820 2024-03-16T09:06:21.165 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.183+0000 7f1b88bf9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84108820 0x7f1b8410ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:21.165 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.183+0000 7f1b88bf9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84108820 0x7f1b8410ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54402/0 (socket says 172.21.15.112:54402) 2024-03-16T09:06:21.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.183+0000 7f1b88bf9700 1 -- 172.21.15.112:0/2725235532 learned_addr learned my addr 172.21.15.112:0/2725235532 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:21.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.183+0000 7f1b88bf9700 1 -- 172.21.15.112:0/2725235532 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8410b150 con 0x7f1b84108820 2024-03-16T09:06:21.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.183+0000 7f1b88bf9700 1 --2- 172.21.15.112:0/2725235532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84108820 0x7f1b8410ac10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b780098a0 tx=0x7f1b78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec77980f8d9974b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:21.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.183+0000 7f1b837fe700 1 -- 172.21.15.112:0/2725235532 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b78004160 con 0x7f1b84108820 2024-03-16T09:06:21.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.183+0000 7f1b837fe700 1 -- 172.21.15.112:0/2725235532 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1b78004300 con 0x7f1b84108820 2024-03-16T09:06:21.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.183+0000 7f1b837fe700 1 -- 172.21.15.112:0/2725235532 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b78011bb0 con 0x7f1b84108820 2024-03-16T09:06:21.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.184+0000 7f1b8ae5d700 1 -- 172.21.15.112:0/2725235532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84108820 msgr2=0x7f1b8410ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:21.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.184+0000 7f1b8ae5d700 1 --2- 172.21.15.112:0/2725235532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84108820 0x7f1b8410ac10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b780098a0 tx=0x7f1b78009bb0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.184+0000 7f1b8ae5d700 1 -- 172.21.15.112:0/2725235532 shutdown_connections 2024-03-16T09:06:21.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.184+0000 7f1b8ae5d700 1 --2- 172.21.15.112:0/2725235532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84108820 0x7f1b8410ac10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.184+0000 7f1b8ae5d700 1 -- 172.21.15.112:0/2725235532 >> 172.21.15.112:0/2725235532 conn(0x7f1b84102230 msgr2=0x7f1b84104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:21.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.184+0000 7f1b8ae5d700 1 -- 172.21.15.112:0/2725235532 shutdown_connections 2024-03-16T09:06:21.166 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.184+0000 7f1b8ae5d700 1 -- 172.21.15.112:0/2725235532 wait complete. 2024-03-16T09:06:21.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.185+0000 7f1b8ae5d700 1 Processor -- start 2024-03-16T09:06:21.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.185+0000 7f1b8ae5d700 1 -- start start 2024-03-16T09:06:21.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.185+0000 7f1b8ae5d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84108820 0x7f1b8419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:21.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.185+0000 7f1b8ae5d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8410c210 con 0x7f1b84108820 2024-03-16T09:06:21.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.185+0000 7f1b88bf9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84108820 0x7f1b8419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:21.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.185+0000 7f1b88bf9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84108820 0x7f1b8419bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54406/0 (socket says 172.21.15.112:54406) 2024-03-16T09:06:21.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.185+0000 7f1b88bf9700 1 -- 172.21.15.112:0/1986369832 learned_addr learned my addr 172.21.15.112:0/1986369832 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:21.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.186+0000 7f1b88bf9700 1 -- 172.21.15.112:0/1986369832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8419c1d0 con 0x7f1b84108820 2024-03-16T09:06:21.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.186+0000 7f1b88bf9700 1 --2- 172.21.15.112:0/1986369832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84108820 0x7f1b8419bc90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b78011870 tx=0x7f1b78011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:21.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.186+0000 7f1b81ffb700 1 -- 172.21.15.112:0/1986369832 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b78003af0 con 0x7f1b84108820 2024-03-16T09:06:21.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.186+0000 7f1b81ffb700 1 -- 172.21.15.112:0/1986369832 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1b78003c90 con 0x7f1b84108820 2024-03-16T09:06:21.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.186+0000 7f1b8ae5d700 1 -- 172.21.15.112:0/1986369832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b8419c460 con 0x7f1b84108820 2024-03-16T09:06:21.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.186+0000 7f1b81ffb700 1 -- 172.21.15.112:0/1986369832 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b78019960 con 0x7f1b84108820 2024-03-16T09:06:21.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.186+0000 7f1b8ae5d700 1 -- 172.21.15.112:0/1986369832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b8419c8c0 con 0x7f1b84108820 2024-03-16T09:06:21.167 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.187+0000 7f1b81ffb700 1 -- 172.21.15.112:0/1986369832 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f1b78019c10 con 0x7f1b84108820 2024-03-16T09:06:21.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.187+0000 7f1b8ae5d700 1 -- 172.21.15.112:0/1986369832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b84063900 con 0x7f1b84108820 2024-03-16T09:06:21.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.188+0000 7f1b81ffb700 1 --2- 172.21.15.112:0/1986369832 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f1b74038f80 0x7f1b7403b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:21.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.188+0000 7f1b81ffb700 1 -- 172.21.15.112:0/1986369832 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1b7804d3f0 con 0x7f1b84108820 2024-03-16T09:06:21.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.188+0000 7f1b83fff700 1 --2- 172.21.15.112:0/1986369832 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f1b74038f80 0x7f1b7403b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:21.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.188+0000 7f1b83fff700 1 --2- 172.21.15.112:0/1986369832 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f1b74038f80 0x7f1b7403b440 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1b70006fd0 tx=0x7f1b70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:21.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.193+0000 7f1b81ffb700 1 -- 172.21.15.112:0/1986369832 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1b7801ad10 con 0x7f1b84108820 2024-03-16T09:06:21.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.422+0000 7f1b8ae5d700 1 -- 172.21.15.112:0/1986369832 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f1b8419f700 con 0x7f1b84108820 2024-03-16T09:06:21.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.424+0000 7f1b81ffb700 1 -- 172.21.15.112:0/1986369832 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f1b780172b0 con 0x7f1b84108820 2024-03-16T09:06:21.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.426+0000 7f1b8ae5d700 1 -- 172.21.15.112:0/1986369832 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f1b74038f80 msgr2=0x7f1b7403b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:21.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.426+0000 7f1b8ae5d700 1 --2- 172.21.15.112:0/1986369832 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f1b74038f80 0x7f1b7403b440 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1b70006fd0 tx=0x7f1b70006e40 comp rx=0 tx=0).stop 2024-03-16T09:06:21.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.426+0000 7f1b8ae5d700 1 -- 172.21.15.112:0/1986369832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84108820 msgr2=0x7f1b8419bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:21.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.426+0000 7f1b8ae5d700 1 --2- 172.21.15.112:0/1986369832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84108820 0x7f1b8419bc90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b78011870 tx=0x7f1b78011d70 comp rx=0 tx=0).stop 2024-03-16T09:06:21.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.427+0000 7f1b8ae5d700 1 -- 172.21.15.112:0/1986369832 shutdown_connections 2024-03-16T09:06:21.168 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.427+0000 7f1b8ae5d700 1 --2- 172.21.15.112:0/1986369832 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f1b74038f80 0x7f1b7403b440 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.427+0000 7f1b8ae5d700 1 --2- 172.21.15.112:0/1986369832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84108820 0x7f1b8419bc90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:21.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.427+0000 7f1b8ae5d700 1 -- 172.21.15.112:0/1986369832 >> 172.21.15.112:0/1986369832 conn(0x7f1b84102230 msgr2=0x7f1b84102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:21.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.427+0000 7f1b8ae5d700 1 -- 172.21.15.112:0/1986369832 shutdown_connections 2024-03-16T09:06:21.169 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:20.427+0000 7f1b8ae5d700 1 -- 172.21.15.112:0/1986369832 wait complete. 2024-03-16T09:06:21.718 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:21 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1986369832' entity='client.admin' 2024-03-16T09:06:22.841 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:22 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' 2024-03-16T09:06:22.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:22 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1208407694' entity='client.admin' 2024-03-16T09:06:22.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.015+0000 7f2b5b5a9700 1 Processor -- start 2024-03-16T09:06:22.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.016+0000 7f2b5b5a9700 1 -- start start 2024-03-16T09:06:22.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.016+0000 7f2b5b5a9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b541066a0 0x7f2b54106ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:22.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.016+0000 7f2b5b5a9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5407bdf0 con 0x7f2b541066a0 2024-03-16T09:06:22.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.016+0000 7f2b59345700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b541066a0 0x7f2b54106ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:22.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.016+0000 7f2b59345700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b541066a0 0x7f2b54106ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33668/0 (socket says 172.21.15.112:33668) 2024-03-16T09:06:22.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.016+0000 7f2b59345700 1 -- 172.21.15.112:0/2554541394 learned_addr learned my addr 172.21.15.112:0/2554541394 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:22.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.016+0000 7f2b59345700 1 -- 172.21.15.112:0/2554541394 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b54107000 con 0x7f2b541066a0 2024-03-16T09:06:22.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.017+0000 7f2b59345700 1 --2- 172.21.15.112:0/2554541394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b541066a0 0x7f2b54106ac0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2b500098a0 tx=0x7f2b50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d1e7013fe5a8cb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:22.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.017+0000 7f2b4bfff700 1 -- 172.21.15.112:0/2554541394 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b50004160 con 0x7f2b541066a0 2024-03-16T09:06:22.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.017+0000 7f2b4bfff700 1 -- 172.21.15.112:0/2554541394 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2b50004300 con 0x7f2b541066a0 2024-03-16T09:06:22.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.017+0000 7f2b4bfff700 1 -- 172.21.15.112:0/2554541394 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b50011bb0 con 0x7f2b541066a0 2024-03-16T09:06:22.857 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.018+0000 7f2b5b5a9700 1 -- 172.21.15.112:0/2554541394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b541066a0 msgr2=0x7f2b54106ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:22.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.018+0000 7f2b5b5a9700 1 --2- 172.21.15.112:0/2554541394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b541066a0 0x7f2b54106ac0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2b500098a0 tx=0x7f2b50009bb0 comp rx=0 tx=0).stop 2024-03-16T09:06:22.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.018+0000 7f2b5b5a9700 1 -- 172.21.15.112:0/2554541394 shutdown_connections 2024-03-16T09:06:22.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.018+0000 7f2b5b5a9700 1 --2- 172.21.15.112:0/2554541394 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b541066a0 0x7f2b54106ac0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:22.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.018+0000 7f2b5b5a9700 1 -- 172.21.15.112:0/2554541394 >> 172.21.15.112:0/2554541394 conn(0x7f2b54102220 msgr2=0x7f2b54104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:22.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.018+0000 7f2b5b5a9700 1 -- 172.21.15.112:0/2554541394 shutdown_connections 2024-03-16T09:06:22.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.018+0000 7f2b5b5a9700 1 -- 172.21.15.112:0/2554541394 wait complete. 2024-03-16T09:06:22.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.019+0000 7f2b5b5a9700 1 Processor -- start 2024-03-16T09:06:22.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.019+0000 7f2b5b5a9700 1 -- start start 2024-03-16T09:06:22.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.019+0000 7f2b5b5a9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b541066a0 0x7f2b5419deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:22.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.019+0000 7f2b5b5a9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2b54108130 con 0x7f2b541066a0 2024-03-16T09:06:22.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.019+0000 7f2b59345700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b541066a0 0x7f2b5419deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:22.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.019+0000 7f2b59345700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b541066a0 0x7f2b5419deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33674/0 (socket says 172.21.15.112:33674) 2024-03-16T09:06:22.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.019+0000 7f2b59345700 1 -- 172.21.15.112:0/1208407694 learned_addr learned my addr 172.21.15.112:0/1208407694 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:22.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.019+0000 7f2b59345700 1 -- 172.21.15.112:0/1208407694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5419e3f0 con 0x7f2b541066a0 2024-03-16T09:06:22.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.019+0000 7f2b59345700 1 --2- 172.21.15.112:0/1208407694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b541066a0 0x7f2b5419deb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2b50011870 tx=0x7f2b50011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:22.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.020+0000 7f2b4a7fc700 1 -- 172.21.15.112:0/1208407694 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b50003af0 con 0x7f2b541066a0 2024-03-16T09:06:22.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.020+0000 7f2b4a7fc700 1 -- 172.21.15.112:0/1208407694 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2b50003c90 con 0x7f2b541066a0 2024-03-16T09:06:22.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.020+0000 7f2b4a7fc700 1 -- 172.21.15.112:0/1208407694 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b50019960 con 0x7f2b541066a0 2024-03-16T09:06:22.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.020+0000 7f2b5b5a9700 1 -- 172.21.15.112:0/1208407694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b5419e680 con 0x7f2b541066a0 2024-03-16T09:06:22.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.020+0000 7f2b5b5a9700 1 -- 172.21.15.112:0/1208407694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b5419eae0 con 0x7f2b541066a0 2024-03-16T09:06:22.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.021+0000 7f2b5b5a9700 1 -- 172.21.15.112:0/1208407694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b54063900 con 0x7f2b541066a0 2024-03-16T09:06:22.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.021+0000 7f2b4a7fc700 1 -- 172.21.15.112:0/1208407694 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f2b50019c60 con 0x7f2b541066a0 2024-03-16T09:06:22.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.021+0000 7f2b4a7fc700 1 --2- 172.21.15.112:0/1208407694 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f2b40038ed0 0x7f2b4003b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:22.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.021+0000 7f2b4a7fc700 1 -- 172.21.15.112:0/1208407694 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2b5004d330 con 0x7f2b541066a0 2024-03-16T09:06:22.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.021+0000 7f2b58b44700 1 --2- 172.21.15.112:0/1208407694 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f2b40038ed0 0x7f2b4003b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:22.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.022+0000 7f2b58b44700 1 --2- 172.21.15.112:0/1208407694 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f2b40038ed0 0x7f2b4003b390 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2b44006fd0 tx=0x7f2b44006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:22.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.030+0000 7f2b4a7fc700 1 -- 172.21.15.112:0/1208407694 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2b5000b3e0 con 0x7f2b541066a0 2024-03-16T09:06:22.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.232+0000 7f2b5b5a9700 1 -- 172.21.15.112:0/1208407694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f2b54062ee0 con 0x7f2b541066a0 2024-03-16T09:06:22.860 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.235+0000 7f2b4a7fc700 1 -- 172.21.15.112:0/1208407694 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f2b5001a430 con 0x7f2b541066a0 2024-03-16T09:06:22.860 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.238+0000 7f2b5b5a9700 1 -- 172.21.15.112:0/1208407694 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f2b40038ed0 msgr2=0x7f2b4003b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:22.860 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.238+0000 7f2b5b5a9700 1 --2- 172.21.15.112:0/1208407694 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f2b40038ed0 0x7f2b4003b390 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2b44006fd0 tx=0x7f2b44006e40 comp rx=0 tx=0).stop 2024-03-16T09:06:22.860 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.238+0000 7f2b5b5a9700 1 -- 172.21.15.112:0/1208407694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b541066a0 msgr2=0x7f2b5419deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:22.860 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.238+0000 7f2b5b5a9700 1 --2- 172.21.15.112:0/1208407694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b541066a0 0x7f2b5419deb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2b50011870 tx=0x7f2b50011d70 comp rx=0 tx=0).stop 2024-03-16T09:06:22.860 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.238+0000 7f2b5b5a9700 1 -- 172.21.15.112:0/1208407694 shutdown_connections 2024-03-16T09:06:22.860 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.238+0000 7f2b5b5a9700 1 --2- 172.21.15.112:0/1208407694 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f2b40038ed0 0x7f2b4003b390 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:22.860 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.238+0000 7f2b5b5a9700 1 --2- 172.21.15.112:0/1208407694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b541066a0 0x7f2b5419deb0 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2b50011870 tx=0x7f2b50011d70 comp rx=0 tx=0).stop 2024-03-16T09:06:22.860 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.238+0000 7f2b5b5a9700 1 -- 172.21.15.112:0/1208407694 >> 172.21.15.112:0/1208407694 conn(0x7f2b54102220 msgr2=0x7f2b54194b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:22.860 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.238+0000 7f2b5b5a9700 1 -- 172.21.15.112:0/1208407694 shutdown_connections 2024-03-16T09:06:22.860 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:22.238+0000 7f2b5b5a9700 1 -- 172.21.15.112:0/1208407694 wait complete. 2024-03-16T09:06:22.860 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling the dashboard module... 2024-03-16T09:06:24.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:23 smithi112 ceph-mon[32307]: from='mgr.14120 172.21.15.112:0/1648013712' entity='mgr.x' 2024-03-16T09:06:24.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:23 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3570748735' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-16T09:06:24.342 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:23 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: ignoring --setuser ceph since I am not root 2024-03-16T09:06:24.342 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:23 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: ignoring --setgroup ceph since I am not root 2024-03-16T09:06:24.680 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.294+0000 7f8504866700 1 Processor -- start 2024-03-16T09:06:24.685 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.294+0000 7f8504866700 1 -- start start 2024-03-16T09:06:24.685 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.294+0000 7f8504866700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8500108870 0x7f850010ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:24.685 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.294+0000 7f8504866700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f850007be30 con 0x7f8500108870 2024-03-16T09:06:24.685 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.294+0000 7f84fe59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8500108870 0x7f850010ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:24.685 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.294+0000 7f84fe59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8500108870 0x7f850010ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33684/0 (socket says 172.21.15.112:33684) 2024-03-16T09:06:24.685 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.294+0000 7f84fe59c700 1 -- 172.21.15.112:0/1762571102 learned_addr learned my addr 172.21.15.112:0/1762571102 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:24.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.294+0000 7f84fe59c700 1 -- 172.21.15.112:0/1762571102 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850010b1a0 con 0x7f8500108870 2024-03-16T09:06:24.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.295+0000 7f84fe59c700 1 --2- 172.21.15.112:0/1762571102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8500108870 0x7f850010ac60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f84f00098a0 tx=0x7f84f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c129565ffd296e60 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:24.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.295+0000 7f84fd59a700 1 -- 172.21.15.112:0/1762571102 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f0004160 con 0x7f8500108870 2024-03-16T09:06:24.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.295+0000 7f84fd59a700 1 -- 172.21.15.112:0/1762571102 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f84f0004300 con 0x7f8500108870 2024-03-16T09:06:24.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.295+0000 7f84fd59a700 1 -- 172.21.15.112:0/1762571102 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f0011bb0 con 0x7f8500108870 2024-03-16T09:06:24.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.295+0000 7f8504866700 1 -- 172.21.15.112:0/1762571102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8500108870 msgr2=0x7f850010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:24.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.295+0000 7f8504866700 1 --2- 172.21.15.112:0/1762571102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8500108870 0x7f850010ac60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f84f00098a0 tx=0x7f84f0009bb0 comp rx=0 tx=0).stop 2024-03-16T09:06:24.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.295+0000 7f8504866700 1 -- 172.21.15.112:0/1762571102 shutdown_connections 2024-03-16T09:06:24.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.295+0000 7f8504866700 1 --2- 172.21.15.112:0/1762571102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8500108870 0x7f850010ac60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:24.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.296+0000 7f8504866700 1 -- 172.21.15.112:0/1762571102 >> 172.21.15.112:0/1762571102 conn(0x7f8500102220 msgr2=0x7f8500104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:24.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.296+0000 7f8504866700 1 -- 172.21.15.112:0/1762571102 shutdown_connections 2024-03-16T09:06:24.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.296+0000 7f8504866700 1 -- 172.21.15.112:0/1762571102 wait complete. 2024-03-16T09:06:24.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.296+0000 7f8504866700 1 Processor -- start 2024-03-16T09:06:24.686 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.296+0000 7f8504866700 1 -- start start 2024-03-16T09:06:24.687 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.296+0000 7f8504866700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8500108870 0x7f850019bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:24.687 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.296+0000 7f8504866700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f850010c210 con 0x7f8500108870 2024-03-16T09:06:24.687 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.297+0000 7f84fe59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8500108870 0x7f850019bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:24.687 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.297+0000 7f84fe59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8500108870 0x7f850019bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33688/0 (socket says 172.21.15.112:33688) 2024-03-16T09:06:24.687 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.297+0000 7f84fe59c700 1 -- 172.21.15.112:0/3570748735 learned_addr learned my addr 172.21.15.112:0/3570748735 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:24.687 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.297+0000 7f84fe59c700 1 -- 172.21.15.112:0/3570748735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850019c1e0 con 0x7f8500108870 2024-03-16T09:06:24.687 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.297+0000 7f84fe59c700 1 --2- 172.21.15.112:0/3570748735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8500108870 0x7f850019bca0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f84f0011870 tx=0x7f84f0011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:24.687 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.297+0000 7f84ef7fe700 1 -- 172.21.15.112:0/3570748735 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f0003af0 con 0x7f8500108870 2024-03-16T09:06:24.687 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.297+0000 7f8504866700 1 -- 172.21.15.112:0/3570748735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850019c470 con 0x7f8500108870 2024-03-16T09:06:24.687 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.297+0000 7f8504866700 1 -- 172.21.15.112:0/3570748735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850019c8d0 con 0x7f8500108870 2024-03-16T09:06:24.687 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.297+0000 7f84ef7fe700 1 -- 172.21.15.112:0/3570748735 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f84f0003c90 con 0x7f8500108870 2024-03-16T09:06:24.687 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.298+0000 7f84ef7fe700 1 -- 172.21.15.112:0/3570748735 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f0019960 con 0x7f8500108870 2024-03-16T09:06:24.687 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.298+0000 7f84ef7fe700 1 -- 172.21.15.112:0/3570748735 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f84f0019c10 con 0x7f8500108870 2024-03-16T09:06:24.687 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.298+0000 7f8504866700 1 -- 172.21.15.112:0/3570748735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e0005360 con 0x7f8500108870 2024-03-16T09:06:24.688 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.298+0000 7f84ef7fe700 1 --2- 172.21.15.112:0/3570748735 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f84e8038f20 0x7f84e803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:24.688 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.298+0000 7f84ef7fe700 1 -- 172.21.15.112:0/3570748735 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f84f004e3d0 con 0x7f8500108870 2024-03-16T09:06:24.688 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.299+0000 7f84fdd9b700 1 --2- 172.21.15.112:0/3570748735 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f84e8038f20 0x7f84e803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:24.688 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.299+0000 7f84fdd9b700 1 --2- 172.21.15.112:0/3570748735 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f84e8038f20 0x7f84e803b3e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f84f4006fd0 tx=0x7f84f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:24.688 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.303+0000 7f84ef7fe700 1 -- 172.21.15.112:0/3570748735 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f84f0023550 con 0x7f8500108870 2024-03-16T09:06:24.688 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.581+0000 7f8504866700 1 -- 172.21.15.112:0/3570748735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f84e0005650 con 0x7f8500108870 2024-03-16T09:06:24.688 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.937+0000 7f84ef7fe700 1 -- 172.21.15.112:0/3570748735 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f84f001ad10 con 0x7f8500108870 2024-03-16T09:06:24.688 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.939+0000 7f84ef7fe700 1 -- 172.21.15.112:0/3570748735 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f84f002dc90 con 0x7f8500108870 2024-03-16T09:06:24.688 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.940+0000 7f8504866700 1 -- 172.21.15.112:0/3570748735 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f84e8038f20 msgr2=0x7f84e803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:24.688 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.940+0000 7f8504866700 1 --2- 172.21.15.112:0/3570748735 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f84e8038f20 0x7f84e803b3e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f84f4006fd0 tx=0x7f84f4006e40 comp rx=0 tx=0).stop 2024-03-16T09:06:24.688 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.940+0000 7f8504866700 1 -- 172.21.15.112:0/3570748735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8500108870 msgr2=0x7f850019bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:24.688 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.940+0000 7f8504866700 1 --2- 172.21.15.112:0/3570748735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8500108870 0x7f850019bca0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f84f0011870 tx=0x7f84f0011d70 comp rx=0 tx=0).stop 2024-03-16T09:06:24.688 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.941+0000 7f8504866700 1 -- 172.21.15.112:0/3570748735 shutdown_connections 2024-03-16T09:06:24.688 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.941+0000 7f8504866700 1 --2- 172.21.15.112:0/3570748735 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f84e8038f20 0x7f84e803b3e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:24.689 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.941+0000 7f8504866700 1 --2- 172.21.15.112:0/3570748735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8500108870 0x7f850019bca0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:24.689 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.941+0000 7f8504866700 1 -- 172.21.15.112:0/3570748735 >> 172.21.15.112:0/3570748735 conn(0x7f8500102220 msgr2=0x7f8500102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:24.689 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.941+0000 7f8504866700 1 -- 172.21.15.112:0/3570748735 shutdown_connections 2024-03-16T09:06:24.689 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:23.941+0000 7f8504866700 1 -- 172.21.15.112:0/3570748735 wait complete. 2024-03-16T09:06:24.804 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:24 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:24.503+0000 7f5028e27200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T09:06:24.805 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:24 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:24.596+0000 7f5028e27200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T09:06:25.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:24 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3570748735' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-16T09:06:25.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:24 smithi112 ceph-mon[32307]: mgrmap e9: x(active, since 19s) 2024-03-16T09:06:25.410 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:25 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:25.264+0000 7f5028e27200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T09:06:25.410 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:25 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:25.359+0000 7f5028e27200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T09:06:25.690 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:25 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:25.506+0000 7f5028e27200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T09:06:25.691 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:25 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:25.602+0000 7f5028e27200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T09:06:25.691 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:25 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:25.687+0000 7f5028e27200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T09:06:26.197 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:25 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2913209392' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-16T09:06:26.197 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:25 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:25.956+0000 7f5028e27200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T09:06:26.197 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:26 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:26.066+0000 7f5028e27200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T09:06:26.197 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:26 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:26.153+0000 7f5028e27200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T09:06:26.204 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-03-16T09:06:26.204 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-16T09:06:26.204 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-16T09:06:26.204 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-16T09:06:26.204 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-16T09:06:26.204 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-03-16T09:06:26.204 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.135+0000 7f867db8a700 1 Processor -- start 2024-03-16T09:06:26.204 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.135+0000 7f867db8a700 1 -- start start 2024-03-16T09:06:26.204 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.136+0000 7f867db8a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8678108320 0x7f8678108740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:26.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.136+0000 7f867db8a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8678108d10 con 0x7f8678108320 2024-03-16T09:06:26.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.136+0000 7f86777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8678108320 0x7f8678108740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:26.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.136+0000 7f86777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8678108320 0x7f8678108740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33720/0 (socket says 172.21.15.112:33720) 2024-03-16T09:06:26.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.136+0000 7f86777fe700 1 -- 172.21.15.112:0/1654907142 learned_addr learned my addr 172.21.15.112:0/1654907142 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:26.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.136+0000 7f86777fe700 1 -- 172.21.15.112:0/1654907142 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8678105e20 con 0x7f8678108320 2024-03-16T09:06:26.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.136+0000 7f86777fe700 1 --2- 172.21.15.112:0/1654907142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8678108320 0x7f8678108740 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f86680098a0 tx=0x7f8668009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a492d8c23760a414 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:26.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.137+0000 7f86767fc700 1 -- 172.21.15.112:0/1654907142 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8668004160 con 0x7f8678108320 2024-03-16T09:06:26.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.137+0000 7f86767fc700 1 -- 172.21.15.112:0/1654907142 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8668004300 con 0x7f8678108320 2024-03-16T09:06:26.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.137+0000 7f86767fc700 1 -- 172.21.15.112:0/1654907142 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8668011bb0 con 0x7f8678108320 2024-03-16T09:06:26.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.137+0000 7f867db8a700 1 -- 172.21.15.112:0/1654907142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8678108320 msgr2=0x7f8678108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:26.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.137+0000 7f867db8a700 1 --2- 172.21.15.112:0/1654907142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8678108320 0x7f8678108740 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f86680098a0 tx=0x7f8668009bb0 comp rx=0 tx=0).stop 2024-03-16T09:06:26.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.137+0000 7f867db8a700 1 -- 172.21.15.112:0/1654907142 shutdown_connections 2024-03-16T09:06:26.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.137+0000 7f867db8a700 1 --2- 172.21.15.112:0/1654907142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8678108320 0x7f8678108740 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:26.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.137+0000 7f867db8a700 1 -- 172.21.15.112:0/1654907142 >> 172.21.15.112:0/1654907142 conn(0x7f8678101a20 msgr2=0x7f8678103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:26.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.138+0000 7f867db8a700 1 -- 172.21.15.112:0/1654907142 shutdown_connections 2024-03-16T09:06:26.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.138+0000 7f867db8a700 1 -- 172.21.15.112:0/1654907142 wait complete. 2024-03-16T09:06:26.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.138+0000 7f867db8a700 1 Processor -- start 2024-03-16T09:06:26.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.138+0000 7f867db8a700 1 -- start start 2024-03-16T09:06:26.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.138+0000 7f867db8a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8678108320 0x7f867819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:26.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.138+0000 7f867db8a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8678106f40 con 0x7f8678108320 2024-03-16T09:06:26.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.139+0000 7f86777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8678108320 0x7f867819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:26.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.139+0000 7f86777fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8678108320 0x7f867819df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33732/0 (socket says 172.21.15.112:33732) 2024-03-16T09:06:26.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.139+0000 7f86777fe700 1 -- 172.21.15.112:0/2913209392 learned_addr learned my addr 172.21.15.112:0/2913209392 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:26.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.139+0000 7f86777fe700 1 -- 172.21.15.112:0/2913209392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867819e490 con 0x7f8678108320 2024-03-16T09:06:26.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.139+0000 7f86777fe700 1 --2- 172.21.15.112:0/2913209392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8678108320 0x7f867819df50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8668011690 tx=0x7f8668011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:26.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.139+0000 7f8674ff9700 1 -- 172.21.15.112:0/2913209392 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8668003930 con 0x7f8678108320 2024-03-16T09:06:26.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.139+0000 7f8674ff9700 1 -- 172.21.15.112:0/2913209392 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8668003ad0 con 0x7f8678108320 2024-03-16T09:06:26.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.139+0000 7f867db8a700 1 -- 172.21.15.112:0/2913209392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867819e720 con 0x7f8678108320 2024-03-16T09:06:26.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.139+0000 7f8674ff9700 1 -- 172.21.15.112:0/2913209392 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8668019730 con 0x7f8678108320 2024-03-16T09:06:26.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.139+0000 7f867db8a700 1 -- 172.21.15.112:0/2913209392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f867819eb80 con 0x7f8678108320 2024-03-16T09:06:26.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.140+0000 7f8674ff9700 1 -- 172.21.15.112:0/2913209392 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f8668019ab0 con 0x7f8678108320 2024-03-16T09:06:26.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.140+0000 7f8674ff9700 1 --2- 172.21.15.112:0/2913209392 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f8660038f20 0x7f866003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:26.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.140+0000 7f8674ff9700 1 -- 172.21.15.112:0/2913209392 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f866804e450 con 0x7f8678108320 2024-03-16T09:06:26.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.140+0000 7f8676ffd700 1 -- 172.21.15.112:0/2913209392 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f8660038f20 msgr2=0x7f866003b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2629104617 2024-03-16T09:06:26.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.140+0000 7f8676ffd700 1 --2- 172.21.15.112:0/2913209392 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f8660038f20 0x7f866003b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:06:26.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.141+0000 7f867db8a700 1 -- 172.21.15.112:0/2913209392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8678063900 con 0x7f8678108320 2024-03-16T09:06:26.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.145+0000 7f8674ff9700 1 -- 172.21.15.112:0/2913209392 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8668003c80 con 0x7f8678108320 2024-03-16T09:06:26.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.341+0000 7f8676ffd700 1 -- 172.21.15.112:0/2913209392 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f8660038f20 msgr2=0x7f866003b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2629104617 2024-03-16T09:06:26.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.341+0000 7f8676ffd700 1 --2- 172.21.15.112:0/2913209392 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f8660038f20 0x7f866003b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T09:06:26.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.407+0000 7f867db8a700 1 -- 172.21.15.112:0/2913209392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f86781a1410 con 0x7f8678108320 2024-03-16T09:06:26.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.408+0000 7f8674ff9700 1 -- 172.21.15.112:0/2913209392 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f86781a1410 con 0x7f8678108320 2024-03-16T09:06:26.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.410+0000 7f867db8a700 1 -- 172.21.15.112:0/2913209392 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f8660038f20 msgr2=0x7f866003b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T09:06:26.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.410+0000 7f867db8a700 1 --2- 172.21.15.112:0/2913209392 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f8660038f20 0x7f866003b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:26.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.410+0000 7f867db8a700 1 -- 172.21.15.112:0/2913209392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8678108320 msgr2=0x7f867819df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:26.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.410+0000 7f867db8a700 1 --2- 172.21.15.112:0/2913209392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8678108320 0x7f867819df50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8668011690 tx=0x7f8668011ca0 comp rx=0 tx=0).stop 2024-03-16T09:06:26.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.410+0000 7f867db8a700 1 -- 172.21.15.112:0/2913209392 shutdown_connections 2024-03-16T09:06:26.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.410+0000 7f867db8a700 1 --2- 172.21.15.112:0/2913209392 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f8660038f20 0x7f866003b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:26.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.410+0000 7f867db8a700 1 --2- 172.21.15.112:0/2913209392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8678108320 0x7f867819df50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:26.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.410+0000 7f867db8a700 1 -- 172.21.15.112:0/2913209392 >> 172.21.15.112:0/2913209392 conn(0x7f8678101a20 msgr2=0x7f8678194610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:26.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.411+0000 7f867db8a700 1 -- 172.21.15.112:0/2913209392 shutdown_connections 2024-03-16T09:06:26.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:25.411+0000 7f867db8a700 1 -- 172.21.15.112:0/2913209392 wait complete. 2024-03-16T09:06:26.208 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for the mgr to restart... 2024-03-16T09:06:26.209 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr epoch 9... 2024-03-16T09:06:26.531 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:26 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:26.253+0000 7f5028e27200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T09:06:26.531 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:26 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:26.435+0000 7f5028e27200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T09:06:27.529 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:27 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:27.397+0000 7f5028e27200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T09:06:27.842 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:27 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:27.526+0000 7f5028e27200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T09:06:28.144 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:27 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:27.856+0000 7f5028e27200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T09:06:28.144 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:27 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:27.945+0000 7f5028e27200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T09:06:28.144 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:28 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:28.141+0000 7f5028e27200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:06:28.591 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:28 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:28.325+0000 7f5028e27200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T09:06:28.866 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:28 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:28.608+0000 7f5028e27200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:06:29.342 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:28 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:28.863+0000 7f5028e27200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T09:06:29.342 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:29 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:29.099+0000 7f5028e27200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T09:06:30.342 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:29 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:29.900+0000 7f5028e27200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T09:06:30.781 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:30 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:30.466+0000 7f5028e27200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T09:06:30.781 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:30 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:30.563+0000 7f5028e27200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T09:06:30.781 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:06:30 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:06:30.779+0000 7f5028e27200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T09:06:31.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:30 smithi112 ceph-mon[32307]: Active manager daemon x restarted 2024-03-16T09:06:31.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:30 smithi112 ceph-mon[32307]: Activating manager daemon x 2024-03-16T09:06:31.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:30 smithi112 ceph-mon[32307]: osdmap e3: 0 total, 0 up, 0 in 2024-03-16T09:06:31.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:30 smithi112 ceph-mon[32307]: mgrmap e10: x(active, starting, since 0.00335358s) 2024-03-16T09:06:31.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:30 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:06:31.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:30 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-16T09:06:31.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:30 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-16T09:06:31.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:30 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-16T09:06:31.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:30 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-16T09:06:31.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:30 smithi112 ceph-mon[32307]: Manager daemon x is now available 2024-03-16T09:06:31.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:30 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:31.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:30 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:32.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-16T09:06:32.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-16T09:06:32.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:31 smithi112 ceph-mon[32307]: mgrmap e11: x(active, since 1.00657s) 2024-03-16T09:06:32.814 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-03-16T09:06:32.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-16T09:06:32.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-16T09:06:32.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-03-16T09:06:32.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.629+0000 7f7d2927d700 1 Processor -- start 2024-03-16T09:06:32.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.629+0000 7f7d2927d700 1 -- start start 2024-03-16T09:06:32.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.629+0000 7f7d2927d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24106090 0x7f7d241064b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:32.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.629+0000 7f7d2927d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d24106a80 con 0x7f7d24106090 2024-03-16T09:06:32.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.629+0000 7f7d22ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24106090 0x7f7d241064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:32.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.629+0000 7f7d22ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24106090 0x7f7d241064b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33744/0 (socket says 172.21.15.112:33744) 2024-03-16T09:06:32.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.629+0000 7f7d22ffd700 1 -- 172.21.15.112:0/3331649745 learned_addr learned my addr 172.21.15.112:0/3331649745 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:32.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.629+0000 7f7d22ffd700 1 -- 172.21.15.112:0/3331649745 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d24109020 con 0x7f7d24106090 2024-03-16T09:06:32.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.629+0000 7f7d22ffd700 1 --2- 172.21.15.112:0/3331649745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24106090 0x7f7d241064b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7d140098a0 tx=0x7f7d14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb00df4120822c82 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:32.815 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.629+0000 7f7d21ffb700 1 -- 172.21.15.112:0/3331649745 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d14004160 con 0x7f7d24106090 2024-03-16T09:06:32.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.630+0000 7f7d21ffb700 1 -- 172.21.15.112:0/3331649745 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7d14004300 con 0x7f7d24106090 2024-03-16T09:06:32.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.630+0000 7f7d21ffb700 1 -- 172.21.15.112:0/3331649745 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d14011bb0 con 0x7f7d24106090 2024-03-16T09:06:32.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.630+0000 7f7d2927d700 1 -- 172.21.15.112:0/3331649745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24106090 msgr2=0x7f7d241064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:32.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.630+0000 7f7d2927d700 1 --2- 172.21.15.112:0/3331649745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24106090 0x7f7d241064b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7d140098a0 tx=0x7f7d14009bb0 comp rx=0 tx=0).stop 2024-03-16T09:06:32.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.630+0000 7f7d2927d700 1 -- 172.21.15.112:0/3331649745 shutdown_connections 2024-03-16T09:06:32.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.630+0000 7f7d2927d700 1 --2- 172.21.15.112:0/3331649745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24106090 0x7f7d241064b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:32.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.630+0000 7f7d2927d700 1 -- 172.21.15.112:0/3331649745 >> 172.21.15.112:0/3331649745 conn(0x7f7d24101a20 msgr2=0x7f7d24103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:32.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.630+0000 7f7d2927d700 1 -- 172.21.15.112:0/3331649745 shutdown_connections 2024-03-16T09:06:32.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.630+0000 7f7d2927d700 1 -- 172.21.15.112:0/3331649745 wait complete. 2024-03-16T09:06:32.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.631+0000 7f7d2927d700 1 Processor -- start 2024-03-16T09:06:32.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.631+0000 7f7d2927d700 1 -- start start 2024-03-16T09:06:32.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.631+0000 7f7d2927d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24106090 0x7f7d2419bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:32.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.631+0000 7f7d2927d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2410c3d0 con 0x7f7d24106090 2024-03-16T09:06:32.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.631+0000 7f7d22ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24106090 0x7f7d2419bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:32.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.631+0000 7f7d22ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24106090 0x7f7d2419bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33748/0 (socket says 172.21.15.112:33748) 2024-03-16T09:06:32.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.631+0000 7f7d22ffd700 1 -- 172.21.15.112:0/1551531253 learned_addr learned my addr 172.21.15.112:0/1551531253 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:32.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.631+0000 7f7d22ffd700 1 -- 172.21.15.112:0/1551531253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2419c2d0 con 0x7f7d24106090 2024-03-16T09:06:32.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.632+0000 7f7d22ffd700 1 --2- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24106090 0x7f7d2419bd90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7d14006af0 tx=0x7f7d14011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:32.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.632+0000 7f7d0bfff700 1 -- 172.21.15.112:0/1551531253 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d14003af0 con 0x7f7d24106090 2024-03-16T09:06:32.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.632+0000 7f7d0bfff700 1 -- 172.21.15.112:0/1551531253 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7d14003c90 con 0x7f7d24106090 2024-03-16T09:06:32.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.632+0000 7f7d0bfff700 1 -- 172.21.15.112:0/1551531253 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d14024900 con 0x7f7d24106090 2024-03-16T09:06:32.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.632+0000 7f7d2927d700 1 -- 172.21.15.112:0/1551531253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2419c560 con 0x7f7d24106090 2024-03-16T09:06:32.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.632+0000 7f7d2927d700 1 -- 172.21.15.112:0/1551531253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d2419f210 con 0x7f7d24106090 2024-03-16T09:06:32.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.633+0000 7f7d0bfff700 1 -- 172.21.15.112:0/1551531253 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f7d14022020 con 0x7f7d24106090 2024-03-16T09:06:32.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.633+0000 7f7d0bfff700 1 --2- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f7d0c038f80 0x7f7d0c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:32.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.633+0000 7f7d0bfff700 1 -- 172.21.15.112:0/1551531253 --> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7d0c03bb50 con 0x7f7d0c038f80 2024-03-16T09:06:32.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.633+0000 7f7d0bfff700 1 -- 172.21.15.112:0/1551531253 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7d1404cf80 con 0x7f7d24106090 2024-03-16T09:06:32.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.633+0000 7f7d227fc700 1 -- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f7d0c038f80 msgr2=0x7f7d0c03b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2629104617 2024-03-16T09:06:32.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.633+0000 7f7d227fc700 1 --2- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f7d0c038f80 0x7f7d0c03b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:06:32.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.834+0000 7f7d227fc700 1 -- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f7d0c038f80 msgr2=0x7f7d0c03b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2629104617 2024-03-16T09:06:32.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:26.834+0000 7f7d227fc700 1 --2- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f7d0c038f80 0x7f7d0c03b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T09:06:32.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:27.235+0000 7f7d227fc700 1 -- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f7d0c038f80 msgr2=0x7f7d0c03b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2629104617 2024-03-16T09:06:32.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:27.235+0000 7f7d227fc700 1 --2- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f7d0c038f80 0x7f7d0c03b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-16T09:06:32.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:28.036+0000 7f7d227fc700 1 -- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f7d0c038f80 msgr2=0x7f7d0c03b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2629104617 2024-03-16T09:06:32.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:28.036+0000 7f7d227fc700 1 --2- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f7d0c038f80 0x7f7d0c03b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-16T09:06:32.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:29.637+0000 7f7d227fc700 1 -- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f7d0c038f80 msgr2=0x7f7d0c03b440 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2629104617 2024-03-16T09:06:32.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:29.637+0000 7f7d227fc700 1 --2- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f7d0c038f80 0x7f7d0c03b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-16T09:06:32.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:30.794+0000 7f7d0bfff700 1 -- 172.21.15.112:0/1551531253 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mgrmap(e 10) ==== 45886+0+0 (secure 0 0 0) 0x7f7d1401a430 con 0x7f7d24106090 2024-03-16T09:06:32.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:30.794+0000 7f7d0bfff700 1 -- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f7d0c038f80 msgr2=0x7f7d0c03b440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T09:06:32.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:30.794+0000 7f7d0bfff700 1 --2- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f7d0c038f80 0x7f7d0c03b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:32.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:31.796+0000 7f7d0bfff700 1 -- 172.21.15.112:0/1551531253 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 11) ==== 46013+0+0 (secure 0 0 0) 0x7f7d1404e7f0 con 0x7f7d24106090 2024-03-16T09:06:32.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:31.796+0000 7f7d0bfff700 1 --2- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7d0c03ca30 0x7f7d0c03ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:32.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:31.796+0000 7f7d0bfff700 1 -- 172.21.15.112:0/1551531253 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7d0c03bb50 con 0x7f7d0c03ca30 2024-03-16T09:06:32.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:31.797+0000 7f7d227fc700 1 --2- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7d0c03ca30 0x7f7d0c03ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:32.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:31.797+0000 7f7d227fc700 1 --2- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7d0c03ca30 0x7f7d0c03ee20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7d18003a10 tx=0x7f7d18009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:32.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:31.798+0000 7f7d0bfff700 1 -- 172.21.15.112:0/1551531253 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f7d0c03bb50 con 0x7f7d0c03ca30 2024-03-16T09:06:32.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:31.804+0000 7f7d2927d700 1 -- 172.21.15.112:0/1551531253 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7d24063900 con 0x7f7d0c03ca30 2024-03-16T09:06:32.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:31.805+0000 7f7d0bfff700 1 -- 172.21.15.112:0/1551531253 <== mgr.14152 v2:172.21.15.112:6800/2628870229 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f7d24063900 con 0x7f7d0c03ca30 2024-03-16T09:06:32.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:31.805+0000 7f7d2927d700 1 -- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7d0c03ca30 msgr2=0x7f7d0c03ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:32.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:31.805+0000 7f7d2927d700 1 --2- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7d0c03ca30 0x7f7d0c03ee20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7d18003a10 tx=0x7f7d18009270 comp rx=0 tx=0).stop 2024-03-16T09:06:32.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:31.805+0000 7f7d2927d700 1 -- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24106090 msgr2=0x7f7d2419bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:32.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:31.805+0000 7f7d2927d700 1 --2- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24106090 0x7f7d2419bd90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7d14006af0 tx=0x7f7d14011c30 comp rx=0 tx=0).stop 2024-03-16T09:06:32.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:31.805+0000 7f7d2927d700 1 -- 172.21.15.112:0/1551531253 shutdown_connections 2024-03-16T09:06:32.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:31.805+0000 7f7d2927d700 1 --2- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7d0c03ca30 0x7f7d0c03ee20 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:32.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:31.805+0000 7f7d2927d700 1 --2- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24106090 0x7f7d2419bd90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:32.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:31.805+0000 7f7d2927d700 1 --2- 172.21.15.112:0/1551531253 >> [v2:172.21.15.112:6800/2629104617,v1:172.21.15.112:6801/2629104617] conn(0x7f7d0c038f80 0x7f7d0c03b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:32.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:31.805+0000 7f7d2927d700 1 -- 172.21.15.112:0/1551531253 >> 172.21.15.112:0/1551531253 conn(0x7f7d24101a20 msgr2=0x7f7d24102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:32.820 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:31.805+0000 7f7d2927d700 1 -- 172.21.15.112:0/1551531253 shutdown_connections 2024-03-16T09:06:32.820 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:31.805+0000 7f7d2927d700 1 -- 172.21.15.112:0/1551531253 wait complete. 2024-03-16T09:06:32.820 INFO:teuthology.orchestra.run.smithi112.stdout:mgr epoch 9 is available 2024-03-16T09:06:32.820 INFO:teuthology.orchestra.run.smithi112.stdout:Generating a dashboard self-signed certificate... 2024-03-16T09:06:33.048 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:32 smithi112 ceph-mon[32307]: [16/Mar/2024:09:06:31] ENGINE Bus STARTING 2024-03-16T09:06:33.048 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:32 smithi112 ceph-mon[32307]: [16/Mar/2024:09:06:31] ENGINE Serving on http://172.21.15.112:8765 2024-03-16T09:06:33.048 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:32 smithi112 ceph-mon[32307]: [16/Mar/2024:09:06:31] ENGINE Serving on https://172.21.15.112:7150 2024-03-16T09:06:33.048 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:32 smithi112 ceph-mon[32307]: [16/Mar/2024:09:06:31] ENGINE Bus STARTED 2024-03-16T09:06:33.048 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:32 smithi112 ceph-mon[32307]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-16T09:06:33.049 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:32 smithi112 ceph-mon[32307]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-16T09:06:33.049 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:32 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:06:34.178 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-16T09:06:34.178 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.257+0000 7fc512e7c700 1 Processor -- start 2024-03-16T09:06:34.178 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.257+0000 7fc512e7c700 1 -- start start 2024-03-16T09:06:34.178 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.257+0000 7fc512e7c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc50c108850 0x7fc50c10ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:34.178 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.257+0000 7fc512e7c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c07be50 con 0x7fc50c108850 2024-03-16T09:06:34.179 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.257+0000 7fc510c18700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc50c108850 0x7fc50c10ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:34.179 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.257+0000 7fc510c18700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc50c108850 0x7fc50c10ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48918/0 (socket says 172.21.15.112:48918) 2024-03-16T09:06:34.179 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.257+0000 7fc510c18700 1 -- 172.21.15.112:0/925226958 learned_addr learned my addr 172.21.15.112:0/925226958 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:34.179 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.258+0000 7fc510c18700 1 -- 172.21.15.112:0/925226958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50c10b180 con 0x7fc50c108850 2024-03-16T09:06:34.179 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.258+0000 7fc510c18700 1 --2- 172.21.15.112:0/925226958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc50c108850 0x7fc50c10ac40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc5000098a0 tx=0x7fc500009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c79051aa8a682b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:34.179 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.258+0000 7fc50b7fe700 1 -- 172.21.15.112:0/925226958 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc500004160 con 0x7fc50c108850 2024-03-16T09:06:34.179 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.258+0000 7fc50b7fe700 1 -- 172.21.15.112:0/925226958 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc500004300 con 0x7fc50c108850 2024-03-16T09:06:34.179 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.258+0000 7fc50b7fe700 1 -- 172.21.15.112:0/925226958 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc500011bb0 con 0x7fc50c108850 2024-03-16T09:06:34.179 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.258+0000 7fc512e7c700 1 -- 172.21.15.112:0/925226958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc50c108850 msgr2=0x7fc50c10ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:34.179 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.258+0000 7fc512e7c700 1 --2- 172.21.15.112:0/925226958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc50c108850 0x7fc50c10ac40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc5000098a0 tx=0x7fc500009bb0 comp rx=0 tx=0).stop 2024-03-16T09:06:34.179 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.258+0000 7fc512e7c700 1 -- 172.21.15.112:0/925226958 shutdown_connections 2024-03-16T09:06:34.179 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.258+0000 7fc512e7c700 1 --2- 172.21.15.112:0/925226958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc50c108850 0x7fc50c10ac40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:34.179 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.258+0000 7fc512e7c700 1 -- 172.21.15.112:0/925226958 >> 172.21.15.112:0/925226958 conn(0x7fc50c102220 msgr2=0x7fc50c104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:34.180 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.258+0000 7fc512e7c700 1 -- 172.21.15.112:0/925226958 shutdown_connections 2024-03-16T09:06:34.180 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.259+0000 7fc512e7c700 1 -- 172.21.15.112:0/925226958 wait complete. 2024-03-16T09:06:34.180 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.259+0000 7fc512e7c700 1 Processor -- start 2024-03-16T09:06:34.180 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.259+0000 7fc512e7c700 1 -- start start 2024-03-16T09:06:34.180 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.259+0000 7fc512e7c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc50c108850 0x7fc50c19bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:34.180 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.259+0000 7fc512e7c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c10c240 con 0x7fc50c108850 2024-03-16T09:06:34.180 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.259+0000 7fc510c18700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc50c108850 0x7fc50c19bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:34.180 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.259+0000 7fc510c18700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc50c108850 0x7fc50c19bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48928/0 (socket says 172.21.15.112:48928) 2024-03-16T09:06:34.180 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.259+0000 7fc510c18700 1 -- 172.21.15.112:0/1938474605 learned_addr learned my addr 172.21.15.112:0/1938474605 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:34.180 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.259+0000 7fc510c18700 1 -- 172.21.15.112:0/1938474605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50c19c240 con 0x7fc50c108850 2024-03-16T09:06:34.180 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.260+0000 7fc510c18700 1 --2- 172.21.15.112:0/1938474605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc50c108850 0x7fc50c19bd00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc500011870 tx=0x7fc500011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:34.180 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.260+0000 7fc509ffb700 1 -- 172.21.15.112:0/1938474605 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc500003af0 con 0x7fc50c108850 2024-03-16T09:06:34.180 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.260+0000 7fc509ffb700 1 -- 172.21.15.112:0/1938474605 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc500003c90 con 0x7fc50c108850 2024-03-16T09:06:34.180 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.260+0000 7fc509ffb700 1 -- 172.21.15.112:0/1938474605 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc500019960 con 0x7fc50c108850 2024-03-16T09:06:34.181 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.260+0000 7fc512e7c700 1 -- 172.21.15.112:0/1938474605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc50c19c4d0 con 0x7fc50c108850 2024-03-16T09:06:34.181 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.260+0000 7fc512e7c700 1 -- 172.21.15.112:0/1938474605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc50c19c930 con 0x7fc50c108850 2024-03-16T09:06:34.181 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.261+0000 7fc509ffb700 1 -- 172.21.15.112:0/1938474605 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) ==== 46013+0+0 (secure 0 0 0) 0x7fc500019c10 con 0x7fc50c108850 2024-03-16T09:06:34.181 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.261+0000 7fc512e7c700 1 -- 172.21.15.112:0/1938474605 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc50c063900 con 0x7fc50c108850 2024-03-16T09:06:34.181 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.261+0000 7fc509ffb700 1 --2- 172.21.15.112:0/1938474605 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc4fc038e60 0x7fc4fc03b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:34.181 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.261+0000 7fc509ffb700 1 -- 172.21.15.112:0/1938474605 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc50004d2a0 con 0x7fc50c108850 2024-03-16T09:06:34.181 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.261+0000 7fc50bfff700 1 --2- 172.21.15.112:0/1938474605 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc4fc038e60 0x7fc4fc03b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:34.181 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.261+0000 7fc50bfff700 1 --2- 172.21.15.112:0/1938474605 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc4fc038e60 0x7fc4fc03b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8006fd0 tx=0x7fc4f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:34.181 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.266+0000 7fc509ffb700 1 -- 172.21.15.112:0/1938474605 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc50000b3e0 con 0x7fc50c108850 2024-03-16T09:06:34.181 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.461+0000 7fc509ffb700 1 -- 172.21.15.112:0/1938474605 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7fc50001a430 con 0x7fc50c108850 2024-03-16T09:06:34.181 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.474+0000 7fc512e7c700 1 -- 172.21.15.112:0/1938474605 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fc50c1033d0 con 0x7fc4fc038e60 2024-03-16T09:06:34.181 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.620+0000 7fc509ffb700 1 -- 172.21.15.112:0/1938474605 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fc50c1033d0 con 0x7fc4fc038e60 2024-03-16T09:06:34.181 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.622+0000 7fc512e7c700 1 -- 172.21.15.112:0/1938474605 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc4fc038e60 msgr2=0x7fc4fc03b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:34.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.622+0000 7fc512e7c700 1 --2- 172.21.15.112:0/1938474605 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc4fc038e60 0x7fc4fc03b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8006fd0 tx=0x7fc4f8006e40 comp rx=0 tx=0).stop 2024-03-16T09:06:34.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.622+0000 7fc512e7c700 1 -- 172.21.15.112:0/1938474605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc50c108850 msgr2=0x7fc50c19bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:34.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.622+0000 7fc512e7c700 1 --2- 172.21.15.112:0/1938474605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc50c108850 0x7fc50c19bd00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc500011870 tx=0x7fc500011d70 comp rx=0 tx=0).stop 2024-03-16T09:06:34.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.622+0000 7fc512e7c700 1 -- 172.21.15.112:0/1938474605 shutdown_connections 2024-03-16T09:06:34.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.622+0000 7fc512e7c700 1 --2- 172.21.15.112:0/1938474605 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc4fc038e60 0x7fc4fc03b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:34.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.622+0000 7fc512e7c700 1 --2- 172.21.15.112:0/1938474605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc50c108850 0x7fc50c19bd00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:34.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.622+0000 7fc512e7c700 1 -- 172.21.15.112:0/1938474605 >> 172.21.15.112:0/1938474605 conn(0x7fc50c102220 msgr2=0x7fc50c102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:34.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.622+0000 7fc512e7c700 1 -- 172.21.15.112:0/1938474605 shutdown_connections 2024-03-16T09:06:34.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:33.622+0000 7fc512e7c700 1 -- 172.21.15.112:0/1938474605 wait complete. 2024-03-16T09:06:34.182 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial admin user... 2024-03-16T09:06:34.557 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:34 smithi112 ceph-mon[32307]: mgrmap e12: x(active, since 2s) 2024-03-16T09:06:34.557 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:34 smithi112 ceph-mon[32307]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:34.557 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:34 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:06:34.557 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:34 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:06:35.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$2BhVDQHEuJzlIE6IFGExROcHFl/WYaM1crr6/3m0IsXFxPIkZ7w9K", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710579995, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-16T09:06:35.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.583+0000 7ffa2941e700 1 Processor -- start 2024-03-16T09:06:35.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.583+0000 7ffa2941e700 1 -- start start 2024-03-16T09:06:35.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.584+0000 7ffa2941e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa24106670 0x7ffa24106a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:35.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.584+0000 7ffa2941e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2407bdf0 con 0x7ffa24106670 2024-03-16T09:06:35.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.584+0000 7ffa22ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa24106670 0x7ffa24106a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:35.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.584+0000 7ffa22ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa24106670 0x7ffa24106a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48936/0 (socket says 172.21.15.112:48936) 2024-03-16T09:06:35.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.584+0000 7ffa22ffd700 1 -- 172.21.15.112:0/1824487382 learned_addr learned my addr 172.21.15.112:0/1824487382 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:35.619 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.584+0000 7ffa22ffd700 1 -- 172.21.15.112:0/1824487382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa24106fd0 con 0x7ffa24106670 2024-03-16T09:06:35.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.584+0000 7ffa22ffd700 1 --2- 172.21.15.112:0/1824487382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa24106670 0x7ffa24106a90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffa140098a0 tx=0x7ffa14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3708d7bbeef44c18 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:35.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.584+0000 7ffa227fc700 1 -- 172.21.15.112:0/1824487382 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa14004160 con 0x7ffa24106670 2024-03-16T09:06:35.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.584+0000 7ffa227fc700 1 -- 172.21.15.112:0/1824487382 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ffa14004300 con 0x7ffa24106670 2024-03-16T09:06:35.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.584+0000 7ffa227fc700 1 -- 172.21.15.112:0/1824487382 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa14011bb0 con 0x7ffa24106670 2024-03-16T09:06:35.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.585+0000 7ffa2941e700 1 -- 172.21.15.112:0/1824487382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa24106670 msgr2=0x7ffa24106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:35.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.585+0000 7ffa2941e700 1 --2- 172.21.15.112:0/1824487382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa24106670 0x7ffa24106a90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffa140098a0 tx=0x7ffa14009bb0 comp rx=0 tx=0).stop 2024-03-16T09:06:35.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.585+0000 7ffa2941e700 1 -- 172.21.15.112:0/1824487382 shutdown_connections 2024-03-16T09:06:35.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.585+0000 7ffa2941e700 1 --2- 172.21.15.112:0/1824487382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa24106670 0x7ffa24106a90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:35.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.585+0000 7ffa2941e700 1 -- 172.21.15.112:0/1824487382 >> 172.21.15.112:0/1824487382 conn(0x7ffa24102230 msgr2=0x7ffa24104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:35.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.585+0000 7ffa2941e700 1 -- 172.21.15.112:0/1824487382 shutdown_connections 2024-03-16T09:06:35.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.585+0000 7ffa2941e700 1 -- 172.21.15.112:0/1824487382 wait complete. 2024-03-16T09:06:35.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.585+0000 7ffa2941e700 1 Processor -- start 2024-03-16T09:06:35.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.585+0000 7ffa2941e700 1 -- start start 2024-03-16T09:06:35.620 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.586+0000 7ffa2941e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa24106670 0x7ffa2419dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:35.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.586+0000 7ffa2941e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffa24108100 con 0x7ffa24106670 2024-03-16T09:06:35.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.586+0000 7ffa22ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa24106670 0x7ffa2419dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:35.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.586+0000 7ffa22ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa24106670 0x7ffa2419dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48938/0 (socket says 172.21.15.112:48938) 2024-03-16T09:06:35.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.586+0000 7ffa22ffd700 1 -- 172.21.15.112:0/1436094985 learned_addr learned my addr 172.21.15.112:0/1436094985 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:35.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.586+0000 7ffa22ffd700 1 -- 172.21.15.112:0/1436094985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2419e400 con 0x7ffa24106670 2024-03-16T09:06:35.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.586+0000 7ffa22ffd700 1 --2- 172.21.15.112:0/1436094985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa24106670 0x7ffa2419dec0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffa14011870 tx=0x7ffa14011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:35.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.586+0000 7ffa20ff9700 1 -- 172.21.15.112:0/1436094985 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa14003af0 con 0x7ffa24106670 2024-03-16T09:06:35.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.586+0000 7ffa20ff9700 1 -- 172.21.15.112:0/1436094985 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ffa14003c90 con 0x7ffa24106670 2024-03-16T09:06:35.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.586+0000 7ffa20ff9700 1 -- 172.21.15.112:0/1436094985 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa14019960 con 0x7ffa24106670 2024-03-16T09:06:35.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.586+0000 7ffa2941e700 1 -- 172.21.15.112:0/1436094985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2419e690 con 0x7ffa24106670 2024-03-16T09:06:35.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.586+0000 7ffa2941e700 1 -- 172.21.15.112:0/1436094985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa2419eaf0 con 0x7ffa24106670 2024-03-16T09:06:35.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.587+0000 7ffa20ff9700 1 -- 172.21.15.112:0/1436094985 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7ffa14019c60 con 0x7ffa24106670 2024-03-16T09:06:35.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.587+0000 7ffa20ff9700 1 --2- 172.21.15.112:0/1436094985 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ffa0c038f20 0x7ffa0c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:35.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.587+0000 7ffa2941e700 1 -- 172.21.15.112:0/1436094985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa24063900 con 0x7ffa24106670 2024-03-16T09:06:35.621 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.587+0000 7ffa20ff9700 1 -- 172.21.15.112:0/1436094985 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffa1404d4d0 con 0x7ffa24106670 2024-03-16T09:06:35.622 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.588+0000 7ffa13fff700 1 --2- 172.21.15.112:0/1436094985 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ffa0c038f20 0x7ffa0c03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:35.622 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.588+0000 7ffa13fff700 1 --2- 172.21.15.112:0/1436094985 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ffa0c038f20 0x7ffa0c03b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffa18006fd0 tx=0x7ffa18006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:35.622 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.592+0000 7ffa20ff9700 1 -- 172.21.15.112:0/1436094985 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ffa1400b3e0 con 0x7ffa24106670 2024-03-16T09:06:35.622 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:34.812+0000 7ffa2941e700 1 -- 172.21.15.112:0/1436094985 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7ffa24197a40 con 0x7ffa0c038f20 2024-03-16T09:06:35.622 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:35.040+0000 7ffa20ff9700 1 -- 172.21.15.112:0/1436094985 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7ffa24197a40 con 0x7ffa0c038f20 2024-03-16T09:06:35.622 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:35.042+0000 7ffa2941e700 1 -- 172.21.15.112:0/1436094985 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ffa0c038f20 msgr2=0x7ffa0c03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:35.622 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:35.042+0000 7ffa2941e700 1 --2- 172.21.15.112:0/1436094985 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ffa0c038f20 0x7ffa0c03b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffa18006fd0 tx=0x7ffa18006e40 comp rx=0 tx=0).stop 2024-03-16T09:06:35.622 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:35.042+0000 7ffa2941e700 1 -- 172.21.15.112:0/1436094985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa24106670 msgr2=0x7ffa2419dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:35.622 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:35.042+0000 7ffa2941e700 1 --2- 172.21.15.112:0/1436094985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa24106670 0x7ffa2419dec0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffa14011870 tx=0x7ffa14011d70 comp rx=0 tx=0).stop 2024-03-16T09:06:35.622 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:35.042+0000 7ffa2941e700 1 -- 172.21.15.112:0/1436094985 shutdown_connections 2024-03-16T09:06:35.622 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:35.042+0000 7ffa2941e700 1 --2- 172.21.15.112:0/1436094985 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ffa0c038f20 0x7ffa0c03b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:35.622 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:35.042+0000 7ffa2941e700 1 --2- 172.21.15.112:0/1436094985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa24106670 0x7ffa2419dec0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:35.622 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:35.042+0000 7ffa2941e700 1 -- 172.21.15.112:0/1436094985 >> 172.21.15.112:0/1436094985 conn(0x7ffa24102230 msgr2=0x7ffa24194b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:35.623 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:35.042+0000 7ffa2941e700 1 -- 172.21.15.112:0/1436094985 shutdown_connections 2024-03-16T09:06:35.623 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:35.042+0000 7ffa2941e700 1 -- 172.21.15.112:0/1436094985 wait complete. 2024-03-16T09:06:35.623 INFO:teuthology.orchestra.run.smithi112.stdout:Fetching dashboard port number... 2024-03-16T09:06:35.947 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:35 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:06:35.947 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:35 smithi112 ceph-mon[32307]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:35.947 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:35 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:06:37.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:36 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2186470926' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-16T09:06:37.321 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 8443 2024-03-16T09:06:37.321 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.046+0000 7fcfd9ffd700 1 Processor -- start 2024-03-16T09:06:37.321 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.046+0000 7fcfd9ffd700 1 -- start start 2024-03-16T09:06:37.321 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.046+0000 7fcfd9ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd41088c0 0x7fcfd410acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:37.321 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.046+0000 7fcfd9ffd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd407bdb0 con 0x7fcfd41088c0 2024-03-16T09:06:37.322 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.046+0000 7fcfd37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd41088c0 0x7fcfd410acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:37.322 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.046+0000 7fcfd37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd41088c0 0x7fcfd410acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48944/0 (socket says 172.21.15.112:48944) 2024-03-16T09:06:37.322 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.046+0000 7fcfd37fe700 1 -- 172.21.15.112:0/3900606210 learned_addr learned my addr 172.21.15.112:0/3900606210 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:37.322 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.047+0000 7fcfd37fe700 1 -- 172.21.15.112:0/3900606210 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd410b1f0 con 0x7fcfd41088c0 2024-03-16T09:06:37.322 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.047+0000 7fcfd37fe700 1 --2- 172.21.15.112:0/3900606210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd41088c0 0x7fcfd410acb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40098a0 tx=0x7fcfc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507a58c5decc547 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:37.322 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.047+0000 7fcfd27fc700 1 -- 172.21.15.112:0/3900606210 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfc4004160 con 0x7fcfd41088c0 2024-03-16T09:06:37.322 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.047+0000 7fcfd27fc700 1 -- 172.21.15.112:0/3900606210 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcfc4004300 con 0x7fcfd41088c0 2024-03-16T09:06:37.322 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.047+0000 7fcfd27fc700 1 -- 172.21.15.112:0/3900606210 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfc4011bb0 con 0x7fcfd41088c0 2024-03-16T09:06:37.322 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.048+0000 7fcfd9ffd700 1 -- 172.21.15.112:0/3900606210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd41088c0 msgr2=0x7fcfd410acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:37.322 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.048+0000 7fcfd9ffd700 1 --2- 172.21.15.112:0/3900606210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd41088c0 0x7fcfd410acb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40098a0 tx=0x7fcfc4009bb0 comp rx=0 tx=0).stop 2024-03-16T09:06:37.322 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.048+0000 7fcfd9ffd700 1 -- 172.21.15.112:0/3900606210 shutdown_connections 2024-03-16T09:06:37.322 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.048+0000 7fcfd9ffd700 1 --2- 172.21.15.112:0/3900606210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd41088c0 0x7fcfd410acb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:37.322 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.048+0000 7fcfd9ffd700 1 -- 172.21.15.112:0/3900606210 >> 172.21.15.112:0/3900606210 conn(0x7fcfd4102230 msgr2=0x7fcfd4104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:37.323 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.048+0000 7fcfd9ffd700 1 -- 172.21.15.112:0/3900606210 shutdown_connections 2024-03-16T09:06:37.323 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.048+0000 7fcfd9ffd700 1 -- 172.21.15.112:0/3900606210 wait complete. 2024-03-16T09:06:37.323 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.049+0000 7fcfd9ffd700 1 Processor -- start 2024-03-16T09:06:37.323 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.049+0000 7fcfd9ffd700 1 -- start start 2024-03-16T09:06:37.323 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.049+0000 7fcfd9ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd41088c0 0x7fcfd419bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:37.323 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.049+0000 7fcfd9ffd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd410c1f0 con 0x7fcfd41088c0 2024-03-16T09:06:37.323 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.049+0000 7fcfd37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd41088c0 0x7fcfd419bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:37.323 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.050+0000 7fcfd37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd41088c0 0x7fcfd419bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48958/0 (socket says 172.21.15.112:48958) 2024-03-16T09:06:37.323 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.050+0000 7fcfd37fe700 1 -- 172.21.15.112:0/2186470926 learned_addr learned my addr 172.21.15.112:0/2186470926 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:37.323 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.050+0000 7fcfd37fe700 1 -- 172.21.15.112:0/2186470926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd419c220 con 0x7fcfd41088c0 2024-03-16T09:06:37.323 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.050+0000 7fcfd37fe700 1 --2- 172.21.15.112:0/2186470926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd41088c0 0x7fcfd419bce0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4011870 tx=0x7fcfc4011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:37.323 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.050+0000 7fcfd0ff9700 1 -- 172.21.15.112:0/2186470926 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfc4003af0 con 0x7fcfd41088c0 2024-03-16T09:06:37.323 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.050+0000 7fcfd9ffd700 1 -- 172.21.15.112:0/2186470926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd419c4b0 con 0x7fcfd41088c0 2024-03-16T09:06:37.324 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.050+0000 7fcfd0ff9700 1 -- 172.21.15.112:0/2186470926 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcfc4003c90 con 0x7fcfd41088c0 2024-03-16T09:06:37.324 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.050+0000 7fcfd0ff9700 1 -- 172.21.15.112:0/2186470926 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfc4019960 con 0x7fcfd41088c0 2024-03-16T09:06:37.324 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.050+0000 7fcfd9ffd700 1 -- 172.21.15.112:0/2186470926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd419c910 con 0x7fcfd41088c0 2024-03-16T09:06:37.324 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.051+0000 7fcfd9ffd700 1 -- 172.21.15.112:0/2186470926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfd4063900 con 0x7fcfd41088c0 2024-03-16T09:06:37.324 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.051+0000 7fcfd0ff9700 1 -- 172.21.15.112:0/2186470926 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7fcfc4019c60 con 0x7fcfd41088c0 2024-03-16T09:06:37.324 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.052+0000 7fcfd0ff9700 1 --2- 172.21.15.112:0/2186470926 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcfbc03d330 0x7fcfbc03f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:37.324 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.052+0000 7fcfd0ff9700 1 -- 172.21.15.112:0/2186470926 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcfc404d490 con 0x7fcfd41088c0 2024-03-16T09:06:37.324 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.052+0000 7fcfd2ffd700 1 --2- 172.21.15.112:0/2186470926 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcfbc03d330 0x7fcfbc03f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:37.324 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.052+0000 7fcfd2ffd700 1 --2- 172.21.15.112:0/2186470926 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcfbc03d330 0x7fcfbc03f7f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8006fd0 tx=0x7fcfc8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:37.324 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.059+0000 7fcfd0ff9700 1 -- 172.21.15.112:0/2186470926 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcfc401ad10 con 0x7fcfd41088c0 2024-03-16T09:06:37.324 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.265+0000 7fcfd9ffd700 1 -- 172.21.15.112:0/2186470926 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fcfd41066b0 con 0x7fcfd41088c0 2024-03-16T09:06:37.324 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.265+0000 7fcfd0ff9700 1 -- 172.21.15.112:0/2186470926 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fcfc40172b0 con 0x7fcfd41088c0 2024-03-16T09:06:37.324 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.267+0000 7fcfd9ffd700 1 -- 172.21.15.112:0/2186470926 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcfbc03d330 msgr2=0x7fcfbc03f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:37.324 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.267+0000 7fcfd9ffd700 1 --2- 172.21.15.112:0/2186470926 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcfbc03d330 0x7fcfbc03f7f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8006fd0 tx=0x7fcfc8006e40 comp rx=0 tx=0).stop 2024-03-16T09:06:37.325 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.267+0000 7fcfd9ffd700 1 -- 172.21.15.112:0/2186470926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd41088c0 msgr2=0x7fcfd419bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:37.325 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.267+0000 7fcfd9ffd700 1 --2- 172.21.15.112:0/2186470926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd41088c0 0x7fcfd419bce0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4011870 tx=0x7fcfc4011d70 comp rx=0 tx=0).stop 2024-03-16T09:06:37.325 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.267+0000 7fcfd9ffd700 1 -- 172.21.15.112:0/2186470926 shutdown_connections 2024-03-16T09:06:37.325 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.267+0000 7fcfd9ffd700 1 --2- 172.21.15.112:0/2186470926 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcfbc03d330 0x7fcfbc03f7f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:37.325 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.267+0000 7fcfd9ffd700 1 --2- 172.21.15.112:0/2186470926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfd41088c0 0x7fcfd419bce0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:37.325 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.267+0000 7fcfd9ffd700 1 -- 172.21.15.112:0/2186470926 >> 172.21.15.112:0/2186470926 conn(0x7fcfd4102230 msgr2=0x7fcfd4102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:37.325 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.267+0000 7fcfd9ffd700 1 -- 172.21.15.112:0/2186470926 shutdown_connections 2024-03-16T09:06:37.325 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:36.267+0000 7fcfd9ffd700 1 -- 172.21.15.112:0/2186470926 wait complete. 2024-03-16T09:06:37.327 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-16T09:06:37.327 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-03-16T09:06:37.334 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-16T09:06:37.334 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-03-16T09:06:37.334 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-03-16T09:06:37.335 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-16T09:06:37.336 INFO:teuthology.orchestra.run.smithi112.stdout:Ceph Dashboard is now available at: 2024-03-16T09:06:37.336 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:06:37.336 INFO:teuthology.orchestra.run.smithi112.stdout: URL: https://smithi112.front.sepia.ceph.com:8443/ 2024-03-16T09:06:37.336 INFO:teuthology.orchestra.run.smithi112.stdout: User: admin 2024-03-16T09:06:37.336 INFO:teuthology.orchestra.run.smithi112.stdout: Password: nahtgx21u4 2024-03-16T09:06:37.336 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:06:37.337 INFO:teuthology.orchestra.run.smithi112.stdout:Saving cluster configuration to /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config directory 2024-03-16T09:06:37.337 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling autotune for osd_memory_target 2024-03-16T09:06:38.690 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.768+0000 7f525efac700 1 Processor -- start 2024-03-16T09:06:38.690 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.768+0000 7f525efac700 1 -- start start 2024-03-16T09:06:38.690 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.768+0000 7f525efac700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5258106610 0x7f5258106a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:38.691 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.768+0000 7f525efac700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f525807be50 con 0x7f5258106610 2024-03-16T09:06:38.691 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.768+0000 7f525cd48700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5258106610 0x7f5258106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:38.691 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.768+0000 7f525cd48700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5258106610 0x7f5258106a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48966/0 (socket says 172.21.15.112:48966) 2024-03-16T09:06:38.691 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.768+0000 7f525cd48700 1 -- 172.21.15.112:0/2655115745 learned_addr learned my addr 172.21.15.112:0/2655115745 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:38.691 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.768+0000 7f525cd48700 1 -- 172.21.15.112:0/2655115745 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5258106f70 con 0x7f5258106610 2024-03-16T09:06:38.691 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.769+0000 7f525cd48700 1 --2- 172.21.15.112:0/2655115745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5258106610 0x7f5258106a30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f524c0098a0 tx=0x7f524c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda9cdff1782cea1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:38.691 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.769+0000 7f52577fe700 1 -- 172.21.15.112:0/2655115745 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f524c004160 con 0x7f5258106610 2024-03-16T09:06:38.691 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.769+0000 7f52577fe700 1 -- 172.21.15.112:0/2655115745 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f524c004300 con 0x7f5258106610 2024-03-16T09:06:38.691 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.769+0000 7f52577fe700 1 -- 172.21.15.112:0/2655115745 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f524c011bb0 con 0x7f5258106610 2024-03-16T09:06:38.691 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.769+0000 7f525efac700 1 -- 172.21.15.112:0/2655115745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5258106610 msgr2=0x7f5258106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:38.691 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.769+0000 7f525efac700 1 --2- 172.21.15.112:0/2655115745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5258106610 0x7f5258106a30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f524c0098a0 tx=0x7f524c009bb0 comp rx=0 tx=0).stop 2024-03-16T09:06:38.691 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.769+0000 7f525efac700 1 -- 172.21.15.112:0/2655115745 shutdown_connections 2024-03-16T09:06:38.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.769+0000 7f525efac700 1 --2- 172.21.15.112:0/2655115745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5258106610 0x7f5258106a30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:38.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.769+0000 7f525efac700 1 -- 172.21.15.112:0/2655115745 >> 172.21.15.112:0/2655115745 conn(0x7f5258102230 msgr2=0x7f5258104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:38.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.769+0000 7f525efac700 1 -- 172.21.15.112:0/2655115745 shutdown_connections 2024-03-16T09:06:38.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.770+0000 7f525efac700 1 -- 172.21.15.112:0/2655115745 wait complete. 2024-03-16T09:06:38.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.770+0000 7f525efac700 1 Processor -- start 2024-03-16T09:06:38.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.770+0000 7f525efac700 1 -- start start 2024-03-16T09:06:38.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.770+0000 7f525efac700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5258106610 0x7f525819dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:38.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.770+0000 7f525efac700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52581080a0 con 0x7f5258106610 2024-03-16T09:06:38.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.770+0000 7f525cd48700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5258106610 0x7f525819dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:38.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.770+0000 7f525cd48700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5258106610 0x7f525819dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48968/0 (socket says 172.21.15.112:48968) 2024-03-16T09:06:38.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.770+0000 7f525cd48700 1 -- 172.21.15.112:0/3552307619 learned_addr learned my addr 172.21.15.112:0/3552307619 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:38.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.771+0000 7f525cd48700 1 -- 172.21.15.112:0/3552307619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525819e420 con 0x7f5258106610 2024-03-16T09:06:38.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.771+0000 7f525cd48700 1 --2- 172.21.15.112:0/3552307619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5258106610 0x7f525819dee0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f524c011870 tx=0x7f524c011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:38.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.771+0000 7f5255ffb700 1 -- 172.21.15.112:0/3552307619 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f524c003af0 con 0x7f5258106610 2024-03-16T09:06:38.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.771+0000 7f525efac700 1 -- 172.21.15.112:0/3552307619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f525819e6b0 con 0x7f5258106610 2024-03-16T09:06:38.693 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.771+0000 7f5255ffb700 1 -- 172.21.15.112:0/3552307619 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f524c003c90 con 0x7f5258106610 2024-03-16T09:06:38.693 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.771+0000 7f5255ffb700 1 -- 172.21.15.112:0/3552307619 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f524c019960 con 0x7f5258106610 2024-03-16T09:06:38.693 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.771+0000 7f525efac700 1 -- 172.21.15.112:0/3552307619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f525819eb10 con 0x7f5258106610 2024-03-16T09:06:38.693 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.772+0000 7f5255ffb700 1 -- 172.21.15.112:0/3552307619 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f524c019c10 con 0x7f5258106610 2024-03-16T09:06:38.693 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.772+0000 7f525efac700 1 -- 172.21.15.112:0/3552307619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5258063900 con 0x7f5258106610 2024-03-16T09:06:38.693 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.772+0000 7f5255ffb700 1 --2- 172.21.15.112:0/3552307619 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5248039020 0x7f524803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:38.693 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.772+0000 7f5255ffb700 1 -- 172.21.15.112:0/3552307619 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f524c04d4f0 con 0x7f5258106610 2024-03-16T09:06:38.693 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.772+0000 7f5257fff700 1 --2- 172.21.15.112:0/3552307619 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5248039020 0x7f524803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:38.693 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.773+0000 7f5257fff700 1 --2- 172.21.15.112:0/3552307619 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5248039020 0x7f524803b4e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5244006fd0 tx=0x7f5244006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:38.693 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.777+0000 7f5255ffb700 1 -- 172.21.15.112:0/3552307619 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f524c00b3e0 con 0x7f5258106610 2024-03-16T09:06:38.693 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.976+0000 7f525efac700 1 -- 172.21.15.112:0/3552307619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f5258062ee0 con 0x7f5258106610 2024-03-16T09:06:38.693 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.976+0000 7f5255ffb700 1 -- 172.21.15.112:0/3552307619 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f524c01ad10 con 0x7f5258106610 2024-03-16T09:06:38.694 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.978+0000 7f525efac700 1 -- 172.21.15.112:0/3552307619 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5248039020 msgr2=0x7f524803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:38.694 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.978+0000 7f525efac700 1 --2- 172.21.15.112:0/3552307619 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5248039020 0x7f524803b4e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5244006fd0 tx=0x7f5244006e40 comp rx=0 tx=0).stop 2024-03-16T09:06:38.694 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.978+0000 7f525efac700 1 -- 172.21.15.112:0/3552307619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5258106610 msgr2=0x7f525819dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:38.694 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.978+0000 7f525efac700 1 --2- 172.21.15.112:0/3552307619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5258106610 0x7f525819dee0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f524c011870 tx=0x7f524c011d70 comp rx=0 tx=0).stop 2024-03-16T09:06:38.694 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.978+0000 7f525efac700 1 -- 172.21.15.112:0/3552307619 shutdown_connections 2024-03-16T09:06:38.694 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.978+0000 7f525efac700 1 --2- 172.21.15.112:0/3552307619 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5248039020 0x7f524803b4e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:38.694 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.978+0000 7f525efac700 1 --2- 172.21.15.112:0/3552307619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5258106610 0x7f525819dee0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:38.694 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.978+0000 7f525efac700 1 -- 172.21.15.112:0/3552307619 >> 172.21.15.112:0/3552307619 conn(0x7f5258102230 msgr2=0x7f5258194ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:38.694 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.979+0000 7f525efac700 1 -- 172.21.15.112:0/3552307619 shutdown_connections 2024-03-16T09:06:38.694 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:37.979+0000 7f525efac700 1 -- 172.21.15.112:0/3552307619 wait complete. 2024-03-16T09:06:38.935 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:38 smithi112 ceph-mon[32307]: mgrmap e13: x(active, since 6s) 2024-03-16T09:06:40.048 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.130+0000 7f51159a2700 1 Processor -- start 2024-03-16T09:06:40.048 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.130+0000 7f51159a2700 1 -- start start 2024-03-16T09:06:40.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.130+0000 7f51159a2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5110105ee0 0x7f5110106300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:40.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.130+0000 7f51159a2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f51101068d0 con 0x7f5110105ee0 2024-03-16T09:06:40.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.130+0000 7f51149a0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5110105ee0 0x7f5110106300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:40.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.130+0000 7f51149a0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5110105ee0 0x7f5110106300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48972/0 (socket says 172.21.15.112:48972) 2024-03-16T09:06:40.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.130+0000 7f51149a0700 1 -- 172.21.15.112:0/1863061872 learned_addr learned my addr 172.21.15.112:0/1863061872 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:40.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.131+0000 7f51149a0700 1 -- 172.21.15.112:0/1863061872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5110108e70 con 0x7f5110105ee0 2024-03-16T09:06:40.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.131+0000 7f51149a0700 1 --2- 172.21.15.112:0/1863061872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5110105ee0 0x7f5110106300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51040098a0 tx=0x7f5104009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d6cff5fe84298 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:40.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.131+0000 7f510f7fe700 1 -- 172.21.15.112:0/1863061872 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5104004160 con 0x7f5110105ee0 2024-03-16T09:06:40.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.131+0000 7f510f7fe700 1 -- 172.21.15.112:0/1863061872 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5104004300 con 0x7f5110105ee0 2024-03-16T09:06:40.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.131+0000 7f510f7fe700 1 -- 172.21.15.112:0/1863061872 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5104011bb0 con 0x7f5110105ee0 2024-03-16T09:06:40.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.131+0000 7f51159a2700 1 -- 172.21.15.112:0/1863061872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5110105ee0 msgr2=0x7f5110106300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:40.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.131+0000 7f51159a2700 1 --2- 172.21.15.112:0/1863061872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5110105ee0 0x7f5110106300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51040098a0 tx=0x7f5104009bb0 comp rx=0 tx=0).stop 2024-03-16T09:06:40.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.131+0000 7f51159a2700 1 -- 172.21.15.112:0/1863061872 shutdown_connections 2024-03-16T09:06:40.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.131+0000 7f51159a2700 1 --2- 172.21.15.112:0/1863061872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5110105ee0 0x7f5110106300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:40.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.131+0000 7f51159a2700 1 -- 172.21.15.112:0/1863061872 >> 172.21.15.112:0/1863061872 conn(0x7f5110101830 msgr2=0x7f5110103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:40.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.131+0000 7f51159a2700 1 -- 172.21.15.112:0/1863061872 shutdown_connections 2024-03-16T09:06:40.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.131+0000 7f51159a2700 1 -- 172.21.15.112:0/1863061872 wait complete. 2024-03-16T09:06:40.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.132+0000 7f51159a2700 1 Processor -- start 2024-03-16T09:06:40.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.132+0000 7f51159a2700 1 -- start start 2024-03-16T09:06:40.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.132+0000 7f51159a2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5110105ee0 0x7f511019bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:40.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.132+0000 7f51159a2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f511010c220 con 0x7f5110105ee0 2024-03-16T09:06:40.050 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.132+0000 7f51149a0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5110105ee0 0x7f511019bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:40.051 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.132+0000 7f51149a0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5110105ee0 0x7f511019bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48974/0 (socket says 172.21.15.112:48974) 2024-03-16T09:06:40.051 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.132+0000 7f51149a0700 1 -- 172.21.15.112:0/3758489824 learned_addr learned my addr 172.21.15.112:0/3758489824 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:40.051 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.133+0000 7f51149a0700 1 -- 172.21.15.112:0/3758489824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511019c080 con 0x7f5110105ee0 2024-03-16T09:06:40.051 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.133+0000 7f51149a0700 1 --2- 172.21.15.112:0/3758489824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5110105ee0 0x7f511019bb40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5104011690 tx=0x7f5104011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:40.051 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.133+0000 7f510dffb700 1 -- 172.21.15.112:0/3758489824 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5104003b70 con 0x7f5110105ee0 2024-03-16T09:06:40.051 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.133+0000 7f510dffb700 1 -- 172.21.15.112:0/3758489824 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f510401a930 con 0x7f5110105ee0 2024-03-16T09:06:40.051 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.133+0000 7f51159a2700 1 -- 172.21.15.112:0/3758489824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f511019c310 con 0x7f5110105ee0 2024-03-16T09:06:40.051 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.133+0000 7f510dffb700 1 -- 172.21.15.112:0/3758489824 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5104019920 con 0x7f5110105ee0 2024-03-16T09:06:40.051 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.133+0000 7f51159a2700 1 -- 172.21.15.112:0/3758489824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f511019c770 con 0x7f5110105ee0 2024-03-16T09:06:40.051 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.134+0000 7f510dffb700 1 -- 172.21.15.112:0/3758489824 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f5104022020 con 0x7f5110105ee0 2024-03-16T09:06:40.051 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.134+0000 7f51159a2700 1 -- 172.21.15.112:0/3758489824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5110063900 con 0x7f5110105ee0 2024-03-16T09:06:40.052 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.134+0000 7f510dffb700 1 --2- 172.21.15.112:0/3758489824 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5100038fc0 0x7f510003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:40.052 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.134+0000 7f510dffb700 1 -- 172.21.15.112:0/3758489824 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f510404d340 con 0x7f5110105ee0 2024-03-16T09:06:40.052 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.134+0000 7f510ffff700 1 --2- 172.21.15.112:0/3758489824 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5100038fc0 0x7f510003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:40.052 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.135+0000 7f510ffff700 1 --2- 172.21.15.112:0/3758489824 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5100038fc0 0x7f510003b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f50fc006fd0 tx=0x7f50fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:40.052 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.139+0000 7f510dffb700 1 -- 172.21.15.112:0/3758489824 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f510402bba0 con 0x7f5110105ee0 2024-03-16T09:06:40.052 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.445+0000 7f51159a2700 1 -- 172.21.15.112:0/3758489824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f511019f140 con 0x7f5110105ee0 2024-03-16T09:06:40.052 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.447+0000 7f510dffb700 1 -- 172.21.15.112:0/3758489824 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f5104010df0 con 0x7f5110105ee0 2024-03-16T09:06:40.052 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.449+0000 7f51159a2700 1 -- 172.21.15.112:0/3758489824 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5100038fc0 msgr2=0x7f510003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:40.052 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.449+0000 7f51159a2700 1 --2- 172.21.15.112:0/3758489824 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5100038fc0 0x7f510003b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f50fc006fd0 tx=0x7f50fc006e40 comp rx=0 tx=0).stop 2024-03-16T09:06:40.052 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.449+0000 7f51159a2700 1 -- 172.21.15.112:0/3758489824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5110105ee0 msgr2=0x7f511019bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:40.052 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.449+0000 7f51159a2700 1 --2- 172.21.15.112:0/3758489824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5110105ee0 0x7f511019bb40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5104011690 tx=0x7f5104011ca0 comp rx=0 tx=0).stop 2024-03-16T09:06:40.053 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.450+0000 7f51159a2700 1 -- 172.21.15.112:0/3758489824 shutdown_connections 2024-03-16T09:06:40.053 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.450+0000 7f51159a2700 1 --2- 172.21.15.112:0/3758489824 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5100038fc0 0x7f510003b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:40.053 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.450+0000 7f51159a2700 1 --2- 172.21.15.112:0/3758489824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5110105ee0 0x7f511019bb40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:40.053 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.450+0000 7f51159a2700 1 -- 172.21.15.112:0/3758489824 >> 172.21.15.112:0/3758489824 conn(0x7f5110101830 msgr2=0x7f51101022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:40.053 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.450+0000 7f51159a2700 1 -- 172.21.15.112:0/3758489824 shutdown_connections 2024-03-16T09:06:40.053 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-03-16T09:06:39.450+0000 7f51159a2700 1 -- 172.21.15.112:0/3758489824 wait complete. 2024-03-16T09:06:40.053 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-16T09:06:40.053 INFO:teuthology.orchestra.run.smithi112.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-16T09:06:40.053 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:06:40.053 INFO:teuthology.orchestra.run.smithi112.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-16T09:06:40.053 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:06:40.053 INFO:teuthology.orchestra.run.smithi112.stdout:Or, if you are only running a single cluster on this host: 2024-03-16T09:06:40.053 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:06:40.054 INFO:teuthology.orchestra.run.smithi112.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-16T09:06:40.054 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:06:40.054 INFO:teuthology.orchestra.run.smithi112.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-16T09:06:40.054 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:06:40.054 INFO:teuthology.orchestra.run.smithi112.stdout: ceph telemetry on 2024-03-16T09:06:40.054 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:06:40.054 INFO:teuthology.orchestra.run.smithi112.stdout:For more information see: 2024-03-16T09:06:40.054 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:06:40.054 INFO:teuthology.orchestra.run.smithi112.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-16T09:06:40.054 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:06:40.054 INFO:teuthology.orchestra.run.smithi112.stdout:Bootstrap complete. 2024-03-16T09:06:40.085 INFO:tasks.cephadm:Fetching config... 2024-03-16T09:06:40.085 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T09:06:40.085 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-16T09:06:40.100 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-16T09:06:40.100 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T09:06:40.101 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-16T09:06:40.156 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-16T09:06:40.156 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T09:06:40.156 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-16T09:06:40.223 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-16T09:06:40.223 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T09:06:40.223 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-16T09:06:40.279 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-16T09:06:40.279 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDFcOmJO9hp3V8Ms/aru4Pan1LEQ5rNjV78XjDDJiH1DoIHboQXmLXbZ5TRFi1cq2QREwty1rPpPRCFnxUXhgU0F1jKlxOQMNgmk7oL1YQ2We/B/ErzxwaMJ7/3joa8Ovw6K/Xps/2sMXuUk+uaxPl+ucZ8Z5P6dHwAkflC2+vkwcg71nYS2eoRqrpZv1SZRqlacY8ky6FVI9owXeE8AiEM9x3S1gEqCGRDHL/EBHXGx67Zs2Aq5cPnoVtla8Njw5VDVbqUgVS+Z8Nis0qBokVC8tAx70EPjCAnWsfh5cm1u7ug9g7E48oQpFnEaSaed6atuF+QZq11pYboVhGt7vZLILkRleRvC9k9bIOmh/rCF49csDhvsJ5jgwuxWfPc2MvAAXhbRs2V+D2xRr0ugfmbIlmWqzQ0yqhDbrn+szcrAn6e13030Oev95yz8aq6hQZlZW8Cgn6nsn+HVxOOvtgoNPdu/iIPtf17h4qPpnh/ZJsD1np8ckzDwOpKMClBEM= ceph-273be2a4-e374-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-16T09:06:40.358 INFO:teuthology.orchestra.run.smithi112.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDFcOmJO9hp3V8Ms/aru4Pan1LEQ5rNjV78XjDDJiH1DoIHboQXmLXbZ5TRFi1cq2QREwty1rPpPRCFnxUXhgU0F1jKlxOQMNgmk7oL1YQ2We/B/ErzxwaMJ7/3joa8Ovw6K/Xps/2sMXuUk+uaxPl+ucZ8Z5P6dHwAkflC2+vkwcg71nYS2eoRqrpZv1SZRqlacY8ky6FVI9owXeE8AiEM9x3S1gEqCGRDHL/EBHXGx67Zs2Aq5cPnoVtla8Njw5VDVbqUgVS+Z8Nis0qBokVC8tAx70EPjCAnWsfh5cm1u7ug9g7E48oQpFnEaSaed6atuF+QZq11pYboVhGt7vZLILkRleRvC9k9bIOmh/rCF49csDhvsJ5jgwuxWfPc2MvAAXhbRs2V+D2xRr0ugfmbIlmWqzQ0yqhDbrn+szcrAn6e13030Oev95yz8aq6hQZlZW8Cgn6nsn+HVxOOvtgoNPdu/iIPtf17h4qPpnh/ZJsD1np8ckzDwOpKMClBEM= ceph-273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:06:40.372 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDFcOmJO9hp3V8Ms/aru4Pan1LEQ5rNjV78XjDDJiH1DoIHboQXmLXbZ5TRFi1cq2QREwty1rPpPRCFnxUXhgU0F1jKlxOQMNgmk7oL1YQ2We/B/ErzxwaMJ7/3joa8Ovw6K/Xps/2sMXuUk+uaxPl+ucZ8Z5P6dHwAkflC2+vkwcg71nYS2eoRqrpZv1SZRqlacY8ky6FVI9owXeE8AiEM9x3S1gEqCGRDHL/EBHXGx67Zs2Aq5cPnoVtla8Njw5VDVbqUgVS+Z8Nis0qBokVC8tAx70EPjCAnWsfh5cm1u7ug9g7E48oQpFnEaSaed6atuF+QZq11pYboVhGt7vZLILkRleRvC9k9bIOmh/rCF49csDhvsJ5jgwuxWfPc2MvAAXhbRs2V+D2xRr0ugfmbIlmWqzQ0yqhDbrn+szcrAn6e13030Oev95yz8aq6hQZlZW8Cgn6nsn+HVxOOvtgoNPdu/iIPtf17h4qPpnh/ZJsD1np8ckzDwOpKMClBEM= ceph-273be2a4-e374-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-16T09:06:40.418 INFO:teuthology.orchestra.run.smithi159.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDFcOmJO9hp3V8Ms/aru4Pan1LEQ5rNjV78XjDDJiH1DoIHboQXmLXbZ5TRFi1cq2QREwty1rPpPRCFnxUXhgU0F1jKlxOQMNgmk7oL1YQ2We/B/ErzxwaMJ7/3joa8Ovw6K/Xps/2sMXuUk+uaxPl+ucZ8Z5P6dHwAkflC2+vkwcg71nYS2eoRqrpZv1SZRqlacY8ky6FVI9owXeE8AiEM9x3S1gEqCGRDHL/EBHXGx67Zs2Aq5cPnoVtla8Njw5VDVbqUgVS+Z8Nis0qBokVC8tAx70EPjCAnWsfh5cm1u7ug9g7E48oQpFnEaSaed6atuF+QZq11pYboVhGt7vZLILkRleRvC9k9bIOmh/rCF49csDhvsJ5jgwuxWfPc2MvAAXhbRs2V+D2xRr0ugfmbIlmWqzQ0yqhDbrn+szcrAn6e13030Oev95yz8aq6hQZlZW8Cgn6nsn+HVxOOvtgoNPdu/iIPtf17h4qPpnh/ZJsD1np8ckzDwOpKMClBEM= ceph-273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:06:40.433 DEBUG:teuthology.orchestra.run.smithi194:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDFcOmJO9hp3V8Ms/aru4Pan1LEQ5rNjV78XjDDJiH1DoIHboQXmLXbZ5TRFi1cq2QREwty1rPpPRCFnxUXhgU0F1jKlxOQMNgmk7oL1YQ2We/B/ErzxwaMJ7/3joa8Ovw6K/Xps/2sMXuUk+uaxPl+ucZ8Z5P6dHwAkflC2+vkwcg71nYS2eoRqrpZv1SZRqlacY8ky6FVI9owXeE8AiEM9x3S1gEqCGRDHL/EBHXGx67Zs2Aq5cPnoVtla8Njw5VDVbqUgVS+Z8Nis0qBokVC8tAx70EPjCAnWsfh5cm1u7ug9g7E48oQpFnEaSaed6atuF+QZq11pYboVhGt7vZLILkRleRvC9k9bIOmh/rCF49csDhvsJ5jgwuxWfPc2MvAAXhbRs2V+D2xRr0ugfmbIlmWqzQ0yqhDbrn+szcrAn6e13030Oev95yz8aq6hQZlZW8Cgn6nsn+HVxOOvtgoNPdu/iIPtf17h4qPpnh/ZJsD1np8ckzDwOpKMClBEM= ceph-273be2a4-e374-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-16T09:06:40.478 INFO:teuthology.orchestra.run.smithi194.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDFcOmJO9hp3V8Ms/aru4Pan1LEQ5rNjV78XjDDJiH1DoIHboQXmLXbZ5TRFi1cq2QREwty1rPpPRCFnxUXhgU0F1jKlxOQMNgmk7oL1YQ2We/B/ErzxwaMJ7/3joa8Ovw6K/Xps/2sMXuUk+uaxPl+ucZ8Z5P6dHwAkflC2+vkwcg71nYS2eoRqrpZv1SZRqlacY8ky6FVI9owXeE8AiEM9x3S1gEqCGRDHL/EBHXGx67Zs2Aq5cPnoVtla8Njw5VDVbqUgVS+Z8Nis0qBokVC8tAx70EPjCAnWsfh5cm1u7ug9g7E48oQpFnEaSaed6atuF+QZq11pYboVhGt7vZLILkRleRvC9k9bIOmh/rCF49csDhvsJ5jgwuxWfPc2MvAAXhbRs2V+D2xRr0ugfmbIlmWqzQ0yqhDbrn+szcrAn6e13030Oev95yz8aq6hQZlZW8Cgn6nsn+HVxOOvtgoNPdu/iIPtf17h4qPpnh/ZJsD1np8ckzDwOpKMClBEM= ceph-273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:06:40.493 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-16T09:06:40.734 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:06:40.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:40 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3758489824' entity='client.admin' 2024-03-16T09:06:43.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.211+0000 7f5d807fb700 1 -- 172.21.15.112:0/2699274393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d78101d00 msgr2=0x7f5d78102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:43.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.211+0000 7f5d807fb700 1 --2- 172.21.15.112:0/2699274393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d78101d00 0x7f5d78102120 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009910 tx=0x7f5d74009c20 comp rx=0 tx=0).stop 2024-03-16T09:06:43.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.211+0000 7f5d807fb700 1 -- 172.21.15.112:0/2699274393 shutdown_connections 2024-03-16T09:06:43.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.211+0000 7f5d807fb700 1 --2- 172.21.15.112:0/2699274393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d78101d00 0x7f5d78102120 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:43.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.211+0000 7f5d807fb700 1 -- 172.21.15.112:0/2699274393 >> 172.21.15.112:0/2699274393 conn(0x7f5d780fd2c0 msgr2=0x7f5d780ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:43.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.211+0000 7f5d807fb700 1 -- 172.21.15.112:0/2699274393 shutdown_connections 2024-03-16T09:06:43.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.211+0000 7f5d807fb700 1 -- 172.21.15.112:0/2699274393 wait complete. 2024-03-16T09:06:43.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.212+0000 7f5d807fb700 1 Processor -- start 2024-03-16T09:06:43.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.212+0000 7f5d807fb700 1 -- start start 2024-03-16T09:06:43.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.212+0000 7f5d807fb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d78101d00 0x7f5d78197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:43.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.212+0000 7f5d807fb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d78107ed0 con 0x7f5d78101d00 2024-03-16T09:06:43.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.212+0000 7f5d7e597700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d78101d00 0x7f5d78197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:43.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.212+0000 7f5d7e597700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d78101d00 0x7f5d78197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55842/0 (socket says 172.21.15.112:55842) 2024-03-16T09:06:43.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.212+0000 7f5d7e597700 1 -- 172.21.15.112:0/1942092080 learned_addr learned my addr 172.21.15.112:0/1942092080 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:43.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.213+0000 7f5d7e597700 1 -- 172.21.15.112:0/1942092080 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d78197970 con 0x7f5d78101d00 2024-03-16T09:06:43.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.213+0000 7f5d7e597700 1 --2- 172.21.15.112:0/1942092080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d78101d00 0x7f5d78197430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5d74005840 tx=0x7f5d740050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:43.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.213+0000 7f5d6f7fe700 1 -- 172.21.15.112:0/1942092080 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d74027070 con 0x7f5d78101d00 2024-03-16T09:06:43.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.213+0000 7f5d6f7fe700 1 -- 172.21.15.112:0/1942092080 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5d74022470 con 0x7f5d78101d00 2024-03-16T09:06:43.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.213+0000 7f5d807fb700 1 -- 172.21.15.112:0/1942092080 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d78197c00 con 0x7f5d78101d00 2024-03-16T09:06:43.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.213+0000 7f5d6f7fe700 1 -- 172.21.15.112:0/1942092080 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d7400f5f0 con 0x7f5d78101d00 2024-03-16T09:06:43.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.213+0000 7f5d807fb700 1 -- 172.21.15.112:0/1942092080 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d7819a8f0 con 0x7f5d78101d00 2024-03-16T09:06:43.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.214+0000 7f5d6f7fe700 1 -- 172.21.15.112:0/1942092080 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f5d7402a070 con 0x7f5d78101d00 2024-03-16T09:06:43.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.214+0000 7f5d807fb700 1 -- 172.21.15.112:0/1942092080 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d78108650 con 0x7f5d78101d00 2024-03-16T09:06:43.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.214+0000 7f5d6f7fe700 1 --2- 172.21.15.112:0/1942092080 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5d64038fc0 0x7f5d6403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:43.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.214+0000 7f5d6f7fe700 1 -- 172.21.15.112:0/1942092080 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5d7401e030 con 0x7f5d78101d00 2024-03-16T09:06:43.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.214+0000 7f5d7dd96700 1 --2- 172.21.15.112:0/1942092080 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5d64038fc0 0x7f5d6403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:43.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.214+0000 7f5d7dd96700 1 --2- 172.21.15.112:0/1942092080 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5d64038fc0 0x7f5d6403b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5d68006fd0 tx=0x7f5d68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:43.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.219+0000 7f5d6f7fe700 1 -- 172.21.15.112:0/1942092080 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5d74013070 con 0x7f5d78101d00 2024-03-16T09:06:43.402 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:06:43.402 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:06:43.403 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:06:43.403 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:06:43.403 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:43.403 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:06:43.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.419+0000 7f5d807fb700 1 -- 172.21.15.112:0/1942092080 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f5d7802dc80 con 0x7f5d78101d00 2024-03-16T09:06:43.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.421+0000 7f5d6f7fe700 1 -- 172.21.15.112:0/1942092080 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f5d7400f7e0 con 0x7f5d78101d00 2024-03-16T09:06:43.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.423+0000 7f5d807fb700 1 -- 172.21.15.112:0/1942092080 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5d64038fc0 msgr2=0x7f5d6403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:43.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.423+0000 7f5d807fb700 1 --2- 172.21.15.112:0/1942092080 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5d64038fc0 0x7f5d6403b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5d68006fd0 tx=0x7f5d68006e40 comp rx=0 tx=0).stop 2024-03-16T09:06:43.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.423+0000 7f5d807fb700 1 -- 172.21.15.112:0/1942092080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d78101d00 msgr2=0x7f5d78197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:43.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.423+0000 7f5d807fb700 1 --2- 172.21.15.112:0/1942092080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d78101d00 0x7f5d78197430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5d74005840 tx=0x7f5d740050b0 comp rx=0 tx=0).stop 2024-03-16T09:06:43.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.423+0000 7f5d807fb700 1 -- 172.21.15.112:0/1942092080 shutdown_connections 2024-03-16T09:06:43.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.423+0000 7f5d807fb700 1 --2- 172.21.15.112:0/1942092080 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5d64038fc0 0x7f5d6403b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:43.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.423+0000 7f5d807fb700 1 --2- 172.21.15.112:0/1942092080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d78101d00 0x7f5d78197430 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:43.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.423+0000 7f5d807fb700 1 -- 172.21.15.112:0/1942092080 >> 172.21.15.112:0/1942092080 conn(0x7f5d780fd2c0 msgr2=0x7f5d780fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:43.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.424+0000 7f5d807fb700 1 -- 172.21.15.112:0/1942092080 shutdown_connections 2024-03-16T09:06:43.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:43.424+0000 7f5d807fb700 1 -- 172.21.15.112:0/1942092080 wait complete. 2024-03-16T09:06:43.999 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-16T09:06:43.999 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-16T09:06:44.693 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:44 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1942092080' entity='client.admin' 2024-03-16T09:06:44.694 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:44 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:44.694 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:44 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:06:44.775 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:06:46.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.281+0000 7f53350d4700 1 -- 172.21.15.112:0/3344090080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5330100020 msgr2=0x7f5330100440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:46.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.281+0000 7f53350d4700 1 --2- 172.21.15.112:0/3344090080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5330100020 0x7f5330100440 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5320009910 tx=0x7f5320009c20 comp rx=0 tx=0).stop 2024-03-16T09:06:46.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.281+0000 7f53350d4700 1 -- 172.21.15.112:0/3344090080 shutdown_connections 2024-03-16T09:06:46.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.281+0000 7f53350d4700 1 --2- 172.21.15.112:0/3344090080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5330100020 0x7f5330100440 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:46.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.281+0000 7f53350d4700 1 -- 172.21.15.112:0/3344090080 >> 172.21.15.112:0/3344090080 conn(0x7f53300fb980 msgr2=0x7f53300fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:46.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.281+0000 7f53350d4700 1 -- 172.21.15.112:0/3344090080 shutdown_connections 2024-03-16T09:06:46.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.281+0000 7f53350d4700 1 -- 172.21.15.112:0/3344090080 wait complete. 2024-03-16T09:06:46.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.282+0000 7f53350d4700 1 Processor -- start 2024-03-16T09:06:46.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.282+0000 7f53350d4700 1 -- start start 2024-03-16T09:06:46.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.282+0000 7f53350d4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5330100020 0x7f5330197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:46.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.282+0000 7f53350d4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5330107ec0 con 0x7f5330100020 2024-03-16T09:06:46.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.282+0000 7f532ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5330100020 0x7f5330197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:46.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.282+0000 7f532ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5330100020 0x7f5330197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55860/0 (socket says 172.21.15.112:55860) 2024-03-16T09:06:46.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.282+0000 7f532ed9d700 1 -- 172.21.15.112:0/2176548998 learned_addr learned my addr 172.21.15.112:0/2176548998 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:46.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.282+0000 7f532ed9d700 1 -- 172.21.15.112:0/2176548998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53301979a0 con 0x7f5330100020 2024-03-16T09:06:46.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.283+0000 7f532ed9d700 1 --2- 172.21.15.112:0/2176548998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5330100020 0x7f5330197460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5320005840 tx=0x7f53200050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:46.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.283+0000 7f5317fff700 1 -- 172.21.15.112:0/2176548998 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5320027070 con 0x7f5330100020 2024-03-16T09:06:46.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.283+0000 7f5317fff700 1 -- 172.21.15.112:0/2176548998 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5320022470 con 0x7f5330100020 2024-03-16T09:06:46.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.283+0000 7f5317fff700 1 -- 172.21.15.112:0/2176548998 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f532000f5f0 con 0x7f5330100020 2024-03-16T09:06:46.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.283+0000 7f53350d4700 1 -- 172.21.15.112:0/2176548998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5330197c30 con 0x7f5330100020 2024-03-16T09:06:46.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.283+0000 7f53350d4700 1 -- 172.21.15.112:0/2176548998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f533019a920 con 0x7f5330100020 2024-03-16T09:06:46.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.284+0000 7f5317fff700 1 -- 172.21.15.112:0/2176548998 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f532002a070 con 0x7f5330100020 2024-03-16T09:06:46.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.284+0000 7f5317fff700 1 --2- 172.21.15.112:0/2176548998 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5318039020 0x7f531803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:46.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.284+0000 7f5317fff700 1 -- 172.21.15.112:0/2176548998 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f532001e030 con 0x7f5330100020 2024-03-16T09:06:46.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.284+0000 7f53350d4700 1 -- 172.21.15.112:0/2176548998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5330108640 con 0x7f5330100020 2024-03-16T09:06:46.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.284+0000 7f532e59c700 1 --2- 172.21.15.112:0/2176548998 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5318039020 0x7f531803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:46.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.284+0000 7f532e59c700 1 --2- 172.21.15.112:0/2176548998 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5318039020 0x7f531803b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5324006fd0 tx=0x7f5324006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:46.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.289+0000 7f5317fff700 1 -- 172.21.15.112:0/2176548998 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5320013070 con 0x7f5330100020 2024-03-16T09:06:46.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.494+0000 7f53350d4700 1 -- 172.21.15.112:0/2176548998 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f533002dc80 con 0x7f5318039020 2024-03-16T09:06:46.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.497+0000 7f5317fff700 1 -- 172.21.15.112:0/2176548998 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f533002dc80 con 0x7f5318039020 2024-03-16T09:06:46.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.499+0000 7f53350d4700 1 -- 172.21.15.112:0/2176548998 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5318039020 msgr2=0x7f531803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:46.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.499+0000 7f53350d4700 1 --2- 172.21.15.112:0/2176548998 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5318039020 0x7f531803b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5324006fd0 tx=0x7f5324006e40 comp rx=0 tx=0).stop 2024-03-16T09:06:46.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.499+0000 7f53350d4700 1 -- 172.21.15.112:0/2176548998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5330100020 msgr2=0x7f5330197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:46.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.499+0000 7f53350d4700 1 --2- 172.21.15.112:0/2176548998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5330100020 0x7f5330197460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5320005840 tx=0x7f53200050b0 comp rx=0 tx=0).stop 2024-03-16T09:06:46.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.500+0000 7f53350d4700 1 -- 172.21.15.112:0/2176548998 shutdown_connections 2024-03-16T09:06:46.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.500+0000 7f53350d4700 1 --2- 172.21.15.112:0/2176548998 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5318039020 0x7f531803b4e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:46.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.500+0000 7f53350d4700 1 --2- 172.21.15.112:0/2176548998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5330100020 0x7f5330197460 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:46.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.500+0000 7f53350d4700 1 -- 172.21.15.112:0/2176548998 >> 172.21.15.112:0/2176548998 conn(0x7f53300fb980 msgr2=0x7f53300fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:46.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.500+0000 7f53350d4700 1 -- 172.21.15.112:0/2176548998 shutdown_connections 2024-03-16T09:06:46.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:46.500+0000 7f53350d4700 1 -- 172.21.15.112:0/2176548998 wait complete. 2024-03-16T09:06:47.255 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi159 2024-03-16T09:06:47.256 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-16T09:06:47.256 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.conf 2024-03-16T09:06:47.274 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-16T09:06:47.274 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:06:47.332 INFO:tasks.cephadm:Adding host smithi159 to orchestrator... 2024-03-16T09:06:47.332 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch host add smithi159 2024-03-16T09:06:47.569 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:06:47.801 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:47 smithi112 ceph-mon[32307]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:47.801 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:06:47.801 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:47.801 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:06:47.801 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:06:47.801 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:47 smithi112 ceph-mon[32307]: Updating smithi112:/etc/ceph/ceph.conf 2024-03-16T09:06:47.801 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:47 smithi112 ceph-mon[32307]: Updating smithi112:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:06:47.801 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:47 smithi112 ceph-mon[32307]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:06:47.801 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:06:47.801 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:06:47.801 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:06:48.755 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:48 smithi112 ceph-mon[32307]: Updating smithi112:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:06:48.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.962+0000 7f82c4f52700 1 -- 172.21.15.112:0/2352416980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c0100e10 msgr2=0x7f82c01011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:48.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.962+0000 7f82c4f52700 1 --2- 172.21.15.112:0/2352416980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c0100e10 0x7f82c01011f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009910 tx=0x7f82b0009c20 comp rx=0 tx=0).stop 2024-03-16T09:06:48.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.962+0000 7f82c4f52700 1 -- 172.21.15.112:0/2352416980 shutdown_connections 2024-03-16T09:06:48.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.962+0000 7f82c4f52700 1 --2- 172.21.15.112:0/2352416980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c0100e10 0x7f82c01011f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:48.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.962+0000 7f82c4f52700 1 -- 172.21.15.112:0/2352416980 >> 172.21.15.112:0/2352416980 conn(0x7f82c00fc6e0 msgr2=0x7f82c00feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:48.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.962+0000 7f82c4f52700 1 -- 172.21.15.112:0/2352416980 shutdown_connections 2024-03-16T09:06:48.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.962+0000 7f82c4f52700 1 -- 172.21.15.112:0/2352416980 wait complete. 2024-03-16T09:06:48.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.963+0000 7f82c4f52700 1 Processor -- start 2024-03-16T09:06:48.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.963+0000 7f82c4f52700 1 -- start start 2024-03-16T09:06:48.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.963+0000 7f82c4f52700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c0100e10 0x7f82c0196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:48.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.963+0000 7f82c4f52700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f82c010b320 con 0x7f82c0100e10 2024-03-16T09:06:48.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.963+0000 7f82be59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c0100e10 0x7f82c0196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:48.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.963+0000 7f82be59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c0100e10 0x7f82c0196740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55882/0 (socket says 172.21.15.112:55882) 2024-03-16T09:06:48.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.963+0000 7f82be59c700 1 -- 172.21.15.112:0/3337004513 learned_addr learned my addr 172.21.15.112:0/3337004513 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:48.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.963+0000 7f82be59c700 1 -- 172.21.15.112:0/3337004513 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c0196c80 con 0x7f82c0100e10 2024-03-16T09:06:48.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.963+0000 7f82be59c700 1 --2- 172.21.15.112:0/3337004513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c0100e10 0x7f82c0196740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f82b00052a0 tx=0x7f82b0004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:48.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.963+0000 7f82af7fe700 1 -- 172.21.15.112:0/3337004513 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82b0026070 con 0x7f82c0100e10 2024-03-16T09:06:48.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.963+0000 7f82c4f52700 1 -- 172.21.15.112:0/3337004513 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c0196f70 con 0x7f82c0100e10 2024-03-16T09:06:48.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.963+0000 7f82af7fe700 1 -- 172.21.15.112:0/3337004513 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f82b000bbf0 con 0x7f82c0100e10 2024-03-16T09:06:48.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.963+0000 7f82af7fe700 1 -- 172.21.15.112:0/3337004513 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82b0003a30 con 0x7f82c0100e10 2024-03-16T09:06:48.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.964+0000 7f82c4f52700 1 -- 172.21.15.112:0/3337004513 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c019b010 con 0x7f82c0100e10 2024-03-16T09:06:48.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.964+0000 7f82af7fe700 1 -- 172.21.15.112:0/3337004513 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f82b0029070 con 0x7f82c0100e10 2024-03-16T09:06:48.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.964+0000 7f82c4f52700 1 -- 172.21.15.112:0/3337004513 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c01022f0 con 0x7f82c0100e10 2024-03-16T09:06:48.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.964+0000 7f82af7fe700 1 --2- 172.21.15.112:0/3337004513 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f82a8038fc0 0x7f82a803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:48.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.965+0000 7f82af7fe700 1 -- 172.21.15.112:0/3337004513 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f82b001e070 con 0x7f82c0100e10 2024-03-16T09:06:48.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.965+0000 7f82bdd9b700 1 --2- 172.21.15.112:0/3337004513 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f82a8038fc0 0x7f82a803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:48.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.965+0000 7f82bdd9b700 1 --2- 172.21.15.112:0/3337004513 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f82a8038fc0 0x7f82a803b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f82b4006fd0 tx=0x7f82b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:48.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:48.969+0000 7f82af7fe700 1 -- 172.21.15.112:0/3337004513 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f82b0013070 con 0x7f82c0100e10 2024-03-16T09:06:49.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:49.169+0000 7f82c4f52700 1 -- 172.21.15.112:0/3337004513 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}) -- 0x7f82c0197aa0 con 0x7f82a8038fc0 2024-03-16T09:06:50.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:50 smithi112 ceph-mon[32307]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:06:50.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:50 smithi112 ceph-mon[32307]: Deploying cephadm binary to smithi159 2024-03-16T09:06:52.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:51 smithi112 ceph-mon[32307]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:06:53.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:53.305+0000 7f82af7fe700 1 -- 172.21.15.112:0/3337004513 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f82c0197aa0 con 0x7f82a8038fc0 2024-03-16T09:06:53.308 INFO:teuthology.orchestra.run.smithi112.stdout:Added host 'smithi159' with addr '172.21.15.159' 2024-03-16T09:06:53.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:53.308+0000 7f82c4f52700 1 -- 172.21.15.112:0/3337004513 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f82a8038fc0 msgr2=0x7f82a803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:53.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:53.308+0000 7f82c4f52700 1 --2- 172.21.15.112:0/3337004513 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f82a8038fc0 0x7f82a803b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f82b4006fd0 tx=0x7f82b4006e40 comp rx=0 tx=0).stop 2024-03-16T09:06:53.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:53.308+0000 7f82c4f52700 1 -- 172.21.15.112:0/3337004513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c0100e10 msgr2=0x7f82c0196740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:53.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:53.308+0000 7f82c4f52700 1 --2- 172.21.15.112:0/3337004513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c0100e10 0x7f82c0196740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f82b00052a0 tx=0x7f82b0004cc0 comp rx=0 tx=0).stop 2024-03-16T09:06:53.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:53.308+0000 7f82c4f52700 1 -- 172.21.15.112:0/3337004513 shutdown_connections 2024-03-16T09:06:53.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:53.308+0000 7f82c4f52700 1 --2- 172.21.15.112:0/3337004513 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f82a8038fc0 0x7f82a803b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:53.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:53.308+0000 7f82c4f52700 1 --2- 172.21.15.112:0/3337004513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c0100e10 0x7f82c0196740 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:53.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:53.308+0000 7f82c4f52700 1 -- 172.21.15.112:0/3337004513 >> 172.21.15.112:0/3337004513 conn(0x7f82c00fc6e0 msgr2=0x7f82c0103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:53.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:53.308+0000 7f82c4f52700 1 -- 172.21.15.112:0/3337004513 shutdown_connections 2024-03-16T09:06:53.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:53.308+0000 7f82c4f52700 1 -- 172.21.15.112:0/3337004513 wait complete. 2024-03-16T09:06:53.906 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:53 smithi112 ceph-mon[32307]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:06:53.906 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:53 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:06:53.906 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:53 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:06:53.907 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:53 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:06:53.934 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-16T09:06:54.170 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:06:54.971 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:54 smithi112 ceph-mon[32307]: Added host smithi159 2024-03-16T09:06:54.971 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:54 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:06:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.773+0000 7fefdc78f700 1 -- 172.21.15.112:0/586306527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefd4102f30 msgr2=0x7fefd4103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.773+0000 7fefdc78f700 1 --2- 172.21.15.112:0/586306527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefd4102f30 0x7fefd4103310 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fefd0009910 tx=0x7fefd0009c20 comp rx=0 tx=0).stop 2024-03-16T09:06:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.773+0000 7fefdc78f700 1 -- 172.21.15.112:0/586306527 shutdown_connections 2024-03-16T09:06:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.773+0000 7fefdc78f700 1 --2- 172.21.15.112:0/586306527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefd4102f30 0x7fefd4103310 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.773+0000 7fefdc78f700 1 -- 172.21.15.112:0/586306527 >> 172.21.15.112:0/586306527 conn(0x7fefd4077970 msgr2=0x7fefd40fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.773+0000 7fefdc78f700 1 -- 172.21.15.112:0/586306527 shutdown_connections 2024-03-16T09:06:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.773+0000 7fefdc78f700 1 -- 172.21.15.112:0/586306527 wait complete. 2024-03-16T09:06:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.774+0000 7fefdc78f700 1 Processor -- start 2024-03-16T09:06:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.774+0000 7fefdc78f700 1 -- start start 2024-03-16T09:06:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.774+0000 7fefdc78f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefd4102f30 0x7fefd419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.774+0000 7fefdc78f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fefd410c450 con 0x7fefd4102f30 2024-03-16T09:06:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.774+0000 7fefda52b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefd4102f30 0x7fefd419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.774+0000 7fefda52b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefd4102f30 0x7fefd419d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50130/0 (socket says 172.21.15.112:50130) 2024-03-16T09:06:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.774+0000 7fefda52b700 1 -- 172.21.15.112:0/3185320904 learned_addr learned my addr 172.21.15.112:0/3185320904 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.774+0000 7fefda52b700 1 -- 172.21.15.112:0/3185320904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd419db50 con 0x7fefd4102f30 2024-03-16T09:06:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.774+0000 7fefda52b700 1 --2- 172.21.15.112:0/3185320904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefd4102f30 0x7fefd419d610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fefd0004c10 tx=0x7fefd0004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.775+0000 7fefcb7fe700 1 -- 172.21.15.112:0/3185320904 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefd0026070 con 0x7fefd4102f30 2024-03-16T09:06:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.775+0000 7fefcb7fe700 1 -- 172.21.15.112:0/3185320904 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fefd0004370 con 0x7fefd4102f30 2024-03-16T09:06:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.775+0000 7fefdc78f700 1 -- 172.21.15.112:0/3185320904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd419dde0 con 0x7fefd4102f30 2024-03-16T09:06:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.775+0000 7fefcb7fe700 1 -- 172.21.15.112:0/3185320904 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefd0003bf0 con 0x7fefd4102f30 2024-03-16T09:06:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.775+0000 7fefdc78f700 1 -- 172.21.15.112:0/3185320904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd41976e0 con 0x7fefd4102f30 2024-03-16T09:06:55.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.775+0000 7fefcb7fe700 1 -- 172.21.15.112:0/3185320904 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7fefd0029070 con 0x7fefd4102f30 2024-03-16T09:06:55.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.776+0000 7fefdc78f700 1 -- 172.21.15.112:0/3185320904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd4069ec0 con 0x7fefd4102f30 2024-03-16T09:06:55.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.776+0000 7fefcb7fe700 1 --2- 172.21.15.112:0/3185320904 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fefc0039020 0x7fefc003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:55.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.776+0000 7fefcb7fe700 1 -- 172.21.15.112:0/3185320904 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fefd001e070 con 0x7fefd4102f30 2024-03-16T09:06:55.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.776+0000 7fefd9d2a700 1 --2- 172.21.15.112:0/3185320904 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fefc0039020 0x7fefc003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:55.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.776+0000 7fefd9d2a700 1 --2- 172.21.15.112:0/3185320904 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fefc0039020 0x7fefc003b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fefc4006fd0 tx=0x7fefc4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:55.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.781+0000 7fefcb7fe700 1 -- 172.21.15.112:0/3185320904 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fefd0050030 con 0x7fefd4102f30 2024-03-16T09:06:55.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.984+0000 7fefdc78f700 1 -- 172.21.15.112:0/3185320904 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fefd4198c10 con 0x7fefc0039020 2024-03-16T09:06:55.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.985+0000 7fefcb7fe700 1 -- 172.21.15.112:0/3185320904 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fefd4198c10 con 0x7fefc0039020 2024-03-16T09:06:55.987 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:06:55.987 INFO:teuthology.orchestra.run.smithi112.stdout:[{"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}, {"addr": "172.21.15.159", "hostname": "smithi159", "labels": [], "status": ""}] 2024-03-16T09:06:55.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.987+0000 7fefdc78f700 1 -- 172.21.15.112:0/3185320904 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fefc0039020 msgr2=0x7fefc003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:55.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.987+0000 7fefdc78f700 1 --2- 172.21.15.112:0/3185320904 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fefc0039020 0x7fefc003b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fefc4006fd0 tx=0x7fefc4006e40 comp rx=0 tx=0).stop 2024-03-16T09:06:55.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.987+0000 7fefdc78f700 1 -- 172.21.15.112:0/3185320904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefd4102f30 msgr2=0x7fefd419d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:55.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.987+0000 7fefdc78f700 1 --2- 172.21.15.112:0/3185320904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefd4102f30 0x7fefd419d610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fefd0004c10 tx=0x7fefd0004cf0 comp rx=0 tx=0).stop 2024-03-16T09:06:55.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.987+0000 7fefdc78f700 1 -- 172.21.15.112:0/3185320904 shutdown_connections 2024-03-16T09:06:55.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.987+0000 7fefdc78f700 1 --2- 172.21.15.112:0/3185320904 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fefc0039020 0x7fefc003b4e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:55.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.987+0000 7fefdc78f700 1 --2- 172.21.15.112:0/3185320904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefd4102f30 0x7fefd419d610 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:55.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.987+0000 7fefdc78f700 1 -- 172.21.15.112:0/3185320904 >> 172.21.15.112:0/3185320904 conn(0x7fefd4077970 msgr2=0x7fefd4103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:55.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.987+0000 7fefdc78f700 1 -- 172.21.15.112:0/3185320904 shutdown_connections 2024-03-16T09:06:55.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:55.987+0000 7fefdc78f700 1 -- 172.21.15.112:0/3185320904 wait complete. 2024-03-16T09:06:56.091 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:55 smithi112 ceph-mon[32307]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:06:56.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:55 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:06:56.542 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi194 2024-03-16T09:06:56.543 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:06:56.543 DEBUG:teuthology.orchestra.run.smithi194:> dd of=/etc/ceph/ceph.conf 2024-03-16T09:06:56.561 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:06:56.561 DEBUG:teuthology.orchestra.run.smithi194:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:06:56.619 INFO:tasks.cephadm:Adding host smithi194 to orchestrator... 2024-03-16T09:06:56.619 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch host add smithi194 2024-03-16T09:06:56.852 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:06:56.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:56 smithi112 ceph-mon[32307]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:06:57.859 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:57 smithi112 ceph-mon[32307]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:06:58.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.302+0000 7fac7eb0a700 1 -- 172.21.15.112:0/3980298276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac78100e30 msgr2=0x7fac78101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:06:58.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.302+0000 7fac7eb0a700 1 --2- 172.21.15.112:0/3980298276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac78100e30 0x7fac78101210 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009910 tx=0x7fac6c009c20 comp rx=0 tx=0).stop 2024-03-16T09:06:58.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.302+0000 7fac7eb0a700 1 -- 172.21.15.112:0/3980298276 shutdown_connections 2024-03-16T09:06:58.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.302+0000 7fac7eb0a700 1 --2- 172.21.15.112:0/3980298276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac78100e30 0x7fac78101210 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:06:58.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.302+0000 7fac7eb0a700 1 -- 172.21.15.112:0/3980298276 >> 172.21.15.112:0/3980298276 conn(0x7fac78077970 msgr2=0x7fac780fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:06:58.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.302+0000 7fac7eb0a700 1 -- 172.21.15.112:0/3980298276 shutdown_connections 2024-03-16T09:06:58.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.302+0000 7fac7eb0a700 1 -- 172.21.15.112:0/3980298276 wait complete. 2024-03-16T09:06:58.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.302+0000 7fac7eb0a700 1 Processor -- start 2024-03-16T09:06:58.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.303+0000 7fac7eb0a700 1 -- start start 2024-03-16T09:06:58.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.303+0000 7fac7eb0a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac78100e30 0x7fac7819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:58.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.303+0000 7fac7eb0a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac7810b2f0 con 0x7fac78100e30 2024-03-16T09:06:58.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.303+0000 7fac7c8a6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac78100e30 0x7fac7819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:58.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.303+0000 7fac7c8a6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac78100e30 0x7fac7819d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50138/0 (socket says 172.21.15.112:50138) 2024-03-16T09:06:58.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.303+0000 7fac7c8a6700 1 -- 172.21.15.112:0/482051488 learned_addr learned my addr 172.21.15.112:0/482051488 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:06:58.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.303+0000 7fac7c8a6700 1 -- 172.21.15.112:0/482051488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac7819dae0 con 0x7fac78100e30 2024-03-16T09:06:58.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.303+0000 7fac7c8a6700 1 --2- 172.21.15.112:0/482051488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac78100e30 0x7fac7819d5a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0052a0 tx=0x7fac6c004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:58.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.303+0000 7fac75ffb700 1 -- 172.21.15.112:0/482051488 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac6c026070 con 0x7fac78100e30 2024-03-16T09:06:58.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.303+0000 7fac75ffb700 1 -- 172.21.15.112:0/482051488 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fac6c00bbf0 con 0x7fac78100e30 2024-03-16T09:06:58.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.303+0000 7fac7eb0a700 1 -- 172.21.15.112:0/482051488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7819ddd0 con 0x7fac78100e30 2024-03-16T09:06:58.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.303+0000 7fac75ffb700 1 -- 172.21.15.112:0/482051488 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac6c003a30 con 0x7fac78100e30 2024-03-16T09:06:58.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.304+0000 7fac7eb0a700 1 -- 172.21.15.112:0/482051488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac78197890 con 0x7fac78100e30 2024-03-16T09:06:58.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.304+0000 7fac75ffb700 1 -- 172.21.15.112:0/482051488 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7fac6c029070 con 0x7fac78100e30 2024-03-16T09:06:58.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.304+0000 7fac7eb0a700 1 -- 172.21.15.112:0/482051488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac781022c0 con 0x7fac78100e30 2024-03-16T09:06:58.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.305+0000 7fac75ffb700 1 --2- 172.21.15.112:0/482051488 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fac68039020 0x7fac6803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:06:58.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.305+0000 7fac75ffb700 1 -- 172.21.15.112:0/482051488 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fac6c01e070 con 0x7fac78100e30 2024-03-16T09:06:58.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.305+0000 7fac77fff700 1 --2- 172.21.15.112:0/482051488 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fac68039020 0x7fac6803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:06:58.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.305+0000 7fac77fff700 1 --2- 172.21.15.112:0/482051488 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fac68039020 0x7fac6803b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fac64006fd0 tx=0x7fac64006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:06:58.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.309+0000 7fac75ffb700 1 -- 172.21.15.112:0/482051488 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fac6c013070 con 0x7fac78100e30 2024-03-16T09:06:58.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:06:58.509+0000 7fac7eb0a700 1 -- 172.21.15.112:0/482051488 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi194", "target": ["mon-mgr", ""]}) -- 0x7fac78198ba0 con 0x7fac68039020 2024-03-16T09:07:00.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:59 smithi112 ceph-mon[32307]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi194", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:00.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:00.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:00.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:00.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:00.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:07:00.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:00.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:07:00.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:59 smithi112 ceph-mon[32307]: Updating smithi159:/etc/ceph/ceph.conf 2024-03-16T09:07:00.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:59 smithi112 ceph-mon[32307]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:00.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:59 smithi112 ceph-mon[32307]: Updating smithi159:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:07:00.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:00.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:00.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:06:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:01.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:00 smithi112 ceph-mon[32307]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:07:01.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:00 smithi112 ceph-mon[32307]: Deploying cephadm binary to smithi194 2024-03-16T09:07:01.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:00 smithi112 ceph-mon[32307]: Updating smithi159:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:07:01.841 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:01 smithi112 ceph-mon[32307]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:02.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:02.578+0000 7fac75ffb700 1 -- 172.21.15.112:0/482051488 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fac78198ba0 con 0x7fac68039020 2024-03-16T09:07:02.581 INFO:teuthology.orchestra.run.smithi112.stdout:Added host 'smithi194' with addr '172.21.15.194' 2024-03-16T09:07:02.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:02.580+0000 7fac7eb0a700 1 -- 172.21.15.112:0/482051488 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fac68039020 msgr2=0x7fac6803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:02.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:02.580+0000 7fac7eb0a700 1 --2- 172.21.15.112:0/482051488 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fac68039020 0x7fac6803b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fac64006fd0 tx=0x7fac64006e40 comp rx=0 tx=0).stop 2024-03-16T09:07:02.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:02.580+0000 7fac7eb0a700 1 -- 172.21.15.112:0/482051488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac78100e30 msgr2=0x7fac7819d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:02.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:02.580+0000 7fac7eb0a700 1 --2- 172.21.15.112:0/482051488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac78100e30 0x7fac7819d5a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fac6c0052a0 tx=0x7fac6c004cc0 comp rx=0 tx=0).stop 2024-03-16T09:07:02.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:02.581+0000 7fac7eb0a700 1 -- 172.21.15.112:0/482051488 shutdown_connections 2024-03-16T09:07:02.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:02.581+0000 7fac7eb0a700 1 --2- 172.21.15.112:0/482051488 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fac68039020 0x7fac6803b4e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:02.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:02.581+0000 7fac7eb0a700 1 --2- 172.21.15.112:0/482051488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac78100e30 0x7fac7819d5a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:02.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:02.581+0000 7fac7eb0a700 1 -- 172.21.15.112:0/482051488 >> 172.21.15.112:0/482051488 conn(0x7fac78077970 msgr2=0x7fac780696d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:02.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:02.581+0000 7fac7eb0a700 1 -- 172.21.15.112:0/482051488 shutdown_connections 2024-03-16T09:07:02.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:02.581+0000 7fac7eb0a700 1 -- 172.21.15.112:0/482051488 wait complete. 2024-03-16T09:07:03.220 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-16T09:07:03.455 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:07:03.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:03 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:03.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:03 smithi112 ceph-mon[32307]: Added host smithi194 2024-03-16T09:07:03.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:03 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:03.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:03 smithi112 ceph-mon[32307]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:03.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:03 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:03.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:03 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:04.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.807+0000 7f716e978700 1 -- 172.21.15.112:0/212987262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7168100dd0 msgr2=0x7f71681011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:04.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.807+0000 7f716e978700 1 --2- 172.21.15.112:0/212987262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7168100dd0 0x7f71681011b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7158009910 tx=0x7f7158009c20 comp rx=0 tx=0).stop 2024-03-16T09:07:04.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.807+0000 7f716e978700 1 -- 172.21.15.112:0/212987262 shutdown_connections 2024-03-16T09:07:04.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.807+0000 7f716e978700 1 --2- 172.21.15.112:0/212987262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7168100dd0 0x7f71681011b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:04.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.807+0000 7f716e978700 1 -- 172.21.15.112:0/212987262 >> 172.21.15.112:0/212987262 conn(0x7f71680fc660 msgr2=0x7f71680fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:04.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.807+0000 7f716e978700 1 -- 172.21.15.112:0/212987262 shutdown_connections 2024-03-16T09:07:04.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.807+0000 7f716e978700 1 -- 172.21.15.112:0/212987262 wait complete. 2024-03-16T09:07:04.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.808+0000 7f716e978700 1 Processor -- start 2024-03-16T09:07:04.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.808+0000 7f716e978700 1 -- start start 2024-03-16T09:07:04.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.808+0000 7f716e978700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7168100dd0 0x7f716819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:04.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.808+0000 7f716e978700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f716810b2e0 con 0x7f7168100dd0 2024-03-16T09:07:04.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.808+0000 7f7167fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7168100dd0 0x7f716819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:04.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.808+0000 7f7167fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7168100dd0 0x7f716819d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39400/0 (socket says 172.21.15.112:39400) 2024-03-16T09:07:04.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.808+0000 7f7167fff700 1 -- 172.21.15.112:0/366993659 learned_addr learned my addr 172.21.15.112:0/366993659 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:07:04.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.809+0000 7f7167fff700 1 -- 172.21.15.112:0/366993659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716819daf0 con 0x7f7168100dd0 2024-03-16T09:07:04.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.809+0000 7f7167fff700 1 --2- 172.21.15.112:0/366993659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7168100dd0 0x7f716819d5b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f71580052a0 tx=0x7f7158004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:04.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.809+0000 7f71657fa700 1 -- 172.21.15.112:0/366993659 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7158026070 con 0x7f7168100dd0 2024-03-16T09:07:04.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.809+0000 7f716e978700 1 -- 172.21.15.112:0/366993659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f716819dde0 con 0x7f7168100dd0 2024-03-16T09:07:04.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.809+0000 7f71657fa700 1 -- 172.21.15.112:0/366993659 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f715800bbf0 con 0x7f7168100dd0 2024-03-16T09:07:04.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.809+0000 7f71657fa700 1 -- 172.21.15.112:0/366993659 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7158003a30 con 0x7f7168100dd0 2024-03-16T09:07:04.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.809+0000 7f716e978700 1 -- 172.21.15.112:0/366993659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71681978a0 con 0x7f7168100dd0 2024-03-16T09:07:04.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.810+0000 7f71657fa700 1 -- 172.21.15.112:0/366993659 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f7158029070 con 0x7f7168100dd0 2024-03-16T09:07:04.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.810+0000 7f716e978700 1 -- 172.21.15.112:0/366993659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71681022b0 con 0x7f7168100dd0 2024-03-16T09:07:04.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.810+0000 7f71657fa700 1 --2- 172.21.15.112:0/366993659 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f715003d3d0 0x7f715003f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:04.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.810+0000 7f71657fa700 1 -- 172.21.15.112:0/366993659 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f715801e070 con 0x7f7168100dd0 2024-03-16T09:07:04.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.810+0000 7f71677fe700 1 --2- 172.21.15.112:0/366993659 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f715003d3d0 0x7f715003f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:04.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.810+0000 7f71677fe700 1 --2- 172.21.15.112:0/366993659 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f715003d3d0 0x7f715003f890 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f715c006fd0 tx=0x7f715c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:04.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:04.815+0000 7f71657fa700 1 -- 172.21.15.112:0/366993659 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7158013070 con 0x7f7168100dd0 2024-03-16T09:07:05.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:05.016+0000 7f716e978700 1 -- 172.21.15.112:0/366993659 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7168198bb0 con 0x7f715003d3d0 2024-03-16T09:07:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:05.018+0000 7f71657fa700 1 -- 172.21.15.112:0/366993659 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f7168198bb0 con 0x7f715003d3d0 2024-03-16T09:07:05.021 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:07:05.021 INFO:teuthology.orchestra.run.smithi112.stdout:[{"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}, {"addr": "172.21.15.159", "hostname": "smithi159", "labels": [], "status": ""}, {"addr": "172.21.15.194", "hostname": "smithi194", "labels": [], "status": ""}] 2024-03-16T09:07:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:05.020+0000 7f716e978700 1 -- 172.21.15.112:0/366993659 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f715003d3d0 msgr2=0x7f715003f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:05.020+0000 7f716e978700 1 --2- 172.21.15.112:0/366993659 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f715003d3d0 0x7f715003f890 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f715c006fd0 tx=0x7f715c006e40 comp rx=0 tx=0).stop 2024-03-16T09:07:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:05.020+0000 7f716e978700 1 -- 172.21.15.112:0/366993659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7168100dd0 msgr2=0x7f716819d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:05.020+0000 7f716e978700 1 --2- 172.21.15.112:0/366993659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7168100dd0 0x7f716819d5b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f71580052a0 tx=0x7f7158004cc0 comp rx=0 tx=0).stop 2024-03-16T09:07:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:05.020+0000 7f716e978700 1 -- 172.21.15.112:0/366993659 shutdown_connections 2024-03-16T09:07:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:05.020+0000 7f716e978700 1 --2- 172.21.15.112:0/366993659 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f715003d3d0 0x7f715003f890 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:05.020+0000 7f716e978700 1 --2- 172.21.15.112:0/366993659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7168100dd0 0x7f716819d5b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:05.020+0000 7f716e978700 1 -- 172.21.15.112:0/366993659 >> 172.21.15.112:0/366993659 conn(0x7f71680fc660 msgr2=0x7f716806a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:05.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:05.020+0000 7f716e978700 1 -- 172.21.15.112:0/366993659 shutdown_connections 2024-03-16T09:07:05.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:05.020+0000 7f716e978700 1 -- 172.21.15.112:0/366993659 wait complete. 2024-03-16T09:07:05.597 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-16T09:07:05.597 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph osd crush tunables default 2024-03-16T09:07:05.832 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:07:05.953 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:05.953 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:05 smithi112 ceph-mon[32307]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:06.935 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:06 smithi112 ceph-mon[32307]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:07:07.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.396+0000 7f246b98b700 1 -- 172.21.15.112:0/262496531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2464100e30 msgr2=0x7f2464101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:07.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.396+0000 7f246b98b700 1 --2- 172.21.15.112:0/262496531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2464100e30 0x7f2464101210 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2460009910 tx=0x7f2460009c20 comp rx=0 tx=0).stop 2024-03-16T09:07:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.396+0000 7f246b98b700 1 -- 172.21.15.112:0/262496531 shutdown_connections 2024-03-16T09:07:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.396+0000 7f246b98b700 1 --2- 172.21.15.112:0/262496531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2464100e30 0x7f2464101210 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.396+0000 7f246b98b700 1 -- 172.21.15.112:0/262496531 >> 172.21.15.112:0/262496531 conn(0x7f2464077970 msgr2=0x7f24640fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.396+0000 7f246b98b700 1 -- 172.21.15.112:0/262496531 shutdown_connections 2024-03-16T09:07:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.396+0000 7f246b98b700 1 -- 172.21.15.112:0/262496531 wait complete. 2024-03-16T09:07:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.397+0000 7f246b98b700 1 Processor -- start 2024-03-16T09:07:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.397+0000 7f246b98b700 1 -- start start 2024-03-16T09:07:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.397+0000 7f246b98b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2464100e30 0x7f246419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.397+0000 7f246b98b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f246410b2f0 con 0x7f2464100e30 2024-03-16T09:07:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.397+0000 7f2469727700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2464100e30 0x7f246419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.397+0000 7f2469727700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2464100e30 0x7f246419d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39418/0 (socket says 172.21.15.112:39418) 2024-03-16T09:07:07.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.397+0000 7f2469727700 1 -- 172.21.15.112:0/997376725 learned_addr learned my addr 172.21.15.112:0/997376725 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:07:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.397+0000 7f2469727700 1 -- 172.21.15.112:0/997376725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246419db20 con 0x7f2464100e30 2024-03-16T09:07:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.397+0000 7f2469727700 1 --2- 172.21.15.112:0/997376725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2464100e30 0x7f246419d5e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f24600052a0 tx=0x7f2460004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.398+0000 7f24567fc700 1 -- 172.21.15.112:0/997376725 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2460026070 con 0x7f2464100e30 2024-03-16T09:07:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.398+0000 7f24567fc700 1 -- 172.21.15.112:0/997376725 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f246000bbf0 con 0x7f2464100e30 2024-03-16T09:07:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.398+0000 7f246b98b700 1 -- 172.21.15.112:0/997376725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f246419de10 con 0x7f2464100e30 2024-03-16T09:07:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.398+0000 7f24567fc700 1 -- 172.21.15.112:0/997376725 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2460003a30 con 0x7f2464100e30 2024-03-16T09:07:07.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.398+0000 7f246b98b700 1 -- 172.21.15.112:0/997376725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2464197880 con 0x7f2464100e30 2024-03-16T09:07:07.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.398+0000 7f24567fc700 1 -- 172.21.15.112:0/997376725 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f2460029070 con 0x7f2464100e30 2024-03-16T09:07:07.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.399+0000 7f24567fc700 1 --2- 172.21.15.112:0/997376725 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2450039020 0x7f245003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:07.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.399+0000 7f24567fc700 1 -- 172.21.15.112:0/997376725 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f246001e070 con 0x7f2464100e30 2024-03-16T09:07:07.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.399+0000 7f246b98b700 1 -- 172.21.15.112:0/997376725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24641022c0 con 0x7f2464100e30 2024-03-16T09:07:07.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.399+0000 7f2468f26700 1 --2- 172.21.15.112:0/997376725 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2450039020 0x7f245003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:07.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.399+0000 7f2468f26700 1 --2- 172.21.15.112:0/997376725 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2450039020 0x7f245003b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2458006fd0 tx=0x7f2458006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:07.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.404+0000 7f24567fc700 1 -- 172.21.15.112:0/997376725 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2460030020 con 0x7f2464100e30 2024-03-16T09:07:07.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.603+0000 7f246b98b700 1 -- 172.21.15.112:0/997376725 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f24640008d0 con 0x7f2464100e30 2024-03-16T09:07:07.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.659+0000 7f24567fc700 1 -- 172.21.15.112:0/997376725 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f2460013070 con 0x7f2464100e30 2024-03-16T09:07:07.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.661+0000 7f246b98b700 1 -- 172.21.15.112:0/997376725 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2450039020 msgr2=0x7f245003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:07.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.661+0000 7f246b98b700 1 --2- 172.21.15.112:0/997376725 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2450039020 0x7f245003b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2458006fd0 tx=0x7f2458006e40 comp rx=0 tx=0).stop 2024-03-16T09:07:07.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.661+0000 7f246b98b700 1 -- 172.21.15.112:0/997376725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2464100e30 msgr2=0x7f246419d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:07.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.661+0000 7f246b98b700 1 --2- 172.21.15.112:0/997376725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2464100e30 0x7f246419d5e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f24600052a0 tx=0x7f2460004cc0 comp rx=0 tx=0).stop 2024-03-16T09:07:07.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.661+0000 7f246b98b700 1 -- 172.21.15.112:0/997376725 shutdown_connections 2024-03-16T09:07:07.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.661+0000 7f246b98b700 1 --2- 172.21.15.112:0/997376725 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2450039020 0x7f245003b4e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:07.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.661+0000 7f246b98b700 1 --2- 172.21.15.112:0/997376725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2464100e30 0x7f246419d5e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:07.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.661+0000 7f246b98b700 1 -- 172.21.15.112:0/997376725 >> 172.21.15.112:0/997376725 conn(0x7f2464077970 msgr2=0x7f24640696d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:07.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.661+0000 7f246b98b700 1 -- 172.21.15.112:0/997376725 shutdown_connections 2024-03-16T09:07:07.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:07.661+0000 7f246b98b700 1 -- 172.21.15.112:0/997376725 wait complete. 2024-03-16T09:07:07.665 INFO:teuthology.orchestra.run.smithi112.stderr:adjusted tunables profile to default 2024-03-16T09:07:08.060 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:07 smithi112 ceph-mon[32307]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:08.060 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:07 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/997376725' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-16T09:07:08.413 INFO:tasks.cephadm:Adding mon.a on smithi112 2024-03-16T09:07:08.413 INFO:tasks.cephadm:Adding mon.b on smithi159 2024-03-16T09:07:08.413 INFO:tasks.cephadm:Adding mon.c on smithi194 2024-03-16T09:07:08.413 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch apply mon '3;smithi112:172.21.15.112=a;smithi159:172.21.15.159=b;smithi194:172.21.15.194=c' 2024-03-16T09:07:08.649 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:07:09.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:08 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/997376725' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-16T09:07:09.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:08 smithi112 ceph-mon[32307]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:07:09.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:09.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:09.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:09.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:09.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi194", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:07:09.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:09.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:07:09.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:08 smithi112 ceph-mon[32307]: Updating smithi194:/etc/ceph/ceph.conf 2024-03-16T09:07:09.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:08 smithi112 ceph-mon[32307]: Updating smithi194:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:07:09.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:09.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:09.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:10.050 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.047+0000 7f2e758f9700 1 -- 172.21.15.194:0/477587782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e70101ef0 msgr2=0x7f2e701022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:10.050 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.047+0000 7f2e758f9700 1 --2- 172.21.15.194:0/477587782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e70101ef0 0x7f2e701022d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800b0a0 tx=0x7f2e6800b3b0 comp rx=0 tx=0).stop 2024-03-16T09:07:10.050 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.047+0000 7f2e758f9700 1 -- 172.21.15.194:0/477587782 shutdown_connections 2024-03-16T09:07:10.050 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.047+0000 7f2e758f9700 1 --2- 172.21.15.194:0/477587782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e70101ef0 0x7f2e701022d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:10.051 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.047+0000 7f2e758f9700 1 -- 172.21.15.194:0/477587782 >> 172.21.15.194:0/477587782 conn(0x7f2e700fd7a0 msgr2=0x7f2e700ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:10.051 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.047+0000 7f2e758f9700 1 -- 172.21.15.194:0/477587782 shutdown_connections 2024-03-16T09:07:10.051 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.047+0000 7f2e758f9700 1 -- 172.21.15.194:0/477587782 wait complete. 2024-03-16T09:07:10.051 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.048+0000 7f2e758f9700 1 Processor -- start 2024-03-16T09:07:10.051 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.048+0000 7f2e758f9700 1 -- start start 2024-03-16T09:07:10.051 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.048+0000 7f2e758f9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e70101ef0 0x7f2e700745d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:10.051 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.048+0000 7f2e758f9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7010c400 con 0x7f2e70101ef0 2024-03-16T09:07:10.053 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.048+0000 7f2e6effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e70101ef0 0x7f2e700745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:10.053 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.048+0000 7f2e6effd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e70101ef0 0x7f2e700745d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.194:60790/0 (socket says 172.21.15.194:60790) 2024-03-16T09:07:10.053 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.048+0000 7f2e6effd700 1 -- 172.21.15.194:0/775004006 learned_addr learned my addr 172.21.15.194:0/775004006 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T09:07:10.053 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.049+0000 7f2e6effd700 1 -- 172.21.15.194:0/775004006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e700764c0 con 0x7f2e70101ef0 2024-03-16T09:07:10.053 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.049+0000 7f2e6effd700 1 --2- 172.21.15.194:0/775004006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e70101ef0 0x7f2e700745d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2e68015040 tx=0x7f2e6800bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:10.053 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.049+0000 7f2e748f7700 1 -- 172.21.15.194:0/775004006 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e68017040 con 0x7f2e70101ef0 2024-03-16T09:07:10.053 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.049+0000 7f2e758f9700 1 -- 172.21.15.194:0/775004006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e70074b70 con 0x7f2e70101ef0 2024-03-16T09:07:10.053 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.049+0000 7f2e748f7700 1 -- 172.21.15.194:0/775004006 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2e68022070 con 0x7f2e70101ef0 2024-03-16T09:07:10.053 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.049+0000 7f2e748f7700 1 -- 172.21.15.194:0/775004006 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e6801d440 con 0x7f2e70101ef0 2024-03-16T09:07:10.054 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.049+0000 7f2e758f9700 1 -- 172.21.15.194:0/775004006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e70075030 con 0x7f2e70101ef0 2024-03-16T09:07:10.054 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.050+0000 7f2e748f7700 1 -- 172.21.15.194:0/775004006 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f2e680041d0 con 0x7f2e70101ef0 2024-03-16T09:07:10.054 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.050+0000 7f2e758f9700 1 -- 172.21.15.194:0/775004006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e701033d0 con 0x7f2e70101ef0 2024-03-16T09:07:10.054 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.050+0000 7f2e748f7700 1 --2- 172.21.15.194:0/775004006 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2e58039010 0x7f2e5803b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:10.054 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.050+0000 7f2e748f7700 1 -- 172.21.15.194:0/775004006 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2e6800f050 con 0x7f2e70101ef0 2024-03-16T09:07:10.054 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.051+0000 7f2e6e7fc700 1 --2- 172.21.15.194:0/775004006 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2e58039010 0x7f2e5803b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:10.054 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.051+0000 7f2e6e7fc700 1 --2- 172.21.15.194:0/775004006 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2e58039010 0x7f2e5803b4d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2e64006fd0 tx=0x7f2e64006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:10.057 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.055+0000 7f2e748f7700 1 -- 172.21.15.194:0/775004006 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2e6801c600 con 0x7f2e70101ef0 2024-03-16T09:07:10.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:09 smithi112 ceph-mon[32307]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:07:10.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:09 smithi112 ceph-mon[32307]: Updating smithi194:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:07:10.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:09 smithi112 ceph-mon[32307]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:10.268 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.266+0000 7f2e758f9700 1 -- 172.21.15.194:0/775004006 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi159:172.21.15.159=b;smithi194:172.21.15.194=c", "target": ["mon-mgr", ""]}) -- 0x7f2e70075260 con 0x7f2e58039010 2024-03-16T09:07:10.275 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.273+0000 7f2e748f7700 1 -- 172.21.15.194:0/775004006 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2e70075260 con 0x7f2e58039010 2024-03-16T09:07:10.275 INFO:teuthology.orchestra.run.smithi194.stdout:Scheduled mon update... 2024-03-16T09:07:10.278 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.275+0000 7f2e758f9700 1 -- 172.21.15.194:0/775004006 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2e58039010 msgr2=0x7f2e5803b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:10.278 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.275+0000 7f2e758f9700 1 --2- 172.21.15.194:0/775004006 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2e58039010 0x7f2e5803b4d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2e64006fd0 tx=0x7f2e64006e40 comp rx=0 tx=0).stop 2024-03-16T09:07:10.278 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.275+0000 7f2e758f9700 1 -- 172.21.15.194:0/775004006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e70101ef0 msgr2=0x7f2e700745d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:10.278 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.275+0000 7f2e758f9700 1 --2- 172.21.15.194:0/775004006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e70101ef0 0x7f2e700745d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2e68015040 tx=0x7f2e6800bb10 comp rx=0 tx=0).stop 2024-03-16T09:07:10.278 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.275+0000 7f2e758f9700 1 -- 172.21.15.194:0/775004006 shutdown_connections 2024-03-16T09:07:10.278 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.275+0000 7f2e758f9700 1 --2- 172.21.15.194:0/775004006 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2e58039010 0x7f2e5803b4d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:10.278 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.275+0000 7f2e758f9700 1 --2- 172.21.15.194:0/775004006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e70101ef0 0x7f2e700745d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:10.278 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.275+0000 7f2e758f9700 1 -- 172.21.15.194:0/775004006 >> 172.21.15.194:0/775004006 conn(0x7f2e700fd7a0 msgr2=0x7f2e701038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:10.278 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.275+0000 7f2e758f9700 1 -- 172.21.15.194:0/775004006 shutdown_connections 2024-03-16T09:07:10.278 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:10.275+0000 7f2e758f9700 1 -- 172.21.15.194:0/775004006 wait complete. 2024-03-16T09:07:10.919 DEBUG:teuthology.orchestra.run.smithi159:mon.b> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mon.b.service 2024-03-16T09:07:10.922 DEBUG:teuthology.orchestra.run.smithi194:mon.c> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mon.c.service 2024-03-16T09:07:10.924 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-16T09:07:10.924 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T09:07:11.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:11 smithi112 ceph-mon[32307]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi159:172.21.15.159=b;smithi194:172.21.15.194=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:11.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:11 smithi112 ceph-mon[32307]: Saving service mon spec with placement smithi112:172.21.15.112=a;smithi159:172.21.15.159=b;smithi194:172.21.15.194=c;count:3 2024-03-16T09:07:11.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:11 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:11.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:11 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:11.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:11 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:11.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:11 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:07:11.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:11 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:11.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:11 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:07:11.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:11 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:11.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:11 smithi112 ceph-mon[32307]: Deploying daemon mon.c on smithi194 2024-03-16T09:07:11.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:11 smithi112 ceph-mon[32307]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:12.078 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:07:14.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:13 smithi112 ceph-mon[32307]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:15.109 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:14 smithi194 systemd[1]: Started Ceph mon.c for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:07:15.109 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.940+0000 7f121c525700 1 -- 172.21.15.194:0/444988207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1214101f40 msgr2=0x7f1214102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:15.110 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.940+0000 7f121c525700 1 --2- 172.21.15.194:0/444988207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1214101f40 0x7f1214102320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1210009910 tx=0x7f1210009c20 comp rx=0 tx=0).stop 2024-03-16T09:07:15.110 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.940+0000 7f121c525700 1 -- 172.21.15.194:0/444988207 shutdown_connections 2024-03-16T09:07:15.110 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.940+0000 7f121c525700 1 --2- 172.21.15.194:0/444988207 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1214101f40 0x7f1214102320 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:15.110 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.940+0000 7f121c525700 1 -- 172.21.15.194:0/444988207 >> 172.21.15.194:0/444988207 conn(0x7f12140fd7f0 msgr2=0x7f12140ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:15.110 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.940+0000 7f121c525700 1 -- 172.21.15.194:0/444988207 shutdown_connections 2024-03-16T09:07:15.110 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.940+0000 7f121c525700 1 -- 172.21.15.194:0/444988207 wait complete. 2024-03-16T09:07:15.110 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.941+0000 7f121c525700 1 Processor -- start 2024-03-16T09:07:15.110 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.941+0000 7f121c525700 1 -- start start 2024-03-16T09:07:15.110 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.941+0000 7f121c525700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1214101f40 0x7f121419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:15.110 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.941+0000 7f121c525700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f121410c450 con 0x7f1214101f40 2024-03-16T09:07:15.110 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.941+0000 7f121a2c1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1214101f40 0x7f121419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:15.110 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.941+0000 7f121a2c1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1214101f40 0x7f121419d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.194:60690/0 (socket says 172.21.15.194:60690) 2024-03-16T09:07:15.111 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.941+0000 7f121a2c1700 1 -- 172.21.15.194:0/2125135243 learned_addr learned my addr 172.21.15.194:0/2125135243 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T09:07:15.111 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.942+0000 7f121a2c1700 1 -- 172.21.15.194:0/2125135243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121419d990 con 0x7f1214101f40 2024-03-16T09:07:15.111 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.942+0000 7f121a2c1700 1 --2- 172.21.15.194:0/2125135243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1214101f40 0x7f121419d450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f12100052a0 tx=0x7f1210004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:15.111 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.942+0000 7f120b7fe700 1 -- 172.21.15.194:0/2125135243 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1210026070 con 0x7f1214101f40 2024-03-16T09:07:15.111 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.942+0000 7f120b7fe700 1 -- 172.21.15.194:0/2125135243 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f121000bbf0 con 0x7f1214101f40 2024-03-16T09:07:15.111 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.942+0000 7f121c525700 1 -- 172.21.15.194:0/2125135243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121419dc80 con 0x7f1214101f40 2024-03-16T09:07:15.111 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.942+0000 7f120b7fe700 1 -- 172.21.15.194:0/2125135243 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1210003a30 con 0x7f1214101f40 2024-03-16T09:07:15.111 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.942+0000 7f121c525700 1 -- 172.21.15.194:0/2125135243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1214197610 con 0x7f1214101f40 2024-03-16T09:07:15.111 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.943+0000 7f120b7fe700 1 -- 172.21.15.194:0/2125135243 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f1210029070 con 0x7f1214101f40 2024-03-16T09:07:15.111 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.943+0000 7f120b7fe700 1 --2- 172.21.15.194:0/2125135243 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1200038fc0 0x7f120003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:15.111 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.943+0000 7f120b7fe700 1 -- 172.21.15.194:0/2125135243 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f121001e070 con 0x7f1214101f40 2024-03-16T09:07:15.111 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.943+0000 7f121c525700 1 -- 172.21.15.194:0/2125135243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1214103420 con 0x7f1214101f40 2024-03-16T09:07:15.111 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.944+0000 7f1219ac0700 1 --2- 172.21.15.194:0/2125135243 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1200038fc0 0x7f120003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:15.111 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.944+0000 7f1219ac0700 1 --2- 172.21.15.194:0/2125135243 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1200038fc0 0x7f120003b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1204006fd0 tx=0x7f1204006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:15.112 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.949+0000 7f120b7fe700 1 -- 172.21.15.194:0/2125135243 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1210030020 con 0x7f1214101f40 2024-03-16T09:07:15.112 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:14.974+0000 7f120b7fe700 1 -- 172.21.15.194:0/2125135243 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f121000ec30 con 0x7f1214101f40 2024-03-16T09:07:15.243 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:15.240+0000 7f121c525700 1 -- 172.21.15.194:0/2125135243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1214063900 con 0x7f1214101f40 2024-03-16T09:07:20.027 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:20.024+0000 7f120b7fe700 1 -- 172.21.15.194:0/2125135243 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f1210013070 con 0x7f1214101f40 2024-03-16T09:07:20.027 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:07:20.027 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":2,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","modified":"2024-03-16T09:07:14.975453Z","created":"2024-03-16T09:05:27.608380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:3300","nonce":0},{"type":"v1","addr":"172.21.15.194:6789","nonce":0}]},"addr":"172.21.15.194:6789/0","public_addr":"172.21.15.194:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-16T09:07:20.029 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:20.027+0000 7f121c525700 1 -- 172.21.15.194:0/2125135243 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1200038fc0 msgr2=0x7f120003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:20.030 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:20.027+0000 7f121c525700 1 --2- 172.21.15.194:0/2125135243 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1200038fc0 0x7f120003b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1204006fd0 tx=0x7f1204006e40 comp rx=0 tx=0).stop 2024-03-16T09:07:20.030 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:20.027+0000 7f121c525700 1 -- 172.21.15.194:0/2125135243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1214101f40 msgr2=0x7f121419d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:20.030 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:20.027+0000 7f121c525700 1 --2- 172.21.15.194:0/2125135243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1214101f40 0x7f121419d450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f12100052a0 tx=0x7f1210004cc0 comp rx=0 tx=0).stop 2024-03-16T09:07:20.030 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:20.027+0000 7f121c525700 1 -- 172.21.15.194:0/2125135243 shutdown_connections 2024-03-16T09:07:20.030 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:20.027+0000 7f121c525700 1 --2- 172.21.15.194:0/2125135243 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1200038fc0 0x7f120003b480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:20.030 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:20.027+0000 7f121c525700 1 --2- 172.21.15.194:0/2125135243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1214101f40 0x7f121419d450 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:20.030 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:20.027+0000 7f121c525700 1 -- 172.21.15.194:0/2125135243 >> 172.21.15.194:0/2125135243 conn(0x7f12140fd7f0 msgr2=0x7f1214103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:20.030 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:20.028+0000 7f121c525700 1 -- 172.21.15.194:0/2125135243 shutdown_connections 2024-03-16T09:07:20.030 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:20.028+0000 7f121c525700 1 -- 172.21.15.194:0/2125135243 wait complete. 2024-03-16T09:07:20.030 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 2 2024-03-16T09:07:20.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:07:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: mon.a calling monitor election 2024-03-16T09:07:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: from='client.? 172.21.15.194:0/2125135243' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T09:07:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: mon.c calling monitor election 2024-03-16T09:07:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-16T09:07:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: monmap epoch 2 2024-03-16T09:07:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: fsid 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:07:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: last_changed 2024-03-16T09:07:14.975453+0000 2024-03-16T09:07:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: created 2024-03-16T09:05:27.608380+0000 2024-03-16T09:07:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: min_mon_release 19 (squid) 2024-03-16T09:07:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: election_strategy: 1 2024-03-16T09:07:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-03-16T09:07:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-03-16T09:07:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: fsmap 2024-03-16T09:07:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:07:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: mgrmap e13: x(active, since 49s) 2024-03-16T09:07:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: overall HEALTH_OK 2024-03-16T09:07:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:07:20.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:20.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:20.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:07:20.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: mon.a calling monitor election 2024-03-16T09:07:20.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:20.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: from='client.? 172.21.15.194:0/2125135243' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T09:07:20.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: mon.c calling monitor election 2024-03-16T09:07:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-16T09:07:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: monmap epoch 2 2024-03-16T09:07:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: fsid 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:07:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: last_changed 2024-03-16T09:07:14.975453+0000 2024-03-16T09:07:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: created 2024-03-16T09:05:27.608380+0000 2024-03-16T09:07:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: min_mon_release 19 (squid) 2024-03-16T09:07:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: election_strategy: 1 2024-03-16T09:07:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-03-16T09:07:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-03-16T09:07:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: fsmap 2024-03-16T09:07:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:07:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: mgrmap e13: x(active, since 49s) 2024-03-16T09:07:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: overall HEALTH_OK 2024-03-16T09:07:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:07:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:21 smithi112 ceph-mon[32307]: Deploying daemon mon.b on smithi159 2024-03-16T09:07:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:21 smithi112 ceph-mon[32307]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:21 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:21.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:21 smithi194 ceph-mon[36729]: Deploying daemon mon.b on smithi159 2024-03-16T09:07:21.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:21 smithi194 ceph-mon[36729]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:21.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:21 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:21.713 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-16T09:07:21.713 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T09:07:21.965 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:07:23.636 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.633+0000 7f9c0d82d700 1 -- 172.21.15.194:0/861390110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c081064c0 msgr2=0x7f9c081068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:23.636 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.633+0000 7f9c0d82d700 1 --2- 172.21.15.194:0/861390110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c081064c0 0x7f9c081068a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000b0a0 tx=0x7f9c0000b3b0 comp rx=0 tx=0).stop 2024-03-16T09:07:23.636 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.634+0000 7f9c0d82d700 1 -- 172.21.15.194:0/861390110 shutdown_connections 2024-03-16T09:07:23.636 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.634+0000 7f9c0d82d700 1 --2- 172.21.15.194:0/861390110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c081064c0 0x7f9c081068a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:23.636 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.634+0000 7f9c0d82d700 1 -- 172.21.15.194:0/861390110 >> 172.21.15.194:0/861390110 conn(0x7f9c080773c0 msgr2=0x7f9c080777d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:23.636 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.634+0000 7f9c0d82d700 1 -- 172.21.15.194:0/861390110 shutdown_connections 2024-03-16T09:07:23.636 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.634+0000 7f9c0d82d700 1 -- 172.21.15.194:0/861390110 wait complete. 2024-03-16T09:07:23.639 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.634+0000 7f9c0d82d700 1 Processor -- start 2024-03-16T09:07:23.639 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.635+0000 7f9c0d82d700 1 -- start start 2024-03-16T09:07:23.639 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.635+0000 7f9c0d82d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c081064c0 0x7f9c0819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:23.639 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.635+0000 7f9c0d82d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c0819dcf0 0x7f9c081979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:23.639 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.635+0000 7f9c0d82d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0810abb0 con 0x7f9c081064c0 2024-03-16T09:07:23.640 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.635+0000 7f9c0d82d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0810aa30 con 0x7f9c0819dcf0 2024-03-16T09:07:23.640 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.635+0000 7f9c067fc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c0819dcf0 0x7f9c081979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:23.640 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.635+0000 7f9c067fc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c0819dcf0 0x7f9c081979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:41514/0 (socket says 172.21.15.194:41514) 2024-03-16T09:07:23.640 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.635+0000 7f9c067fc700 1 -- 172.21.15.194:0/50875191 learned_addr learned my addr 172.21.15.194:0/50875191 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T09:07:23.640 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.635+0000 7f9c067fc700 1 -- 172.21.15.194:0/50875191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c0819dcf0 msgr2=0x7f9c081979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-16T09:07:23.640 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.635+0000 7f9c067fc700 1 -- 172.21.15.194:0/50875191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c0819dcf0 msgr2=0x7f9c081979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-16T09:07:23.640 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.635+0000 7f9c067fc700 1 --2- 172.21.15.194:0/50875191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c0819dcf0 0x7f9c081979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T09:07:23.640 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.635+0000 7f9c067fc700 1 --2- 172.21.15.194:0/50875191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c0819dcf0 0x7f9c081979c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:07:23.640 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.635+0000 7f9c06ffd700 1 --2- 172.21.15.194:0/50875191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c081064c0 0x7f9c0819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:23.640 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.635+0000 7f9c06ffd700 1 -- 172.21.15.194:0/50875191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c0819dcf0 msgr2=0x7f9c081979c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T09:07:23.640 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.635+0000 7f9c06ffd700 1 --2- 172.21.15.194:0/50875191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c0819dcf0 0x7f9c081979c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:23.640 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.636+0000 7f9c06ffd700 1 -- 172.21.15.194:0/50875191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0819e330 con 0x7f9c081064c0 2024-03-16T09:07:23.640 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.636+0000 7f9c06ffd700 1 --2- 172.21.15.194:0/50875191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c081064c0 0x7f9c0819d7b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000b830 tx=0x7f9c000078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:23.640 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.636+0000 7f9c0c82b700 1 -- 172.21.15.194:0/50875191 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9c00007bc0 con 0x7f9c081064c0 2024-03-16T09:07:23.641 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.636+0000 7f9c0c82b700 1 -- 172.21.15.194:0/50875191 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9c00022070 con 0x7f9c081064c0 2024-03-16T09:07:23.641 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.636+0000 7f9c0d82d700 1 -- 172.21.15.194:0/50875191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c08197f00 con 0x7f9c081064c0 2024-03-16T09:07:23.641 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.636+0000 7f9c0c82b700 1 -- 172.21.15.194:0/50875191 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9c00012790 con 0x7f9c081064c0 2024-03-16T09:07:23.641 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.636+0000 7f9c0d82d700 1 -- 172.21.15.194:0/50875191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c08198490 con 0x7f9c081064c0 2024-03-16T09:07:23.641 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.637+0000 7f9c0d82d700 1 -- 172.21.15.194:0/50875191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c08101b70 con 0x7f9c081064c0 2024-03-16T09:07:23.641 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.637+0000 7f9c0c82b700 1 -- 172.21.15.194:0/50875191 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f9c0001d440 con 0x7f9c081064c0 2024-03-16T09:07:23.641 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.637+0000 7f9c0c82b700 1 --2- 172.21.15.194:0/50875191 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9bf4038d80 0x7f9bf403b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:23.641 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.637+0000 7f9c0c82b700 1 -- 172.21.15.194:0/50875191 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9c0000f050 con 0x7f9c081064c0 2024-03-16T09:07:23.641 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.637+0000 7f9c067fc700 1 --2- 172.21.15.194:0/50875191 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9bf4038d80 0x7f9bf403b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:23.641 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.638+0000 7f9c067fc700 1 --2- 172.21.15.194:0/50875191 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9bf4038d80 0x7f9bf403b240 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009180 tx=0x7f9bf0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:23.645 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.643+0000 7f9c0c82b700 1 -- 172.21.15.194:0/50875191 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9c0001d730 con 0x7f9c081064c0 2024-03-16T09:07:23.920 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.918+0000 7f9c0d82d700 1 -- 172.21.15.194:0/50875191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9c08198ea0 con 0x7f9c081064c0 2024-03-16T09:07:23.920 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.918+0000 7f9c0c82b700 1 -- 172.21.15.194:0/50875191 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f9c0001db80 con 0x7f9c081064c0 2024-03-16T09:07:23.921 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:07:23.921 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":2,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","modified":"2024-03-16T09:07:14.975453Z","created":"2024-03-16T09:05:27.608380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:3300","nonce":0},{"type":"v1","addr":"172.21.15.194:6789","nonce":0}]},"addr":"172.21.15.194:6789/0","public_addr":"172.21.15.194:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-16T09:07:23.923 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.920+0000 7f9c0d82d700 1 -- 172.21.15.194:0/50875191 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9bf4038d80 msgr2=0x7f9bf403b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:23.923 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.920+0000 7f9c0d82d700 1 --2- 172.21.15.194:0/50875191 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9bf4038d80 0x7f9bf403b240 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009180 tx=0x7f9bf0008040 comp rx=0 tx=0).stop 2024-03-16T09:07:23.923 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.920+0000 7f9c0d82d700 1 -- 172.21.15.194:0/50875191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c081064c0 msgr2=0x7f9c0819d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:23.923 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.920+0000 7f9c0d82d700 1 --2- 172.21.15.194:0/50875191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c081064c0 0x7f9c0819d7b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000b830 tx=0x7f9c000078e0 comp rx=0 tx=0).stop 2024-03-16T09:07:23.923 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.921+0000 7f9c0d82d700 1 -- 172.21.15.194:0/50875191 shutdown_connections 2024-03-16T09:07:23.923 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.921+0000 7f9c0d82d700 1 --2- 172.21.15.194:0/50875191 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9bf4038d80 0x7f9bf403b240 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:23.923 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.921+0000 7f9c0d82d700 1 --2- 172.21.15.194:0/50875191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c081064c0 0x7f9c0819d7b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:23.923 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.921+0000 7f9c0d82d700 1 --2- 172.21.15.194:0/50875191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c0819dcf0 0x7f9c081979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:23.923 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.921+0000 7f9c0d82d700 1 -- 172.21.15.194:0/50875191 >> 172.21.15.194:0/50875191 conn(0x7f9c080773c0 msgr2=0x7f9c08108bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:23.923 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.921+0000 7f9c0d82d700 1 -- 172.21.15.194:0/50875191 shutdown_connections 2024-03-16T09:07:23.924 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:23.921+0000 7f9c0d82d700 1 -- 172.21.15.194:0/50875191 wait complete. 2024-03-16T09:07:23.924 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 2 2024-03-16T09:07:24.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:23 smithi194 ceph-mon[36729]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:24.220 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-16T09:07:24.220 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-16T09:07:24.220 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-16T09:07:24.220 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-16T09:07:24.220 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-16T09:07:24.220 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-16T09:07:24.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:23 smithi112 ceph-mon[32307]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:24.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-16T09:07:24.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T09:07:24.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T09:07:24.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-16T09:07:24.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:24.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:07:24.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:24.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi194", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:24.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:07:24.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:24.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:07:24.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: Updating smithi159:/etc/ceph/ceph.conf 2024-03-16T09:07:24.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:24.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: Updating smithi159:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:07:24.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:07:24.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: Deploying cephadm binary to smithi194 2024-03-16T09:07:24.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: Updating smithi159:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:07:24.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:24.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: Added host smithi194 2024-03-16T09:07:24.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:24.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:24.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:24.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:07:24.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:24.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/997376725' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-16T09:07:24.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/997376725' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-16T09:07:24.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:07:24.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi194", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:07:24.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:24.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:07:24.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: Updating smithi194:/etc/ceph/ceph.conf 2024-03-16T09:07:24.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: Updating smithi194:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:07:24.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.491 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.491 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.491 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:07:24.491 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: Updating smithi194:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-16T09:07:24.491 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:24.491 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi159:172.21.15.159=b;smithi194:172.21.15.194=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:24.491 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: Saving service mon spec with placement smithi112:172.21.15.112=a;smithi159:172.21.15.159=b;smithi194:172.21.15.194=c;count:3 2024-03-16T09:07:24.491 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.491 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:24.491 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:24.491 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:07:24.491 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.491 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:07:24.491 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:24.492 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: Deploying daemon mon.c on smithi194 2024-03-16T09:07:24.492 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:24.492 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:24.492 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:24.492 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:07:24.492 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: mon.a calling monitor election 2024-03-16T09:07:24.492 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:24.492 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='client.? 172.21.15.194:0/2125135243' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-16T09:07:24.492 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:24.493 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:24.493 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:24.493 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: mon.c calling monitor election 2024-03-16T09:07:24.493 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:24.493 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:24.493 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:24.493 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:24.493 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-16T09:07:24.493 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: monmap epoch 2 2024-03-16T09:07:24.493 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: fsid 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:07:24.493 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: last_changed 2024-03-16T09:07:14.975453+0000 2024-03-16T09:07:24.493 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: created 2024-03-16T09:05:27.608380+0000 2024-03-16T09:07:24.493 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: min_mon_release 19 (squid) 2024-03-16T09:07:24.494 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: election_strategy: 1 2024-03-16T09:07:24.494 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-03-16T09:07:24.494 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-03-16T09:07:24.494 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: fsmap 2024-03-16T09:07:24.494 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:07:24.494 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: mgrmap e13: x(active, since 49s) 2024-03-16T09:07:24.494 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: overall HEALTH_OK 2024-03-16T09:07:24.494 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.494 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.494 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:24.494 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:07:24.494 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:24.494 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: Deploying daemon mon.b on smithi159 2024-03-16T09:07:24.494 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:24.495 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:24.495 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:24.495 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 ceph-mon[36739]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-16T09:07:24.495 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 bash[36719]: 0bfb0dba10badc8831570b04a8f0138804bb26a05aa20369b7ce5853e313d74b 2024-03-16T09:07:24.495 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:24 smithi159 systemd[1]: Started Ceph mon.b for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:07:25.511 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-16T09:07:25.511 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-16T09:07:25.753 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:07:29.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:07:29.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: mon.a calling monitor election 2024-03-16T09:07:29.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: mon.c calling monitor election 2024-03-16T09:07:29.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:29.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:29.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:29.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:29.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:29.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:29.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:29.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:29.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:29.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:29.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-16T09:07:29.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: monmap epoch 3 2024-03-16T09:07:29.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: fsid 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:07:29.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: last_changed 2024-03-16T09:07:24.224253+0000 2024-03-16T09:07:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: created 2024-03-16T09:05:27.608380+0000 2024-03-16T09:07:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: min_mon_release 19 (squid) 2024-03-16T09:07:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: election_strategy: 1 2024-03-16T09:07:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-03-16T09:07:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-03-16T09:07:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: 2: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.b 2024-03-16T09:07:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: fsmap 2024-03-16T09:07:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:07:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: mgrmap e13: x(active, since 58s) 2024-03-16T09:07:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-16T09:07:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-16T09:07:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-16T09:07:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: mon.b (rank 2) addr [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] is down (out of quorum) 2024-03-16T09:07:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:29.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:29.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:29.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:29.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:07:29.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: mon.a calling monitor election 2024-03-16T09:07:29.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: mon.c calling monitor election 2024-03-16T09:07:29.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:29.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:29.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:29.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:29.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:29.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:29.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:29.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:29.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:29.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:29.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-16T09:07:29.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: monmap epoch 3 2024-03-16T09:07:29.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: fsid 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:07:29.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: last_changed 2024-03-16T09:07:24.224253+0000 2024-03-16T09:07:29.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: created 2024-03-16T09:05:27.608380+0000 2024-03-16T09:07:29.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: min_mon_release 19 (squid) 2024-03-16T09:07:29.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: election_strategy: 1 2024-03-16T09:07:29.654 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-03-16T09:07:29.654 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-03-16T09:07:29.654 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: 2: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.b 2024-03-16T09:07:29.654 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: fsmap 2024-03-16T09:07:29.654 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:07:29.654 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: mgrmap e13: x(active, since 58s) 2024-03-16T09:07:29.654 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-16T09:07:29.654 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-16T09:07:29.654 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-16T09:07:29.654 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: mon.b (rank 2) addr [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] is down (out of quorum) 2024-03-16T09:07:29.654 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:29.654 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:29.654 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:29.655 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:30.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.324+0000 7f9c0fc09700 1 -- 172.21.15.194:0/1788592747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c08101a30 msgr2=0x7f9bf4005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:30.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.324+0000 7f9c0fc09700 1 --2- 172.21.15.194:0/1788592747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c08101a30 0x7f9bf4005610 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009bf0 tx=0x7f9c00009f00 comp rx=0 tx=0).stop 2024-03-16T09:07:30.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.324+0000 7f9c0fc09700 1 -- 172.21.15.194:0/1788592747 shutdown_connections 2024-03-16T09:07:30.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.324+0000 7f9c0fc09700 1 --2- 172.21.15.194:0/1788592747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c08101a30 0x7f9bf4005610 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:30.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.324+0000 7f9c0fc09700 1 -- 172.21.15.194:0/1788592747 >> 172.21.15.194:0/1788592747 conn(0x7f9c080fbad0 msgr2=0x7f9c080fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:30.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.324+0000 7f9c0fc09700 1 -- 172.21.15.194:0/1788592747 shutdown_connections 2024-03-16T09:07:30.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.324+0000 7f9c0fc09700 1 -- 172.21.15.194:0/1788592747 wait complete. 2024-03-16T09:07:30.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.324+0000 7f9c0fc09700 1 Processor -- start 2024-03-16T09:07:30.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0fc09700 1 -- start start 2024-03-16T09:07:30.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0fc09700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c08101a30 0x7f9c0819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:30.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0fc09700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c0819dc00 0x7f9c08197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:30.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0fc09700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c08197dc0 0x7f9c08198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:30.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0fc09700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9c08108e20 con 0x7f9c08101a30 2024-03-16T09:07:30.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0fc09700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9c08108ca0 con 0x7f9c0819dc00 2024-03-16T09:07:30.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0fc09700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9c08108fa0 con 0x7f9c08197dc0 2024-03-16T09:07:30.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0e1a6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c08197dc0 0x7f9c08198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:30.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0e1a6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c08197dc0 0x7f9c08198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:39404/0 (socket says 172.21.15.194:39404) 2024-03-16T09:07:30.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0e1a6700 1 -- 172.21.15.194:0/2923635587 learned_addr learned my addr 172.21.15.194:0/2923635587 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T09:07:30.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0d9a5700 1 --2- 172.21.15.194:0/2923635587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c08101a30 0x7f9c0819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:30.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0d1a4700 1 --2- 172.21.15.194:0/2923635587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c0819dc00 0x7f9c08197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:30.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0d1a4700 1 -- 172.21.15.194:0/2923635587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c0819dc00 msgr2=0x7f9c08197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-16T09:07:30.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0e1a6700 1 -- 172.21.15.194:0/2923635587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c0819dc00 msgr2=0x7f9c08197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:30.331 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0d1a4700 1 -- 172.21.15.194:0/2923635587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c0819dc00 msgr2=0x7f9c08197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-16T09:07:30.331 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0d1a4700 1 --2- 172.21.15.194:0/2923635587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c0819dc00 0x7f9c08197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T09:07:30.331 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0d1a4700 1 --2- 172.21.15.194:0/2923635587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c0819dc00 0x7f9c08197880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T09:07:30.331 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0e1a6700 1 --2- 172.21.15.194:0/2923635587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c0819dc00 0x7f9c08197880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:30.331 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0e1a6700 1 -- 172.21.15.194:0/2923635587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c08101a30 msgr2=0x7f9c0819d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:30.331 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0e1a6700 1 --2- 172.21.15.194:0/2923635587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c08101a30 0x7f9c0819d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:30.332 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0e1a6700 1 -- 172.21.15.194:0/2923635587 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c081a4590 con 0x7f9c08197dc0 2024-03-16T09:07:30.332 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.325+0000 7f9c0d9a5700 1 --2- 172.21.15.194:0/2923635587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c08101a30 0x7f9c0819d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:07:30.332 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.326+0000 7f9c0e1a6700 1 --2- 172.21.15.194:0/2923635587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c08197dc0 0x7f9c08198270 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400b550 tx=0x7f9c0400b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:30.332 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.326+0000 7f9bfeffd700 1 -- 172.21.15.194:0/2923635587 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04015020 con 0x7f9c08197dc0 2024-03-16T09:07:30.332 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.326+0000 7f9bfeffd700 1 -- 172.21.15.194:0/2923635587 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9c04011d30 con 0x7f9c08197dc0 2024-03-16T09:07:30.332 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.326+0000 7f9bfeffd700 1 -- 172.21.15.194:0/2923635587 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0400f2e0 con 0x7f9c08197dc0 2024-03-16T09:07:30.332 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.326+0000 7f9c0fc09700 1 -- 172.21.15.194:0/2923635587 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c081a47c0 con 0x7f9c08197dc0 2024-03-16T09:07:30.332 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.326+0000 7f9c0fc09700 1 -- 172.21.15.194:0/2923635587 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c081a4d50 con 0x7f9c08197dc0 2024-03-16T09:07:30.332 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.327+0000 7f9bfeffd700 1 -- 172.21.15.194:0/2923635587 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f9c04018020 con 0x7f9c08197dc0 2024-03-16T09:07:30.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.327+0000 7f9bfeffd700 1 --2- 172.21.15.194:0/2923635587 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9bec039450 0x7f9bec03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:30.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.327+0000 7f9bfeffd700 1 -- 172.21.15.194:0/2923635587 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9c0404b8f0 con 0x7f9c08197dc0 2024-03-16T09:07:30.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.327+0000 7f9c0fc09700 1 -- 172.21.15.194:0/2923635587 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c08063900 con 0x7f9c08197dc0 2024-03-16T09:07:30.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.327+0000 7f9c0d9a5700 1 --2- 172.21.15.194:0/2923635587 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9bec039450 0x7f9bec03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:30.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.328+0000 7f9c0d9a5700 1 --2- 172.21.15.194:0/2923635587 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9bec039450 0x7f9bec03b910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9c00007950 tx=0x7f9c00007f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:30.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.333+0000 7f9bfeffd700 1 -- 172.21.15.194:0/2923635587 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9c0401f020 con 0x7f9c08197dc0 2024-03-16T09:07:30.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:30 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:30.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:30 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:30.620 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.618+0000 7f9c0fc09700 1 -- 172.21.15.194:0/2923635587 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9c081a50a0 con 0x7f9c08197dc0 2024-03-16T09:07:30.620 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:30 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:30.620 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:30 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:30.621 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.618+0000 7f9bfeffd700 1 -- 172.21.15.194:0/2923635587 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f9c04010380 con 0x7f9c08197dc0 2024-03-16T09:07:30.621 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-03-16T09:07:30.621 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":3,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","modified":"2024-03-16T09:07:24.224253Z","created":"2024-03-16T09:05:27.608380Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:3300","nonce":0},{"type":"v1","addr":"172.21.15.194:6789","nonce":0}]},"addr":"172.21.15.194:6789/0","public_addr":"172.21.15.194:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-16T09:07:30.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.620+0000 7f9c0fc09700 1 -- 172.21.15.194:0/2923635587 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9bec039450 msgr2=0x7f9bec03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:30.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.621+0000 7f9c0fc09700 1 --2- 172.21.15.194:0/2923635587 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9bec039450 0x7f9bec03b910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9c00007950 tx=0x7f9c00007f60 comp rx=0 tx=0).stop 2024-03-16T09:07:30.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.621+0000 7f9c0fc09700 1 -- 172.21.15.194:0/2923635587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c08197dc0 msgr2=0x7f9c08198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:30.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.621+0000 7f9c0fc09700 1 --2- 172.21.15.194:0/2923635587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c08197dc0 0x7f9c08198270 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400b550 tx=0x7f9c0400b860 comp rx=0 tx=0).stop 2024-03-16T09:07:30.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.621+0000 7f9c0fc09700 1 -- 172.21.15.194:0/2923635587 shutdown_connections 2024-03-16T09:07:30.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.621+0000 7f9c0fc09700 1 --2- 172.21.15.194:0/2923635587 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9bec039450 0x7f9bec03b910 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:30.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.621+0000 7f9c0fc09700 1 --2- 172.21.15.194:0/2923635587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c08197dc0 0x7f9c08198270 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:30.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.621+0000 7f9c0fc09700 1 --2- 172.21.15.194:0/2923635587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c08101a30 0x7f9c0819d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:30.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.621+0000 7f9c0fc09700 1 --2- 172.21.15.194:0/2923635587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c0819dc00 0x7f9c08197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:30.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.621+0000 7f9c0fc09700 1 -- 172.21.15.194:0/2923635587 >> 172.21.15.194:0/2923635587 conn(0x7f9c080fbad0 msgr2=0x7f9c080fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:30.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.621+0000 7f9c0fc09700 1 -- 172.21.15.194:0/2923635587 shutdown_connections 2024-03-16T09:07:30.623 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:30.621+0000 7f9c0fc09700 1 -- 172.21.15.194:0/2923635587 wait complete. 2024-03-16T09:07:30.624 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 3 2024-03-16T09:07:31.365 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-16T09:07:31.366 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph config generate-minimal-conf 2024-03-16T09:07:31.621 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:07:31.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:07:31.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: mon.a calling monitor election 2024-03-16T09:07:31.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: mon.c calling monitor election 2024-03-16T09:07:31.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:31.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:07:31.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:31.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:31.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:31.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:31.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:31.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:31.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:31.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:31.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-16T09:07:31.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: monmap epoch 3 2024-03-16T09:07:31.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: fsid 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:07:31.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: last_changed 2024-03-16T09:07:24.224253+0000 2024-03-16T09:07:31.738 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: created 2024-03-16T09:05:27.608380+0000 2024-03-16T09:07:31.738 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: min_mon_release 19 (squid) 2024-03-16T09:07:31.738 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: election_strategy: 1 2024-03-16T09:07:31.738 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-03-16T09:07:31.738 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-03-16T09:07:31.738 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: 2: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.b 2024-03-16T09:07:31.738 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: fsmap 2024-03-16T09:07:31.738 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:07:31.738 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: mgrmap e13: x(active, since 58s) 2024-03-16T09:07:31.738 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-16T09:07:31.738 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-16T09:07:31.738 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-16T09:07:31.738 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: mon.b (rank 2) addr [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] is down (out of quorum) 2024-03-16T09:07:31.738 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:31.738 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:31.739 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:31.739 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:31.739 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:31.739 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:32.063 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: mon.b calling monitor election 2024-03-16T09:07:32.063 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: mon.b calling monitor election 2024-03-16T09:07:32.063 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: mon.c calling monitor election 2024-03-16T09:07:32.063 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: mon.a calling monitor election 2024-03-16T09:07:32.063 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-16T09:07:32.064 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: monmap epoch 3 2024-03-16T09:07:32.064 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: fsid 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:07:32.064 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: last_changed 2024-03-16T09:07:24.224253+0000 2024-03-16T09:07:32.064 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: created 2024-03-16T09:05:27.608380+0000 2024-03-16T09:07:32.064 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: min_mon_release 19 (squid) 2024-03-16T09:07:32.064 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: election_strategy: 1 2024-03-16T09:07:32.064 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-03-16T09:07:32.064 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-03-16T09:07:32.064 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: 2: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.b 2024-03-16T09:07:32.064 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: fsmap 2024-03-16T09:07:32.064 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:07:32.064 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: mgrmap e13: x(active, since 60s) 2024-03-16T09:07:32.064 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-16T09:07:32.064 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: Cluster is now healthy 2024-03-16T09:07:32.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: overall HEALTH_OK 2024-03-16T09:07:32.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.065 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:07:32.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:07:32.066 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:32.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: mon.b calling monitor election 2024-03-16T09:07:32.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: mon.b calling monitor election 2024-03-16T09:07:32.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: mon.c calling monitor election 2024-03-16T09:07:32.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: mon.a calling monitor election 2024-03-16T09:07:32.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-16T09:07:32.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: monmap epoch 3 2024-03-16T09:07:32.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: fsid 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:07:32.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: last_changed 2024-03-16T09:07:24.224253+0000 2024-03-16T09:07:32.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: created 2024-03-16T09:05:27.608380+0000 2024-03-16T09:07:32.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: min_mon_release 19 (squid) 2024-03-16T09:07:32.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: election_strategy: 1 2024-03-16T09:07:32.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-03-16T09:07:32.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-03-16T09:07:32.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: 2: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.b 2024-03-16T09:07:32.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: fsmap 2024-03-16T09:07:32.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:07:32.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: mgrmap e13: x(active, since 60s) 2024-03-16T09:07:32.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-16T09:07:32.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: Cluster is now healthy 2024-03-16T09:07:32.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: overall HEALTH_OK 2024-03-16T09:07:32.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.154 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.154 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.154 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.154 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.154 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.154 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.154 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:07:32.154 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:07:32.154 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:32.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: mon.b calling monitor election 2024-03-16T09:07:32.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: mon.b calling monitor election 2024-03-16T09:07:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: mon.c calling monitor election 2024-03-16T09:07:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: mon.a calling monitor election 2024-03-16T09:07:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-16T09:07:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: monmap epoch 3 2024-03-16T09:07:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: fsid 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:07:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: last_changed 2024-03-16T09:07:24.224253+0000 2024-03-16T09:07:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: created 2024-03-16T09:05:27.608380+0000 2024-03-16T09:07:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: min_mon_release 19 (squid) 2024-03-16T09:07:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: election_strategy: 1 2024-03-16T09:07:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-03-16T09:07:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-03-16T09:07:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: 2: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.b 2024-03-16T09:07:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: fsmap 2024-03-16T09:07:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: osdmap e4: 0 total, 0 up, 0 in 2024-03-16T09:07:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: mgrmap e13: x(active, since 60s) 2024-03-16T09:07:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-16T09:07:32.238 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: Cluster is now healthy 2024-03-16T09:07:32.238 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: overall HEALTH_OK 2024-03-16T09:07:32.238 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.238 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.238 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.238 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.238 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.238 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.238 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.238 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.238 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.238 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.238 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:32.238 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:07:32.238 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:07:32.239 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:33.052 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:32 smithi112 ceph-mon[32307]: Updating smithi112:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:07:33.052 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:32 smithi112 ceph-mon[32307]: Updating smithi159:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:07:33.052 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:32 smithi112 ceph-mon[32307]: Updating smithi194:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:07:33.053 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:32 smithi112 ceph-mon[32307]: Reconfiguring mon.a (unknown last config time)... 2024-03-16T09:07:33.053 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:32 smithi112 ceph-mon[32307]: Reconfiguring daemon mon.a on smithi112 2024-03-16T09:07:33.053 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:32 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:33.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:32 smithi194 ceph-mon[36729]: Updating smithi112:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:07:33.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:32 smithi194 ceph-mon[36729]: Updating smithi159:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:07:33.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:32 smithi194 ceph-mon[36729]: Updating smithi194:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:07:33.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:32 smithi194 ceph-mon[36729]: Reconfiguring mon.a (unknown last config time)... 2024-03-16T09:07:33.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:32 smithi194 ceph-mon[36729]: Reconfiguring daemon mon.a on smithi112 2024-03-16T09:07:33.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:32 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:33.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:32 smithi159 ceph-mon[36739]: Updating smithi112:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:07:33.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:32 smithi159 ceph-mon[36739]: Updating smithi159:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:07:33.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:32 smithi159 ceph-mon[36739]: Updating smithi194:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-16T09:07:33.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:32 smithi159 ceph-mon[36739]: Reconfiguring mon.a (unknown last config time)... 2024-03-16T09:07:33.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:32 smithi159 ceph-mon[36739]: Reconfiguring daemon mon.a on smithi112 2024-03-16T09:07:33.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:32 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:07:33.907 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:33 smithi112 ceph-mon[32307]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:33.908 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:33.908 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:33.908 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:07:33.908 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:07:33.908 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:34.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.105+0000 7ff08775f700 1 -- 172.21.15.112:0/1357413402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff080102320 msgr2=0x7ff0801027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:34.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.105+0000 7ff08775f700 1 --2- 172.21.15.112:0/1357413402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff080102320 0x7ff0801027a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009990 tx=0x7ff07c009ca0 comp rx=0 tx=0).stop 2024-03-16T09:07:34.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.105+0000 7ff08775f700 1 -- 172.21.15.112:0/1357413402 shutdown_connections 2024-03-16T09:07:34.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.105+0000 7ff08775f700 1 --2- 172.21.15.112:0/1357413402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff080102ce0 0x7ff08010f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:34.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.105+0000 7ff08775f700 1 --2- 172.21.15.112:0/1357413402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0801083d0 0x7ff0801087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:34.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.105+0000 7ff08775f700 1 --2- 172.21.15.112:0/1357413402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff080102320 0x7ff0801027a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:34.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.105+0000 7ff08775f700 1 -- 172.21.15.112:0/1357413402 >> 172.21.15.112:0/1357413402 conn(0x7ff080079400 msgr2=0x7ff080079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:34.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.105+0000 7ff08775f700 1 -- 172.21.15.112:0/1357413402 shutdown_connections 2024-03-16T09:07:34.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.105+0000 7ff08775f700 1 -- 172.21.15.112:0/1357413402 wait complete. 2024-03-16T09:07:34.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.106+0000 7ff08775f700 1 Processor -- start 2024-03-16T09:07:34.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.106+0000 7ff08775f700 1 -- start start 2024-03-16T09:07:34.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.106+0000 7ff08775f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff080102320 0x7ff0800745d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:34.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.106+0000 7ff08775f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff080102ce0 0x7ff080074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:34.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.106+0000 7ff08775f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0801083d0 0x7ff080075050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:34.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.106+0000 7ff08775f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0801127a0 con 0x7ff0801083d0 2024-03-16T09:07:34.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.106+0000 7ff08775f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff080112620 con 0x7ff080102ce0 2024-03-16T09:07:34.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.106+0000 7ff08775f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff080112920 con 0x7ff080102320 2024-03-16T09:07:34.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.106+0000 7ff085cfc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0801083d0 0x7ff080075050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:34.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.106+0000 7ff085cfc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0801083d0 0x7ff080075050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40304/0 (socket says 172.21.15.112:40304) 2024-03-16T09:07:34.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.106+0000 7ff085cfc700 1 -- 172.21.15.112:0/937235376 learned_addr learned my addr 172.21.15.112:0/937235376 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:07:34.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.106+0000 7ff084cfa700 1 --2- 172.21.15.112:0/937235376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff080102ce0 0x7ff080074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:34.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.106+0000 7ff0854fb700 1 --2- 172.21.15.112:0/937235376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff080102320 0x7ff0800745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:34.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.106+0000 7ff085cfc700 1 -- 172.21.15.112:0/937235376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff080102320 msgr2=0x7ff0800745d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:34.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.106+0000 7ff085cfc700 1 --2- 172.21.15.112:0/937235376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff080102320 0x7ff0800745d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:34.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.106+0000 7ff085cfc700 1 -- 172.21.15.112:0/937235376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff080102ce0 msgr2=0x7ff080074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:34.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.106+0000 7ff085cfc700 1 --2- 172.21.15.112:0/937235376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff080102ce0 0x7ff080074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:34.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.106+0000 7ff085cfc700 1 -- 172.21.15.112:0/937235376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0801a7180 con 0x7ff0801083d0 2024-03-16T09:07:34.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.107+0000 7ff085cfc700 1 --2- 172.21.15.112:0/937235376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0801083d0 0x7ff080075050 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff078007ee0 tx=0x7ff07800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:34.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.107+0000 7ff0767fc700 1 -- 172.21.15.112:0/937235376 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07800ddd0 con 0x7ff0801083d0 2024-03-16T09:07:34.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.107+0000 7ff0767fc700 1 -- 172.21.15.112:0/937235376 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff078004020 con 0x7ff0801083d0 2024-03-16T09:07:34.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.107+0000 7ff08775f700 1 -- 172.21.15.112:0/937235376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0801a7470 con 0x7ff0801083d0 2024-03-16T09:07:34.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.107+0000 7ff0767fc700 1 -- 172.21.15.112:0/937235376 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078015660 con 0x7ff0801083d0 2024-03-16T09:07:34.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.107+0000 7ff08775f700 1 -- 172.21.15.112:0/937235376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff080198990 con 0x7ff0801083d0 2024-03-16T09:07:34.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.108+0000 7ff0767fc700 1 -- 172.21.15.112:0/937235376 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7ff0780041d0 con 0x7ff0801083d0 2024-03-16T09:07:34.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.108+0000 7ff08775f700 1 -- 172.21.15.112:0/937235376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff080103a10 con 0x7ff0801083d0 2024-03-16T09:07:34.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.108+0000 7ff0767fc700 1 --2- 172.21.15.112:0/937235376 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff06c039200 0x7ff06c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:34.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.109+0000 7ff0767fc700 1 -- 172.21.15.112:0/937235376 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff07804bb50 con 0x7ff0801083d0 2024-03-16T09:07:34.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.109+0000 7ff0854fb700 1 --2- 172.21.15.112:0/937235376 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff06c039200 0x7ff06c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:34.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.109+0000 7ff0854fb700 1 --2- 172.21.15.112:0/937235376 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff06c039200 0x7ff06c03b6c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff070006fd0 tx=0x7ff070008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:34.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.113+0000 7ff0767fc700 1 -- 172.21.15.112:0/937235376 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff078010070 con 0x7ff0801083d0 2024-03-16T09:07:34.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:33 smithi194 ceph-mon[36729]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:34.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:34.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:34.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:07:34.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:07:34.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:34.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:33 smithi159 ceph-mon[36739]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:34.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:34.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:34.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:07:34.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:07:34.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:34.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.308+0000 7ff08775f700 1 -- 172.21.15.112:0/937235376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff08002db70 con 0x7ff0801083d0 2024-03-16T09:07:34.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.308+0000 7ff0767fc700 1 -- 172.21.15.112:0/937235376 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7ff07800e140 con 0x7ff0801083d0 2024-03-16T09:07:34.310 INFO:teuthology.orchestra.run.smithi112.stdout:# minimal ceph.conf for 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:07:34.310 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-03-16T09:07:34.310 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:07:34.310 INFO:teuthology.orchestra.run.smithi112.stdout: mon_host = [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] 2024-03-16T09:07:34.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.310+0000 7ff08775f700 1 -- 172.21.15.112:0/937235376 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff06c039200 msgr2=0x7ff06c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:34.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.310+0000 7ff08775f700 1 --2- 172.21.15.112:0/937235376 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff06c039200 0x7ff06c03b6c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff070006fd0 tx=0x7ff070008040 comp rx=0 tx=0).stop 2024-03-16T09:07:34.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.310+0000 7ff08775f700 1 -- 172.21.15.112:0/937235376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0801083d0 msgr2=0x7ff080075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:34.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.310+0000 7ff08775f700 1 --2- 172.21.15.112:0/937235376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0801083d0 0x7ff080075050 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff078007ee0 tx=0x7ff07800d3b0 comp rx=0 tx=0).stop 2024-03-16T09:07:34.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.310+0000 7ff08775f700 1 -- 172.21.15.112:0/937235376 shutdown_connections 2024-03-16T09:07:34.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.310+0000 7ff08775f700 1 --2- 172.21.15.112:0/937235376 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff06c039200 0x7ff06c03b6c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:34.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.310+0000 7ff08775f700 1 --2- 172.21.15.112:0/937235376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0801083d0 0x7ff080075050 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:34.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.310+0000 7ff08775f700 1 --2- 172.21.15.112:0/937235376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff080102320 0x7ff0800745d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:34.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.310+0000 7ff08775f700 1 --2- 172.21.15.112:0/937235376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff080102ce0 0x7ff080074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:34.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.310+0000 7ff08775f700 1 -- 172.21.15.112:0/937235376 >> 172.21.15.112:0/937235376 conn(0x7ff080079400 msgr2=0x7ff080100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:34.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.310+0000 7ff08775f700 1 -- 172.21.15.112:0/937235376 shutdown_connections 2024-03-16T09:07:34.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:34.310+0000 7ff08775f700 1 -- 172.21.15.112:0/937235376 wait complete. 2024-03-16T09:07:35.015 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-16T09:07:35.016 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T09:07:35.016 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.conf 2024-03-16T09:07:35.046 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T09:07:35.046 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:07:35.067 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:34 smithi159 ceph-mon[36739]: Reconfiguring mon.b (monmap changed)... 2024-03-16T09:07:35.067 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:34 smithi159 ceph-mon[36739]: Reconfiguring daemon mon.b on smithi159 2024-03-16T09:07:35.067 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:34 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/937235376' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:35.112 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-16T09:07:35.112 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.conf 2024-03-16T09:07:35.142 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-16T09:07:35.142 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:07:35.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:34 smithi194 ceph-mon[36729]: Reconfiguring mon.b (monmap changed)... 2024-03-16T09:07:35.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:34 smithi194 ceph-mon[36729]: Reconfiguring daemon mon.b on smithi159 2024-03-16T09:07:35.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:34 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/937235376' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:35.209 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:07:35.209 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/ceph/ceph.conf 2024-03-16T09:07:35.243 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:07:35.243 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-16T09:07:35.313 INFO:tasks.cephadm:Adding mgr.x on smithi112 2024-03-16T09:07:35.313 INFO:tasks.cephadm:Adding mgr.y on smithi159 2024-03-16T09:07:35.314 INFO:tasks.cephadm:Adding mgr.z on smithi194 2024-03-16T09:07:35.314 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch apply mgr '3;smithi112=x;smithi159=y;smithi194=z' 2024-03-16T09:07:35.341 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:34 smithi112 ceph-mon[32307]: Reconfiguring mon.b (monmap changed)... 2024-03-16T09:07:35.341 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:34 smithi112 ceph-mon[32307]: Reconfiguring daemon mon.b on smithi159 2024-03-16T09:07:35.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:34 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/937235376' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:35.587 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:07:36.027 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:35 smithi194 ceph-mon[36729]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:36.027 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:35 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:36.027 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:35 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:36.028 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:35 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:07:36.028 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:35 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:07:36.028 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:35 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:36.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:35 smithi159 ceph-mon[36739]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:36.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:35 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:36.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:35 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:36.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:35 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:07:36.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:35 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:07:36.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:35 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:35 smithi112 ceph-mon[32307]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:35 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:35 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:35 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-16T09:07:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:35 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-16T09:07:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:35 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:37.015 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:36 smithi194 ceph-mon[36729]: Reconfiguring mon.c (monmap changed)... 2024-03-16T09:07:37.015 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:36 smithi194 ceph-mon[36729]: Reconfiguring daemon mon.c on smithi194 2024-03-16T09:07:37.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:36 smithi159 ceph-mon[36739]: Reconfiguring mon.c (monmap changed)... 2024-03-16T09:07:37.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:36 smithi159 ceph-mon[36739]: Reconfiguring daemon mon.c on smithi194 2024-03-16T09:07:37.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:36 smithi112 ceph-mon[32307]: Reconfiguring mon.c (monmap changed)... 2024-03-16T09:07:37.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:36 smithi112 ceph-mon[32307]: Reconfiguring daemon mon.c on smithi194 2024-03-16T09:07:37.701 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.696+0000 7f4167762700 1 -- 172.21.15.194:0/3289425631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4160100d30 msgr2=0x7f4160101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:37.701 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.696+0000 7f4167762700 1 --2- 172.21.15.194:0/3289425631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4160100d30 0x7f4160101150 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f415c009910 tx=0x7f415c009c20 comp rx=0 tx=0).stop 2024-03-16T09:07:37.701 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.696+0000 7f4167762700 1 -- 172.21.15.194:0/3289425631 shutdown_connections 2024-03-16T09:07:37.701 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.696+0000 7f4167762700 1 --2- 172.21.15.194:0/3289425631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4160100d30 0x7f4160101150 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:37.701 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.696+0000 7f4167762700 1 -- 172.21.15.194:0/3289425631 >> 172.21.15.194:0/3289425631 conn(0x7f41600fc2d0 msgr2=0x7f41600fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:37.701 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.696+0000 7f4167762700 1 -- 172.21.15.194:0/3289425631 shutdown_connections 2024-03-16T09:07:37.701 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.696+0000 7f4167762700 1 -- 172.21.15.194:0/3289425631 wait complete. 2024-03-16T09:07:37.701 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.696+0000 7f4167762700 1 Processor -- start 2024-03-16T09:07:37.701 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.697+0000 7f4167762700 1 -- start start 2024-03-16T09:07:37.701 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.697+0000 7f4167762700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4160100d30 0x7f4160196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:37.702 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.697+0000 7f4167762700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4160196b70 0x7f416019bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:37.702 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.697+0000 7f4167762700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4160196ff0 0x7f416019e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:37.702 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.697+0000 7f4167762700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4160106eb0 con 0x7f4160196b70 2024-03-16T09:07:37.702 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.697+0000 7f4167762700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4160106d30 con 0x7f4160196ff0 2024-03-16T09:07:37.702 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.697+0000 7f4167762700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4160107030 con 0x7f4160100d30 2024-03-16T09:07:37.702 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.697+0000 7f41654fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4160100d30 0x7f4160196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:37.702 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.697+0000 7f41654fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4160100d30 0x7f4160196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:37244/0 (socket says 172.21.15.194:37244) 2024-03-16T09:07:37.702 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.697+0000 7f41654fe700 1 -- 172.21.15.194:0/1659960144 learned_addr learned my addr 172.21.15.194:0/1659960144 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T09:07:37.702 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.697+0000 7f4164cfd700 1 --2- 172.21.15.194:0/1659960144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4160196b70 0x7f416019bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:37.702 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.697+0000 7f4165cff700 1 --2- 172.21.15.194:0/1659960144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4160196ff0 0x7f416019e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:37.702 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.697+0000 7f41654fe700 1 -- 172.21.15.194:0/1659960144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4160196ff0 msgr2=0x7f416019e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:37.702 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.697+0000 7f41654fe700 1 --2- 172.21.15.194:0/1659960144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4160196ff0 0x7f416019e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:37.702 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.697+0000 7f41654fe700 1 -- 172.21.15.194:0/1659960144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4160196b70 msgr2=0x7f416019bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:37.702 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.697+0000 7f41654fe700 1 --2- 172.21.15.194:0/1659960144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4160196b70 0x7f416019bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:37.703 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.697+0000 7f41654fe700 1 -- 172.21.15.194:0/1659960144 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416019ea40 con 0x7f4160100d30 2024-03-16T09:07:37.703 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.697+0000 7f41654fe700 1 --2- 172.21.15.194:0/1659960144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4160100d30 0x7f4160196630 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f415c000c00 tx=0x7f415c00b9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:37.703 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.698+0000 7f41567fc700 1 -- 172.21.15.194:0/1659960144 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c026070 con 0x7f4160100d30 2024-03-16T09:07:37.703 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.698+0000 7f41567fc700 1 -- 172.21.15.194:0/1659960144 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f415c00be00 con 0x7f4160100d30 2024-03-16T09:07:37.703 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.698+0000 7f4167762700 1 -- 172.21.15.194:0/1659960144 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416019ecd0 con 0x7f4160100d30 2024-03-16T09:07:37.703 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.698+0000 7f41567fc700 1 -- 172.21.15.194:0/1659960144 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415c00fc50 con 0x7f4160100d30 2024-03-16T09:07:37.703 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.698+0000 7f4167762700 1 -- 172.21.15.194:0/1659960144 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416019f1b0 con 0x7f4160100d30 2024-03-16T09:07:37.703 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.698+0000 7f41567fc700 1 -- 172.21.15.194:0/1659960144 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7f415c029070 con 0x7f4160100d30 2024-03-16T09:07:37.703 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.699+0000 7f4167762700 1 -- 172.21.15.194:0/1659960144 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4160063900 con 0x7f4160100d30 2024-03-16T09:07:37.703 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.699+0000 7f41567fc700 1 --2- 172.21.15.194:0/1659960144 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f414c039450 0x7f414c03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:37.703 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.699+0000 7f41567fc700 1 -- 172.21.15.194:0/1659960144 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f415c04cde0 con 0x7f4160100d30 2024-03-16T09:07:37.703 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.699+0000 7f4164cfd700 1 --2- 172.21.15.194:0/1659960144 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f414c039450 0x7f414c03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:37.704 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.699+0000 7f4164cfd700 1 --2- 172.21.15.194:0/1659960144 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f414c039450 0x7f414c03b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4150009ad0 tx=0x7f4150009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:37.706 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.704+0000 7f41567fc700 1 -- 172.21.15.194:0/1659960144 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f415c052020 con 0x7f4160100d30 2024-03-16T09:07:37.916 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.914+0000 7f4167762700 1 -- 172.21.15.194:0/1659960144 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi159=y;smithi194=z", "target": ["mon-mgr", ""]}) -- 0x7f4160062710 con 0x7f414c039450 2024-03-16T09:07:37.924 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.923+0000 7f41567fc700 1 -- 172.21.15.194:0/1659960144 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4160062710 con 0x7f414c039450 2024-03-16T09:07:37.925 INFO:teuthology.orchestra.run.smithi194.stdout:Scheduled mgr update... 2024-03-16T09:07:37.927 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.925+0000 7f4167762700 1 -- 172.21.15.194:0/1659960144 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f414c039450 msgr2=0x7f414c03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:37.927 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.925+0000 7f4167762700 1 --2- 172.21.15.194:0/1659960144 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f414c039450 0x7f414c03b910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4150009ad0 tx=0x7f4150009210 comp rx=0 tx=0).stop 2024-03-16T09:07:37.927 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.925+0000 7f4167762700 1 -- 172.21.15.194:0/1659960144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4160100d30 msgr2=0x7f4160196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:37.927 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.925+0000 7f4167762700 1 --2- 172.21.15.194:0/1659960144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4160100d30 0x7f4160196630 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f415c000c00 tx=0x7f415c00b9c0 comp rx=0 tx=0).stop 2024-03-16T09:07:37.927 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.925+0000 7f4167762700 1 -- 172.21.15.194:0/1659960144 shutdown_connections 2024-03-16T09:07:37.927 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.925+0000 7f4167762700 1 --2- 172.21.15.194:0/1659960144 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f414c039450 0x7f414c03b910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:37.927 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.925+0000 7f4167762700 1 --2- 172.21.15.194:0/1659960144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4160196ff0 0x7f416019e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:37.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.925+0000 7f4167762700 1 --2- 172.21.15.194:0/1659960144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4160100d30 0x7f4160196630 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:37.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.925+0000 7f4167762700 1 --2- 172.21.15.194:0/1659960144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4160196b70 0x7f416019bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:37.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.925+0000 7f4167762700 1 -- 172.21.15.194:0/1659960144 >> 172.21.15.194:0/1659960144 conn(0x7f41600fc2d0 msgr2=0x7f41600fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:37.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.925+0000 7f4167762700 1 -- 172.21.15.194:0/1659960144 shutdown_connections 2024-03-16T09:07:37.928 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:07:37.925+0000 7f4167762700 1 -- 172.21.15.194:0/1659960144 wait complete. 2024-03-16T09:07:37.937 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:37 smithi194 ceph-mon[36729]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:38.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:37 smithi159 ceph-mon[36739]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:38.341 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:37 smithi112 ceph-mon[32307]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:38.700 DEBUG:teuthology.orchestra.run.smithi159:mgr.y> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mgr.y.service 2024-03-16T09:07:38.703 DEBUG:teuthology.orchestra.run.smithi194:mgr.z> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mgr.z.service 2024-03-16T09:07:38.705 INFO:tasks.cephadm:Deploying OSDs... 2024-03-16T09:07:38.705 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T09:07:38.705 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2024-03-16T09:07:38.721 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-16T09:07:38.721 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2024-03-16T09:07:38.775 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-16T09:07:38.775 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:07:38.775 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-16T09:07:38.775 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:07:38.775 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:07:38.775 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-03-16 09:06:43.812338684 +0000 2024-03-16T09:07:38.775 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-03-16 09:00:52.360579481 +0000 2024-03-16T09:07:38.775 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-03-16 09:00:52.360579481 +0000 2024-03-16T09:07:38.775 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-03-16 09:00:52.360579481 +0000 2024-03-16T09:07:38.776 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-16T09:07:38.841 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-03-16T09:07:38.841 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-03-16T09:07:38.841 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000197436 s, 2.6 MB/s 2024-03-16T09:07:38.842 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-16T09:07:38.898 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2024-03-16T09:07:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-16T09:07:38.953 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:07:38.953 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-16T09:07:38.953 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:07:38.953 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:07:38.953 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-03-16 09:06:43.845337821 +0000 2024-03-16T09:07:38.953 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-03-16 09:00:52.700570834 +0000 2024-03-16T09:07:38.954 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-03-16 09:00:52.700570834 +0000 2024-03-16T09:07:38.954 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-03-16 09:00:52.700570834 +0000 2024-03-16T09:07:38.954 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-16T09:07:38.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:38 smithi159 ceph-mon[36739]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi159=y;smithi194=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:38.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:38 smithi159 ceph-mon[36739]: Saving service mgr spec with placement smithi112=x;smithi159=y;smithi194=z;count:3 2024-03-16T09:07:38.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:38 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:38.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:38 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:38.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:38 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:38.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:38 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:38.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:38 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:38.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:38 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:07:38.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:38 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:38.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:38 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:07:38.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:38 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T09:07:38.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:38 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:07:38.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:38 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-03-16T09:07:39.018 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-03-16T09:07:39.019 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000178075 s, 2.9 MB/s 2024-03-16T09:07:39.020 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-16T09:07:39.076 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2024-03-16T09:07:39.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:38 smithi112 ceph-mon[32307]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi159=y;smithi194=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:39.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:38 smithi112 ceph-mon[32307]: Saving service mgr spec with placement smithi112=x;smithi159=y;smithi194=z;count:3 2024-03-16T09:07:39.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:38 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:39.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:38 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:39.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:38 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:39.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:38 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:39.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:38 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:39.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:38 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:07:39.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:38 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:39.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:38 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:07:39.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:38 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T09:07:39.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:38 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:07:39.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:38 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-16T09:07:39.106 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:07:39.106 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-16T09:07:39.106 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:07:39.106 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:07:39.106 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-03-16 09:06:43.877336984 +0000 2024-03-16T09:07:39.106 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-03-16 09:00:53.042562136 +0000 2024-03-16T09:07:39.106 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-03-16 09:00:53.042562136 +0000 2024-03-16T09:07:39.107 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-03-16 09:00:53.042562136 +0000 2024-03-16T09:07:39.107 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-16T09:07:39.171 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-03-16T09:07:39.171 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-03-16T09:07:39.171 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000177563 s, 2.9 MB/s 2024-03-16T09:07:39.173 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-16T09:07:39.229 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2024-03-16T09:07:39.284 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-16T09:07:39.284 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:07:39.284 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-03-16T09:07:39.284 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:07:39.284 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:07:39.284 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-03-16 09:06:43.910336121 +0000 2024-03-16T09:07:39.285 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-03-16 09:00:53.366553896 +0000 2024-03-16T09:07:39.285 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-03-16 09:00:53.366553896 +0000 2024-03-16T09:07:39.285 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-03-16 09:00:53.366553896 +0000 2024-03-16T09:07:39.285 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-16T09:07:39.349 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-03-16T09:07:39.349 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-03-16T09:07:39.350 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000216764 s, 2.4 MB/s 2024-03-16T09:07:39.350 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:38 smithi194 ceph-mon[36729]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi159=y;smithi194=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:39.350 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:38 smithi194 ceph-mon[36729]: Saving service mgr spec with placement smithi112=x;smithi159=y;smithi194=z;count:3 2024-03-16T09:07:39.350 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:38 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:39.351 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:38 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:39.351 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:38 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:39.351 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:38 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:39.351 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:38 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:39.351 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:38 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:07:39.351 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:38 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:39.351 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:38 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:07:39.351 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:38 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T09:07:39.351 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:38 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:07:39.351 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:38 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:39.352 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-16T09:07:39.408 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-16T09:07:39.408 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/scratch_devs of=/dev/stdout 2024-03-16T09:07:39.427 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-16T09:07:39.427 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_1 2024-03-16T09:07:39.485 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-16T09:07:39.486 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:07:39.486 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-03-16T09:07:39.486 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:07:39.486 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:07:39.486 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-03-16 09:07:23.678865529 +0000 2024-03-16T09:07:39.486 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-03-16 09:01:08.299324094 +0000 2024-03-16T09:07:39.486 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-03-16 09:01:08.299324094 +0000 2024-03-16T09:07:39.486 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-03-16 09:01:08.299324094 +0000 2024-03-16T09:07:39.486 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-16T09:07:39.556 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-03-16T09:07:39.556 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-03-16T09:07:39.556 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000294499 s, 1.7 MB/s 2024-03-16T09:07:39.558 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-16T09:07:39.615 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_2 2024-03-16T09:07:39.672 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-16T09:07:39.672 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:07:39.672 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-03-16T09:07:39.672 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:07:39.672 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:07:39.672 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-03-16 09:07:23.678865529 +0000 2024-03-16T09:07:39.672 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-03-16 09:01:08.639323899 +0000 2024-03-16T09:07:39.672 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-03-16 09:01:08.639323899 +0000 2024-03-16T09:07:39.672 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-03-16 09:01:08.639323899 +0000 2024-03-16T09:07:39.673 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-16T09:07:39.739 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-03-16T09:07:39.739 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-03-16T09:07:39.740 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000230736 s, 2.2 MB/s 2024-03-16T09:07:39.741 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-16T09:07:39.798 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_3 2024-03-16T09:07:39.854 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-16T09:07:39.855 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:07:39.855 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-03-16T09:07:39.855 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:07:39.855 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:07:39.855 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-03-16 09:07:23.679865528 +0000 2024-03-16T09:07:39.855 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-03-16 09:01:08.992323695 +0000 2024-03-16T09:07:39.855 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-03-16 09:01:08.992323695 +0000 2024-03-16T09:07:39.855 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-03-16 09:01:08.992323695 +0000 2024-03-16T09:07:39.855 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-16T09:07:39.923 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-03-16T09:07:39.923 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-03-16T09:07:39.923 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000253732 s, 2.0 MB/s 2024-03-16T09:07:39.924 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-16T09:07:39.980 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_4 2024-03-16T09:07:40.036 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-16T09:07:40.036 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:07:40.036 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-03-16T09:07:40.036 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:07:40.036 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:07:40.036 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-03-16 09:07:23.679865528 +0000 2024-03-16T09:07:40.036 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-03-16 09:01:09.332323499 +0000 2024-03-16T09:07:40.036 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-03-16 09:01:09.332323499 +0000 2024-03-16T09:07:40.036 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-03-16 09:01:09.332323499 +0000 2024-03-16T09:07:40.036 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-16T09:07:40.101 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-03-16T09:07:40.101 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-03-16T09:07:40.101 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000198737 s, 2.6 MB/s 2024-03-16T09:07:40.102 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-16T09:07:40.158 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-03-16T09:07:40.158 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/scratch_devs of=/dev/stdout 2024-03-16T09:07:40.174 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-16T09:07:40.174 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_1 2024-03-16T09:07:40.203 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:39 smithi194 ceph-mon[36729]: Deploying daemon mgr.z on smithi194 2024-03-16T09:07:40.203 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:39 smithi194 ceph-mon[36729]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:40.218 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-16T09:07:40.218 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:07:40.218 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-03-16T09:07:40.218 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:07:40.218 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:07:40.218 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-03-16 09:07:14.003963090 +0000 2024-03-16T09:07:40.218 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-03-16 09:01:22.180221828 +0000 2024-03-16T09:07:40.218 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-03-16 09:01:22.180221828 +0000 2024-03-16T09:07:40.218 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-03-16 09:01:22.180221828 +0000 2024-03-16T09:07:40.219 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-16T09:07:40.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:39 smithi159 ceph-mon[36739]: Deploying daemon mgr.z on smithi194 2024-03-16T09:07:40.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:39 smithi159 ceph-mon[36739]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:40.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:39 smithi112 ceph-mon[32307]: Deploying daemon mgr.z on smithi194 2024-03-16T09:07:40.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:39 smithi112 ceph-mon[32307]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:40.375 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-03-16T09:07:40.375 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-03-16T09:07:40.375 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000268202 s, 1.9 MB/s 2024-03-16T09:07:40.377 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-16T09:07:40.392 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_2 2024-03-16T09:07:40.448 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-16T09:07:40.448 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:07:40.448 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-03-16T09:07:40.448 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:07:40.449 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:07:40.449 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-03-16 09:07:40.314408775 +0000 2024-03-16T09:07:40.449 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-03-16 09:01:22.510214803 +0000 2024-03-16T09:07:40.449 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-03-16 09:01:22.510214803 +0000 2024-03-16T09:07:40.449 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-03-16 09:01:22.510214803 +0000 2024-03-16T09:07:40.449 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-16T09:07:40.564 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-03-16T09:07:40.564 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-03-16T09:07:40.564 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000183032 s, 2.8 MB/s 2024-03-16T09:07:40.565 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-16T09:07:40.581 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_3 2024-03-16T09:07:40.637 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-16T09:07:40.637 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:07:40.637 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-03-16T09:07:40.637 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:07:40.637 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:07:40.637 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-03-16 09:07:40.504404772 +0000 2024-03-16T09:07:40.637 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-03-16 09:01:22.851207544 +0000 2024-03-16T09:07:40.637 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-03-16 09:01:22.851207544 +0000 2024-03-16T09:07:40.637 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-03-16 09:01:22.851207544 +0000 2024-03-16T09:07:40.637 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-16T09:07:40.704 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-03-16T09:07:40.705 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-03-16T09:07:40.705 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000206789 s, 2.5 MB/s 2024-03-16T09:07:40.706 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-16T09:07:40.762 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_4 2024-03-16T09:07:40.816 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-16T09:07:40.816 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T09:07:40.816 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-03-16T09:07:40.816 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:07:40.816 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T09:07:40.816 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-03-16 09:07:40.504404772 +0000 2024-03-16T09:07:40.816 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-03-16 09:01:23.191200306 +0000 2024-03-16T09:07:40.816 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-03-16 09:01:23.191200306 +0000 2024-03-16T09:07:40.816 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-03-16 09:01:23.191200306 +0000 2024-03-16T09:07:40.816 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-16T09:07:40.882 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-03-16T09:07:40.882 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-03-16T09:07:40.882 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000285992 s, 1.8 MB/s 2024-03-16T09:07:40.883 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-16T09:07:40.938 INFO:tasks.cephadm:Deploying osd.0 on smithi112 with /dev/vg_nvme/lv_4... 2024-03-16T09:07:40.938 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-16T09:07:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:07:42.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:41 smithi194 ceph-mon[36729]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:42.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:41 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:42.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:41 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:42.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:41 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:42.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:41 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:07:42.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:41 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T09:07:42.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:41 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:07:42.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:41 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:42.166 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:41 smithi159 ceph-mon[36739]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:42.166 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:41 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:42.166 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:41 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:42.166 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:41 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:42.166 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:41 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:07:42.166 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:41 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T09:07:42.166 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:41 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:07:42.167 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:41 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:42.211 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:41 smithi112 ceph-mon[32307]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:42.211 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:41 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:42.211 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:41 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:42.211 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:41 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:42.211 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:41 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:07:42.211 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:41 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-16T09:07:42.212 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:41 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:07:42.212 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:41 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:43.051 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:42 smithi159 ceph-mon[36739]: Deploying daemon mgr.y on smithi159 2024-03-16T09:07:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-16T09:07:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-16T09:07:43.074 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-03-16T09:07:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-03-16T09:07:43.074 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115128 s, 911 MB/s 2024-03-16T09:07:43.074 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-03-16T09:07:43.145 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:42 smithi194 ceph-mon[36729]: Deploying daemon mgr.y on smithi159 2024-03-16T09:07:43.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:42 smithi112 ceph-mon[32307]: Deploying daemon mgr.y on smithi159 2024-03-16T09:07:43.401 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:07:43 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z[38619]: 2024-03-16T09:07:43.336+0000 7f4d3b3c6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T09:07:43.677 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi112:vg_nvme/lv_4 2024-03-16T09:07:43.736 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:43 smithi159 systemd[1]: Started Ceph mgr.y for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:07:43.901 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:07:43 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z[38619]: 2024-03-16T09:07:43.430+0000 7f4d3b3c6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T09:07:43.901 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:07:43 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z[38619]: 2024-03-16T09:07:43.605+0000 7f4d3b3c6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T09:07:43.920 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:07:44.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:43 smithi159 ceph-mon[36739]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:44.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:43 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:44.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:43 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:44.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:43 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:44.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:43 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:44.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:43 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:44.237 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:44 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:44.174+0000 7f948d494200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T09:07:44.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:43 smithi112 ceph-mon[32307]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:44.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:44.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:44.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:44.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:44.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:44.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:43 smithi194 ceph-mon[36729]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:44.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:43 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:44.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:43 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:44.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:43 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:44.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:43 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:44.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:43 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:44.730 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:44 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:44.271+0000 7f948d494200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T09:07:44.901 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:07:44 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z[38619]: 2024-03-16T09:07:44.622+0000 7f4d3b3c6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T09:07:44.902 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:07:44 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z[38619]: 2024-03-16T09:07:44.754+0000 7f4d3b3c6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T09:07:44.986 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:44 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:44.963+0000 7f948d494200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T09:07:45.307 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:45.055+0000 7f948d494200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T09:07:45.307 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:45.206+0000 7f948d494200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T09:07:45.307 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:45.305+0000 7f948d494200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T09:07:45.392 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z[38619]: 2024-03-16T09:07:45.102+0000 7f4d3b3c6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T09:07:45.392 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z[38619]: 2024-03-16T09:07:45.195+0000 7f4d3b3c6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T09:07:45.392 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z[38619]: 2024-03-16T09:07:45.390+0000 7f4d3b3c6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:07:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.439+0000 7fbfaf065700 1 -- 172.21.15.112:0/3681067308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa81022d0 msgr2=0x7fbfa81026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.439+0000 7fbfaf065700 1 --2- 172.21.15.112:0/3681067308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa81022d0 0x7fbfa81026f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009920 tx=0x7fbf9c009c30 comp rx=0 tx=0).stop 2024-03-16T09:07:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.439+0000 7fbfaf065700 1 -- 172.21.15.112:0/3681067308 shutdown_connections 2024-03-16T09:07:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.439+0000 7fbfaf065700 1 --2- 172.21.15.112:0/3681067308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbfa8103e70 0x7fbfa810a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.439+0000 7fbfaf065700 1 --2- 172.21.15.112:0/3681067308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa81022d0 0x7fbfa81026f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.439+0000 7fbfaf065700 1 --2- 172.21.15.112:0/3681067308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfa81034d0 0x7fbfa8103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:45.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.439+0000 7fbfaf065700 1 -- 172.21.15.112:0/3681067308 >> 172.21.15.112:0/3681067308 conn(0x7fbfa80fd8b0 msgr2=0x7fbfa80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:07:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.439+0000 7fbfaf065700 1 -- 172.21.15.112:0/3681067308 shutdown_connections 2024-03-16T09:07:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.439+0000 7fbfaf065700 1 -- 172.21.15.112:0/3681067308 wait complete. 2024-03-16T09:07:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.440+0000 7fbfaf065700 1 Processor -- start 2024-03-16T09:07:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.440+0000 7fbfaf065700 1 -- start start 2024-03-16T09:07:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.440+0000 7fbfaf065700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfa81022d0 0x7fbfa8198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.440+0000 7fbfaf065700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbfa81034d0 0x7fbfa8198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.440+0000 7fbfaf065700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa8103e70 0x7fbfa819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.440+0000 7fbfaf065700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa810e010 con 0x7fbfa8103e70 2024-03-16T09:07:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.440+0000 7fbfaf065700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa810de90 con 0x7fbfa81022d0 2024-03-16T09:07:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.440+0000 7fbfaf065700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa810e190 con 0x7fbfa81034d0 2024-03-16T09:07:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.440+0000 7fbfad602700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa8103e70 0x7fbfa819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:45.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.440+0000 7fbfa7fff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbfa81034d0 0x7fbfa8198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:45.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.440+0000 7fbface01700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfa81022d0 0x7fbfa8198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:45.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.440+0000 7fbface01700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfa81022d0 0x7fbfa8198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:52838/0 (socket says 172.21.15.112:52838) 2024-03-16T09:07:45.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.440+0000 7fbface01700 1 -- 172.21.15.112:0/559467444 learned_addr learned my addr 172.21.15.112:0/559467444 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:07:45.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.440+0000 7fbface01700 1 -- 172.21.15.112:0/559467444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbfa81034d0 msgr2=0x7fbfa8198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:45.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.440+0000 7fbface01700 1 --2- 172.21.15.112:0/559467444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbfa81034d0 0x7fbfa8198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:45.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.440+0000 7fbface01700 1 -- 172.21.15.112:0/559467444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa8103e70 msgr2=0x7fbfa819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:07:45.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.440+0000 7fbface01700 1 --2- 172.21.15.112:0/559467444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa8103e70 0x7fbfa819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:07:45.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.440+0000 7fbface01700 1 -- 172.21.15.112:0/559467444 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa81a0290 con 0x7fbfa81022d0 2024-03-16T09:07:45.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.441+0000 7fbface01700 1 --2- 172.21.15.112:0/559467444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfa81022d0 0x7fbfa8198630 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00fd90 tx=0x7fbf9c005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:45.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.441+0000 7fbfa5ffb700 1 -- 172.21.15.112:0/559467444 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c027070 con 0x7fbfa81022d0 2024-03-16T09:07:45.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.441+0000 7fbfaf065700 1 -- 172.21.15.112:0/559467444 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa81a0520 con 0x7fbfa81022d0 2024-03-16T09:07:45.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.441+0000 7fbfa5ffb700 1 -- 172.21.15.112:0/559467444 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbf9c00b560 con 0x7fbfa81022d0 2024-03-16T09:07:45.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.441+0000 7fbfa5ffb700 1 -- 172.21.15.112:0/559467444 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c004bd0 con 0x7fbfa81022d0 2024-03-16T09:07:45.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.441+0000 7fbfaf065700 1 -- 172.21.15.112:0/559467444 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa81a09d0 con 0x7fbfa81022d0 2024-03-16T09:07:45.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.442+0000 7fbfaf065700 1 -- 172.21.15.112:0/559467444 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa8068f90 con 0x7fbfa81022d0 2024-03-16T09:07:45.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.442+0000 7fbfa5ffb700 1 -- 172.21.15.112:0/559467444 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) ==== 46163+0+0 (secure 0 0 0) 0x7fbf9c02a070 con 0x7fbfa81022d0 2024-03-16T09:07:45.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.442+0000 7fbfa5ffb700 1 --2- 172.21.15.112:0/559467444 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fbf98039200 0x7fbf9803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:07:45.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.442+0000 7fbfa5ffb700 1 -- 172.21.15.112:0/559467444 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbf9c04cf90 con 0x7fbfa81022d0 2024-03-16T09:07:45.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.442+0000 7fbfa7fff700 1 --2- 172.21.15.112:0/559467444 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fbf98039200 0x7fbf9803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:07:45.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.443+0000 7fbfa7fff700 1 --2- 172.21.15.112:0/559467444 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fbf98039200 0x7fbf9803b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009750 tx=0x7fbf94009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:07:45.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.450+0000 7fbfa5ffb700 1 -- 172.21.15.112:0/559467444 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbf9c013030 con 0x7fbfa81022d0 2024-03-16T09:07:45.651 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z[38619]: 2024-03-16T09:07:45.573+0000 7f4d3b3c6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T09:07:45.656 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:45.388+0000 7f948d494200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T09:07:45.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:45.659+0000 7fbfaf065700 1 -- 172.21.15.112:0/559467444 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbfa8062710 con 0x7fbf98039200 2024-03-16T09:07:45.938 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:45.654+0000 7f948d494200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T09:07:45.939 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:45.762+0000 7f948d494200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T09:07:45.939 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:45.845+0000 7f948d494200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T09:07:46.117 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z[38619]: 2024-03-16T09:07:45.852+0000 7f4d3b3c6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:07:46.117 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-mon[36729]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:46.117 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:46.118 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:46.118 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:46.118 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:46.118 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:46.118 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:07:46.118 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:46.118 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:07:46.118 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:07:46.118 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:46.118 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:07:46.118 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:07:46.118 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:46.118 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:45 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:46.190 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:45 smithi112 ceph-mon[32307]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:46.190 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:45 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:46.190 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:45 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:46.190 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:45 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:46.190 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:45 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:46.190 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:45 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:46.190 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:45 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:07:46.190 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:45 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:46.190 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:45 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:07:46.190 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:45 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:07:46.191 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:45 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:46.191 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:45 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:07:46.191 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:45 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:07:46.191 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:45 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:46.191 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:45 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:46.236 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:45.941+0000 7f948d494200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T09:07:46.236 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:46 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:46.117+0000 7f948d494200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T09:07:46.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-mon[36739]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:46.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:46.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:46.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:46.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:46.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:46.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:07:46.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:46.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-16T09:07:46.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-16T09:07:46.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:46.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:07:46.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:07:46.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:46.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:45 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:46.401 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:07:46 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z[38619]: 2024-03-16T09:07:46.115+0000 7f4d3b3c6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T09:07:46.402 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:07:46 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z[38619]: 2024-03-16T09:07:46.347+0000 7f4d3b3c6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T09:07:47.212 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:46 smithi159 ceph-mon[36739]: Reconfiguring mgr.x (unknown last config time)... 2024-03-16T09:07:47.213 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:46 smithi159 ceph-mon[36739]: Reconfiguring daemon mgr.x on smithi112 2024-03-16T09:07:47.213 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:46 smithi159 ceph-mon[36739]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:47.213 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:47 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:47.082+0000 7f948d494200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T09:07:47.213 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:47 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:47.210+0000 7f948d494200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T09:07:47.233 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:46 smithi112 ceph-mon[32307]: Reconfiguring mgr.x (unknown last config time)... 2024-03-16T09:07:47.233 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:46 smithi112 ceph-mon[32307]: Reconfiguring daemon mgr.x on smithi112 2024-03-16T09:07:47.233 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:46 smithi112 ceph-mon[32307]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:47.401 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:07:47 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z[38619]: 2024-03-16T09:07:47.143+0000 7f4d3b3c6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T09:07:47.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:46 smithi194 ceph-mon[36729]: Reconfiguring mgr.x (unknown last config time)... 2024-03-16T09:07:47.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:46 smithi194 ceph-mon[36729]: Reconfiguring daemon mgr.x on smithi112 2024-03-16T09:07:47.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:46 smithi194 ceph-mon[36729]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:07:47.829 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:47 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:47.544+0000 7f948d494200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T09:07:47.829 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:47 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:47.634+0000 7f948d494200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T09:07:47.829 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:47 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:47.827+0000 7f948d494200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:07:47.966 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:07:47 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z[38619]: 2024-03-16T09:07:47.708+0000 7f4d3b3c6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T09:07:47.966 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:07:47 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z[38619]: 2024-03-16T09:07:47.803+0000 7f4d3b3c6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T09:07:48.236 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:48 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:48.009+0000 7f948d494200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T09:07:48.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:47 smithi159 ceph-mon[36739]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:48.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:47 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:48.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:47 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:48.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:47 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:48.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:47 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:48.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:47 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:07:48.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:47 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:48.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:47 smithi112 ceph-mon[32307]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:48.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:48.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:48.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:48.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:48.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:07:48.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:48.402 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:07:48 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z[38619]: 2024-03-16T09:07:48.014+0000 7f4d3b3c6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T09:07:48.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:47 smithi194 ceph-mon[36729]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:48.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:47 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:48.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:47 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:48.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:47 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:07:48.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:47 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:48.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:47 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:07:48.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:47 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:07:48.553 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:48 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:48.290+0000 7f948d494200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T09:07:48.553 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:48 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:48.551+0000 7f948d494200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T09:07:48.968 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:48 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:48.785+0000 7f948d494200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T09:07:48.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:48.972+0000 7fbfa5ffb700 1 -- 172.21.15.112:0/559467444 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mgrmap(e 14) ==== 91699+0+0 (secure 0 0 0) 0x7fbf9c00bb10 con 0x7fbfa81022d0 2024-03-16T09:07:49.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:48 smithi159 ceph-mon[36739]: Standby manager daemon z started 2024-03-16T09:07:49.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:48 smithi159 ceph-mon[36739]: from='mgr.? 172.21.15.194:0/790821716' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-16T09:07:49.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:48 smithi159 ceph-mon[36739]: from='mgr.? 172.21.15.194:0/790821716' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:07:49.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:48 smithi159 ceph-mon[36739]: from='mgr.? 172.21.15.194:0/790821716' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-16T09:07:49.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:48 smithi159 ceph-mon[36739]: from='mgr.? 172.21.15.194:0/790821716' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:07:49.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:48 smithi112 ceph-mon[32307]: Standby manager daemon z started 2024-03-16T09:07:49.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:48 smithi112 ceph-mon[32307]: from='mgr.? 172.21.15.194:0/790821716' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-16T09:07:49.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:48 smithi112 ceph-mon[32307]: from='mgr.? 172.21.15.194:0/790821716' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:07:49.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:48 smithi112 ceph-mon[32307]: from='mgr.? 172.21.15.194:0/790821716' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-16T09:07:49.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:48 smithi112 ceph-mon[32307]: from='mgr.? 172.21.15.194:0/790821716' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:07:49.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:48 smithi194 ceph-mon[36729]: Standby manager daemon z started 2024-03-16T09:07:49.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:48 smithi194 ceph-mon[36729]: from='mgr.? 172.21.15.194:0/790821716' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-16T09:07:49.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:48 smithi194 ceph-mon[36729]: from='mgr.? 172.21.15.194:0/790821716' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:07:49.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:48 smithi194 ceph-mon[36729]: from='mgr.? 172.21.15.194:0/790821716' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-16T09:07:49.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:48 smithi194 ceph-mon[36729]: from='mgr.? 172.21.15.194:0/790821716' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:07:49.973 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:49 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:49.577+0000 7f948d494200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T09:07:50.236 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:50 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:50.142+0000 7f948d494200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T09:07:50.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:49 smithi159 ceph-mon[36739]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:50.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:49 smithi159 ceph-mon[36739]: mgrmap e14: x(active, since 78s), standbys: z 2024-03-16T09:07:50.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:49 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-16T09:07:50.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:49 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3524985125' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cac77ab4-0b17-41b1-ac04-3ca10c14ea93"}]: dispatch 2024-03-16T09:07:50.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:49 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3524985125' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cac77ab4-0b17-41b1-ac04-3ca10c14ea93"}]': finished 2024-03-16T09:07:50.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:49 smithi159 ceph-mon[36739]: osdmap e5: 1 total, 0 up, 1 in 2024-03-16T09:07:50.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:49 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:07:50.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:49 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3343129629' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:07:50.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:49 smithi112 ceph-mon[32307]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:50.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:49 smithi112 ceph-mon[32307]: mgrmap e14: x(active, since 78s), standbys: z 2024-03-16T09:07:50.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:49 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-16T09:07:50.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:49 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3524985125' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cac77ab4-0b17-41b1-ac04-3ca10c14ea93"}]: dispatch 2024-03-16T09:07:50.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:49 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3524985125' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cac77ab4-0b17-41b1-ac04-3ca10c14ea93"}]': finished 2024-03-16T09:07:50.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:49 smithi112 ceph-mon[32307]: osdmap e5: 1 total, 0 up, 1 in 2024-03-16T09:07:50.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:49 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:07:50.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:49 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3343129629' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:07:50.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:49 smithi194 ceph-mon[36729]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:50.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:49 smithi194 ceph-mon[36729]: mgrmap e14: x(active, since 78s), standbys: z 2024-03-16T09:07:50.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:49 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-16T09:07:50.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:49 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3524985125' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cac77ab4-0b17-41b1-ac04-3ca10c14ea93"}]: dispatch 2024-03-16T09:07:50.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:49 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3524985125' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cac77ab4-0b17-41b1-ac04-3ca10c14ea93"}]': finished 2024-03-16T09:07:50.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:49 smithi194 ceph-mon[36729]: osdmap e5: 1 total, 0 up, 1 in 2024-03-16T09:07:50.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:49 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:07:50.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:49 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3343129629' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:07:50.736 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:50 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:50.239+0000 7f948d494200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T09:07:50.736 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:07:50 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y[38132]: 2024-03-16T09:07:50.451+0000 7f948d494200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T09:07:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:07:50.981+0000 7fbfa5ffb700 1 -- 172.21.15.112:0/559467444 <== mon.2 v2:172.21.15.159:3300/0 8 ==== mgrmap(e 15) ==== 137235+0+0 (secure 0 0 0) 0x7fbf9c02a360 con 0x7fbfa81022d0 2024-03-16T09:07:51.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:50 smithi159 ceph-mon[36739]: Standby manager daemon y started 2024-03-16T09:07:51.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:50 smithi159 ceph-mon[36739]: from='mgr.? 172.21.15.159:0/4234411185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-16T09:07:51.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:50 smithi159 ceph-mon[36739]: from='mgr.? 172.21.15.159:0/4234411185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:07:51.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:50 smithi159 ceph-mon[36739]: from='mgr.? 172.21.15.159:0/4234411185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-16T09:07:51.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:50 smithi159 ceph-mon[36739]: from='mgr.? 172.21.15.159:0/4234411185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:07:51.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:50 smithi112 ceph-mon[32307]: Standby manager daemon y started 2024-03-16T09:07:51.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:50 smithi112 ceph-mon[32307]: from='mgr.? 172.21.15.159:0/4234411185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-16T09:07:51.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:50 smithi112 ceph-mon[32307]: from='mgr.? 172.21.15.159:0/4234411185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:07:51.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:50 smithi112 ceph-mon[32307]: from='mgr.? 172.21.15.159:0/4234411185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-16T09:07:51.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:50 smithi112 ceph-mon[32307]: from='mgr.? 172.21.15.159:0/4234411185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:07:51.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:50 smithi194 ceph-mon[36729]: Standby manager daemon y started 2024-03-16T09:07:51.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:50 smithi194 ceph-mon[36729]: from='mgr.? 172.21.15.159:0/4234411185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-16T09:07:51.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:50 smithi194 ceph-mon[36729]: from='mgr.? 172.21.15.159:0/4234411185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-16T09:07:51.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:50 smithi194 ceph-mon[36729]: from='mgr.? 172.21.15.159:0/4234411185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-16T09:07:51.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:50 smithi194 ceph-mon[36729]: from='mgr.? 172.21.15.159:0/4234411185' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-16T09:07:52.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:51 smithi159 ceph-mon[36739]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:52.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:51 smithi159 ceph-mon[36739]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-16T09:07:52.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:51 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-16T09:07:52.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:51 smithi112 ceph-mon[32307]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:52.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:51 smithi112 ceph-mon[32307]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-16T09:07:52.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:51 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-16T09:07:52.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:51 smithi194 ceph-mon[36729]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:52.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:51 smithi194 ceph-mon[36729]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-03-16T09:07:52.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:51 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-16T09:07:54.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:53 smithi159 ceph-mon[36739]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:54.252 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:53 smithi112 ceph-mon[32307]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:54.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:53 smithi194 ceph-mon[36729]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:56.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:55 smithi159 ceph-mon[36739]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:56.271 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:55 smithi112 ceph-mon[32307]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:56.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:55 smithi194 ceph-mon[36729]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:57.191 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T09:07:57.191 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:57.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:56 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T09:07:57.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:56 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:57.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-16T09:07:57.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:07:58.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:57 smithi112 ceph-mon[32307]: Deploying daemon osd.0 on smithi112 2024-03-16T09:07:58.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:57 smithi112 ceph-mon[32307]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:58.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:57 smithi194 ceph-mon[36729]: Deploying daemon osd.0 on smithi112 2024-03-16T09:07:58.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:57 smithi194 ceph-mon[36729]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:07:58.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:57 smithi159 ceph-mon[36739]: Deploying daemon osd.0 on smithi112 2024-03-16T09:07:58.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:57 smithi159 ceph-mon[36739]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:08:00.076 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:07:59 smithi112 ceph-mon[32307]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:08:00.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:07:59 smithi194 ceph-mon[36729]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:08:00.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:07:59 smithi159 ceph-mon[36739]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:08:02.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:01 smithi112 ceph-mon[32307]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:08:02.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:02.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:02.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:02.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:01 smithi194 ceph-mon[36729]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:08:02.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:02.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:02.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:02.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:01 smithi159 ceph-mon[36739]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:08:02.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:02.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:02.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:04.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:03 smithi112 ceph-mon[32307]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:08:04.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:03 smithi194 ceph-mon[36729]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:08:04.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:03 smithi159 ceph-mon[36739]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:08:05.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:05.267+0000 7fbfa5ffb700 1 -- 172.21.15.112:0/559467444 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbfa8062710 con 0x7fbf98039200 2024-03-16T09:08:05.270 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 0 on host 'smithi112' 2024-03-16T09:08:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:05.271+0000 7fbfaf065700 1 -- 172.21.15.112:0/559467444 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fbf98039200 msgr2=0x7fbf9803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:05.271+0000 7fbfaf065700 1 --2- 172.21.15.112:0/559467444 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fbf98039200 0x7fbf9803b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009750 tx=0x7fbf94009310 comp rx=0 tx=0).stop 2024-03-16T09:08:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:05.271+0000 7fbfaf065700 1 -- 172.21.15.112:0/559467444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfa81022d0 msgr2=0x7fbfa8198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:05.271+0000 7fbfaf065700 1 --2- 172.21.15.112:0/559467444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfa81022d0 0x7fbfa8198630 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00fd90 tx=0x7fbf9c005040 comp rx=0 tx=0).stop 2024-03-16T09:08:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:05.271+0000 7fbfaf065700 1 -- 172.21.15.112:0/559467444 shutdown_connections 2024-03-16T09:08:05.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:05.271+0000 7fbfaf065700 1 --2- 172.21.15.112:0/559467444 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fbf98039200 0x7fbf9803b6c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:05.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:05.271+0000 7fbfaf065700 1 --2- 172.21.15.112:0/559467444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfa81022d0 0x7fbfa8198630 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:05.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:05.271+0000 7fbfaf065700 1 --2- 172.21.15.112:0/559467444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfa8103e70 0x7fbfa819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:05.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:05.271+0000 7fbfaf065700 1 --2- 172.21.15.112:0/559467444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbfa81034d0 0x7fbfa8198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:05.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:05.271+0000 7fbfaf065700 1 -- 172.21.15.112:0/559467444 >> 172.21.15.112:0/559467444 conn(0x7fbfa80fd8b0 msgr2=0x7fbfa81067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:05.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:05.272+0000 7fbfaf065700 1 -- 172.21.15.112:0/559467444 shutdown_connections 2024-03-16T09:08:05.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:05.272+0000 7fbfaf065700 1 -- 172.21.15.112:0/559467444 wait complete. 2024-03-16T09:08:05.936 DEBUG:teuthology.orchestra.run.smithi112:osd.0> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.0.service 2024-03-16T09:08:05.939 INFO:tasks.cephadm:Deploying osd.1 on smithi159 with /dev/vg_nvme/lv_4... 2024-03-16T09:08:05.939 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-16T09:08:06.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:05 smithi194 ceph-mon[36729]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:08:06.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:06.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:06.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:06.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:06.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:06.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:06.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:06.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:06.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:05 smithi194 ceph-mon[36729]: from='osd.0 [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-16T09:08:06.170 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:05 smithi112 ceph-mon[32307]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:08:06.170 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:06.170 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:06.170 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:06.170 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:06.170 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:06.170 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:06.170 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:06.170 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:06.170 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:05 smithi112 ceph-mon[32307]: from='osd.0 [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-16T09:08:06.183 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T09:08:06.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:05 smithi159 ceph-mon[36739]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:08:06.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:06.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:06.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:06.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:06.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:06.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:06.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:06.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:06.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:05 smithi159 ceph-mon[36739]: from='osd.0 [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-16T09:08:07.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:07 smithi159 ceph-mon[36739]: from='osd.0 [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-16T09:08:07.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:07 smithi159 ceph-mon[36739]: osdmap e6: 1 total, 0 up, 1 in 2024-03-16T09:08:07.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:07 smithi159 ceph-mon[36739]: from='osd.0 [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-03-16T09:08:07.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:07 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:08:07.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:07 smithi159 ceph-mon[36739]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:08:07.484 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:07 smithi112 ceph-mon[32307]: from='osd.0 [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-16T09:08:07.484 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:07 smithi112 ceph-mon[32307]: osdmap e6: 1 total, 0 up, 1 in 2024-03-16T09:08:07.484 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:07 smithi112 ceph-mon[32307]: from='osd.0 [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-03-16T09:08:07.484 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:07 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:08:07.484 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:07 smithi112 ceph-mon[32307]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:08:07.484 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:08:07 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-0[47070]: 2024-03-16T09:08:07.257+0000 7f9066afc700 -1 osd.0 0 waiting for initial osdmap 2024-03-16T09:08:07.485 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:08:07 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-0[47070]: 2024-03-16T09:08:07.261+0000 7f90628ef700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T09:08:07.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:07 smithi194 ceph-mon[36729]: from='osd.0 [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-16T09:08:07.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:07 smithi194 ceph-mon[36729]: osdmap e6: 1 total, 0 up, 1 in 2024-03-16T09:08:07.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:07 smithi194 ceph-mon[36729]: from='osd.0 [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-03-16T09:08:07.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:07 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:08:07.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:07 smithi194 ceph-mon[36729]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:08:08.002 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-16T09:08:08.002 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-16T09:08:08.003 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2024-03-16T09:08:08.003 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2024-03-16T09:08:08.003 INFO:teuthology.orchestra.run.smithi159.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011479 s, 913 MB/s 2024-03-16T09:08:08.003 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2024-03-16T09:08:08.564 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:08 smithi159 ceph-mon[36739]: from='osd.0 [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-03-16T09:08:08.564 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:08 smithi159 ceph-mon[36739]: osdmap e7: 1 total, 0 up, 1 in 2024-03-16T09:08:08.564 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:08:08.564 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:08:08.591 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi159:vg_nvme/lv_4 2024-03-16T09:08:08.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:08 smithi112 ceph-mon[32307]: from='osd.0 [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-03-16T09:08:08.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:08 smithi112 ceph-mon[32307]: osdmap e7: 1 total, 0 up, 1 in 2024-03-16T09:08:08.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:08:08.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:08:08.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:08 smithi194 ceph-mon[36729]: from='osd.0 [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-03-16T09:08:08.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:08 smithi194 ceph-mon[36729]: osdmap e7: 1 total, 0 up, 1 in 2024-03-16T09:08:08.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:08:08.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:08:08.838 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T09:08:09.267 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:09 smithi112 ceph-mon[32307]: purged_snaps scrub starts 2024-03-16T09:08:09.267 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:09 smithi112 ceph-mon[32307]: purged_snaps scrub ok 2024-03-16T09:08:09.267 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:09 smithi112 ceph-mon[32307]: osd.0 [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115] boot 2024-03-16T09:08:09.268 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:09 smithi112 ceph-mon[32307]: osdmap e8: 1 total, 1 up, 1 in 2024-03-16T09:08:09.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:09 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:08:09.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:09 smithi112 ceph-mon[32307]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:08:09.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:09 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:09.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:09 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:09.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:09 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:08:09.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:09 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:09.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:09 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:09.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:09 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:09.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:09 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:09.610 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:09 smithi159 ceph-mon[36739]: purged_snaps scrub starts 2024-03-16T09:08:09.611 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:09 smithi159 ceph-mon[36739]: purged_snaps scrub ok 2024-03-16T09:08:09.611 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:09 smithi159 ceph-mon[36739]: osd.0 [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115] boot 2024-03-16T09:08:09.611 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:09 smithi159 ceph-mon[36739]: osdmap e8: 1 total, 1 up, 1 in 2024-03-16T09:08:09.611 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:09 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:08:09.611 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:09 smithi159 ceph-mon[36739]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:08:09.611 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:09 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:09.611 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:09 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:09.611 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:09 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:08:09.611 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:09 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:09.611 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:09 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:09.611 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:09 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:09.611 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:09 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:09.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:09 smithi194 ceph-mon[36729]: purged_snaps scrub starts 2024-03-16T09:08:09.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:09 smithi194 ceph-mon[36729]: purged_snaps scrub ok 2024-03-16T09:08:09.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:09 smithi194 ceph-mon[36729]: osd.0 [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115] boot 2024-03-16T09:08:09.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:09 smithi194 ceph-mon[36729]: osdmap e8: 1 total, 1 up, 1 in 2024-03-16T09:08:09.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:09 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-16T09:08:09.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:09 smithi194 ceph-mon[36729]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-16T09:08:09.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:09 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:09.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:09 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:09.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:09 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:08:09.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:09 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:09.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:09 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:09.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:09 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:09.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:09 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:10.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.324+0000 7fa20f370700 1 -- 172.21.15.159:0/1451682777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa208103520 msgr2=0x7fa208103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:10.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.324+0000 7fa20f370700 1 --2- 172.21.15.159:0/1451682777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa208103520 0x7fa208103980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80099c0 tx=0x7fa1f8009cd0 comp rx=0 tx=0).stop 2024-03-16T09:08:10.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.324+0000 7fa20f370700 1 -- 172.21.15.159:0/1451682777 shutdown_connections 2024-03-16T09:08:10.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.324+0000 7fa20f370700 1 --2- 172.21.15.159:0/1451682777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa208103ec0 0x7fa20810a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:10.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.324+0000 7fa20f370700 1 --2- 172.21.15.159:0/1451682777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa208102320 0x7fa208102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:10.331 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.324+0000 7fa20f370700 1 --2- 172.21.15.159:0/1451682777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa208103520 0x7fa208103980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:10.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.324+0000 7fa20f370700 1 -- 172.21.15.159:0/1451682777 >> 172.21.15.159:0/1451682777 conn(0x7fa2080fd8e0 msgr2=0x7fa2080ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:10.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.324+0000 7fa20f370700 1 -- 172.21.15.159:0/1451682777 shutdown_connections 2024-03-16T09:08:10.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.324+0000 7fa20f370700 1 -- 172.21.15.159:0/1451682777 wait complete. 2024-03-16T09:08:10.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.325+0000 7fa20f370700 1 Processor -- start 2024-03-16T09:08:10.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.325+0000 7fa20f370700 1 -- start start 2024-03-16T09:08:10.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.325+0000 7fa20f370700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa208102320 0x7fa208076070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:10.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.325+0000 7fa20f370700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa208103520 0x7fa2080746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:10.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.325+0000 7fa20f370700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa208103ec0 0x7fa208074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:10.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.325+0000 7fa20f370700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa20810e0a0 con 0x7fa208103ec0 2024-03-16T09:08:10.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.325+0000 7fa20f370700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa20810df20 con 0x7fa208102320 2024-03-16T09:08:10.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.325+0000 7fa20f370700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa20810e220 con 0x7fa208103520 2024-03-16T09:08:10.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.325+0000 7fa20d10c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa208102320 0x7fa208076070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:10.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.325+0000 7fa20d10c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa208102320 0x7fa208076070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:40906/0 (socket says 172.21.15.159:40906) 2024-03-16T09:08:10.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.325+0000 7fa20d10c700 1 -- 172.21.15.159:0/2471384999 learned_addr learned my addr 172.21.15.159:0/2471384999 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-03-16T09:08:10.332 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.325+0000 7fa20c90b700 1 --2- 172.21.15.159:0/2471384999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa208103520 0x7fa2080746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:10.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.325+0000 7fa20d90d700 1 --2- 172.21.15.159:0/2471384999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa208103ec0 0x7fa208074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:10.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.325+0000 7fa20d10c700 1 -- 172.21.15.159:0/2471384999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa208103520 msgr2=0x7fa2080746c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:10.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.325+0000 7fa20d10c700 1 --2- 172.21.15.159:0/2471384999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa208103520 0x7fa2080746c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:10.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.325+0000 7fa20d10c700 1 -- 172.21.15.159:0/2471384999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa208103ec0 msgr2=0x7fa208074c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:10.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.325+0000 7fa20d10c700 1 --2- 172.21.15.159:0/2471384999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa208103ec0 0x7fa208074c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:10.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.325+0000 7fa20d10c700 1 -- 172.21.15.159:0/2471384999 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2080754b0 con 0x7fa208102320 2024-03-16T09:08:10.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.325+0000 7fa20d90d700 1 --2- 172.21.15.159:0/2471384999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa208103ec0 0x7fa208074c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:08:10.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.326+0000 7fa20d10c700 1 --2- 172.21.15.159:0/2471384999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa208102320 0x7fa208076070 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa20400d980 tx=0x7fa20400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:10.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.326+0000 7fa1fe7fc700 1 -- 172.21.15.159:0/2471384999 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204017030 con 0x7fa208102320 2024-03-16T09:08:10.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.326+0000 7fa20f370700 1 -- 172.21.15.159:0/2471384999 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2081a2750 con 0x7fa208102320 2024-03-16T09:08:10.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.326+0000 7fa1fe7fc700 1 -- 172.21.15.159:0/2471384999 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa204004520 con 0x7fa208102320 2024-03-16T09:08:10.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.326+0000 7fa1fe7fc700 1 -- 172.21.15.159:0/2471384999 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204003b10 con 0x7fa208102320 2024-03-16T09:08:10.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.326+0000 7fa20f370700 1 -- 172.21.15.159:0/2471384999 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2081a2c90 con 0x7fa208102320 2024-03-16T09:08:10.333 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.327+0000 7fa20f370700 1 -- 172.21.15.159:0/2471384999 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa208068f90 con 0x7fa208102320 2024-03-16T09:08:10.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.328+0000 7fa1fe7fc700 1 -- 172.21.15.159:0/2471384999 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 15) ==== 137235+0+0 (secure 0 0 0) 0x7fa20401a070 con 0x7fa208102320 2024-03-16T09:08:10.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.328+0000 7fa1fe7fc700 1 --2- 172.21.15.159:0/2471384999 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa1f40a2800 0x7fa1f40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:10.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.329+0000 7fa1fe7fc700 1 -- 172.21.15.159:0/2471384999 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fa2040cccd0 con 0x7fa208102320 2024-03-16T09:08:10.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.329+0000 7fa20c90b700 1 --2- 172.21.15.159:0/2471384999 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa1f40a2800 0x7fa1f40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:10.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.329+0000 7fa20c90b700 1 --2- 172.21.15.159:0/2471384999 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa1f40a2800 0x7fa1f40a4cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009990 tx=0x7fa1f8005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:10.334 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.332+0000 7fa1fe7fc700 1 -- 172.21.15.159:0/2471384999 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa20409b7e0 con 0x7fa208102320 2024-03-16T09:08:10.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:10 smithi159 ceph-mon[36739]: Detected new or changed devices on smithi112 2024-03-16T09:08:10.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:10 smithi159 ceph-mon[36739]: Adjusting osd_memory_target on smithi112 to 17088M 2024-03-16T09:08:10.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:10 smithi159 ceph-mon[36739]: osdmap e9: 1 total, 1 up, 1 in 2024-03-16T09:08:10.536 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:10.534+0000 7fa20f370700 1 -- 172.21.15.159:0/2471384999 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa208062710 con 0x7fa1f40a2800 2024-03-16T09:08:10.553 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:10 smithi194 ceph-mon[36729]: Detected new or changed devices on smithi112 2024-03-16T09:08:10.553 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:10 smithi194 ceph-mon[36729]: Adjusting osd_memory_target on smithi112 to 17088M 2024-03-16T09:08:10.553 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:10 smithi194 ceph-mon[36729]: osdmap e9: 1 total, 1 up, 1 in 2024-03-16T09:08:10.591 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:10 smithi112 ceph-mon[32307]: Detected new or changed devices on smithi112 2024-03-16T09:08:10.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:10 smithi112 ceph-mon[32307]: Adjusting osd_memory_target on smithi112 to 17088M 2024-03-16T09:08:10.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:10 smithi112 ceph-mon[32307]: osdmap e9: 1 total, 1 up, 1 in 2024-03-16T09:08:11.467 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:11 smithi159 ceph-mon[36739]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:11.467 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:11 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:08:11.467 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:11 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:08:11.467 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:11 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:11.467 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:11 smithi159 ceph-mon[36739]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:11.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:11 smithi112 ceph-mon[32307]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:11.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:11 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:08:11.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:11 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:08:11.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:11 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:11.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:11 smithi112 ceph-mon[32307]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:11.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:11 smithi194 ceph-mon[36729]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:11.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:11 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:08:11.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:11 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:08:11.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:11 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:11.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:11 smithi194 ceph-mon[36729]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:14.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:13 smithi194 ceph-mon[36729]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:14.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:13 smithi194 ceph-mon[36729]: from='client.? 172.21.15.159:0/374523453' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b46a1d96-d8cc-4b30-b9d7-b656c94ac581"}]: dispatch 2024-03-16T09:08:14.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:13 smithi194 ceph-mon[36729]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b46a1d96-d8cc-4b30-b9d7-b656c94ac581"}]: dispatch 2024-03-16T09:08:14.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:13 smithi194 ceph-mon[36729]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b46a1d96-d8cc-4b30-b9d7-b656c94ac581"}]': finished 2024-03-16T09:08:14.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:13 smithi194 ceph-mon[36729]: osdmap e10: 2 total, 1 up, 2 in 2024-03-16T09:08:14.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:13 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:08:14.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:13 smithi194 ceph-mon[36729]: from='client.? 172.21.15.159:0/2418914255' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:08:14.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:13 smithi159 ceph-mon[36739]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:14.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:13 smithi159 ceph-mon[36739]: from='client.? 172.21.15.159:0/374523453' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b46a1d96-d8cc-4b30-b9d7-b656c94ac581"}]: dispatch 2024-03-16T09:08:14.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:13 smithi159 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b46a1d96-d8cc-4b30-b9d7-b656c94ac581"}]: dispatch 2024-03-16T09:08:14.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:13 smithi159 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b46a1d96-d8cc-4b30-b9d7-b656c94ac581"}]': finished 2024-03-16T09:08:14.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:13 smithi159 ceph-mon[36739]: osdmap e10: 2 total, 1 up, 2 in 2024-03-16T09:08:14.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:13 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:08:14.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:13 smithi159 ceph-mon[36739]: from='client.? 172.21.15.159:0/2418914255' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:08:14.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:13 smithi112 ceph-mon[32307]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:14.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:13 smithi112 ceph-mon[32307]: from='client.? 172.21.15.159:0/374523453' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b46a1d96-d8cc-4b30-b9d7-b656c94ac581"}]: dispatch 2024-03-16T09:08:14.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:13 smithi112 ceph-mon[32307]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b46a1d96-d8cc-4b30-b9d7-b656c94ac581"}]: dispatch 2024-03-16T09:08:14.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:13 smithi112 ceph-mon[32307]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b46a1d96-d8cc-4b30-b9d7-b656c94ac581"}]': finished 2024-03-16T09:08:14.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:13 smithi112 ceph-mon[32307]: osdmap e10: 2 total, 1 up, 2 in 2024-03-16T09:08:14.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:13 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:08:14.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:13 smithi112 ceph-mon[32307]: from='client.? 172.21.15.159:0/2418914255' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:08:16.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:15 smithi112 ceph-mon[32307]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:16.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:15 smithi194 ceph-mon[36729]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:16.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:15 smithi159 ceph-mon[36739]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:18.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:17 smithi194 ceph-mon[36729]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:18.163 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:17 smithi159 ceph-mon[36739]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:17 smithi112 ceph-mon[32307]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:20.144 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:19 smithi112 ceph-mon[32307]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:20.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:19 smithi194 ceph-mon[36729]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:20.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:19 smithi159 ceph-mon[36739]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:21.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T09:08:21.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:21.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:20 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T09:08:21.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:20 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-16T09:08:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:21.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:21 smithi159 ceph-mon[36739]: Deploying daemon osd.1 on smithi159 2024-03-16T09:08:21.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:21 smithi159 ceph-mon[36739]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:22.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:21 smithi194 ceph-mon[36729]: Deploying daemon osd.1 on smithi159 2024-03-16T09:08:22.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:21 smithi194 ceph-mon[36729]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:22.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:21 smithi112 ceph-mon[32307]: Deploying daemon osd.1 on smithi159 2024-03-16T09:08:22.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:21 smithi112 ceph-mon[32307]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:24.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:23 smithi194 ceph-mon[36729]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:24.185 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:23 smithi159 ceph-mon[36739]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:24.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:23 smithi112 ceph-mon[32307]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:25.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:25 smithi159 ceph-mon[36739]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:25.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:25 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:25.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:25 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:25.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:25 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:26.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:25 smithi194 ceph-mon[36729]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:26.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:25 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:26.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:25 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:26.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:25 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:26.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:25 smithi112 ceph-mon[32307]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:26.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:25 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:26.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:25 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:26.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:25 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:28.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:27 smithi194 ceph-mon[36729]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:28.153 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:27 smithi159 ceph-mon[36739]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:28.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:27 smithi112 ceph-mon[32307]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:29.197 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 1 on host 'smithi159' 2024-03-16T09:08:29.197 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:29.194+0000 7fa1fe7fc700 1 -- 172.21.15.159:0/2471384999 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa208062710 con 0x7fa1f40a2800 2024-03-16T09:08:29.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:29.197+0000 7fa20f370700 1 -- 172.21.15.159:0/2471384999 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa1f40a2800 msgr2=0x7fa1f40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:29.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:29.198+0000 7fa20f370700 1 --2- 172.21.15.159:0/2471384999 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa1f40a2800 0x7fa1f40a4cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009990 tx=0x7fa1f8005bc0 comp rx=0 tx=0).stop 2024-03-16T09:08:29.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:29.198+0000 7fa20f370700 1 -- 172.21.15.159:0/2471384999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa208102320 msgr2=0x7fa208076070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:29.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:29.198+0000 7fa20f370700 1 --2- 172.21.15.159:0/2471384999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa208102320 0x7fa208076070 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa20400d980 tx=0x7fa20400dc90 comp rx=0 tx=0).stop 2024-03-16T09:08:29.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:29.198+0000 7fa20f370700 1 -- 172.21.15.159:0/2471384999 shutdown_connections 2024-03-16T09:08:29.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:29.198+0000 7fa20f370700 1 --2- 172.21.15.159:0/2471384999 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa1f40a2800 0x7fa1f40a4cc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:29.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:29.198+0000 7fa20f370700 1 --2- 172.21.15.159:0/2471384999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa208102320 0x7fa208076070 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:29.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:29.198+0000 7fa20f370700 1 --2- 172.21.15.159:0/2471384999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa208103ec0 0x7fa208074c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:29.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:29.198+0000 7fa20f370700 1 --2- 172.21.15.159:0/2471384999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa208103520 0x7fa2080746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:29.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:29.198+0000 7fa20f370700 1 -- 172.21.15.159:0/2471384999 >> 172.21.15.159:0/2471384999 conn(0x7fa2080fd8e0 msgr2=0x7fa208106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:29.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:29.198+0000 7fa20f370700 1 -- 172.21.15.159:0/2471384999 shutdown_connections 2024-03-16T09:08:29.200 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:08:29.198+0000 7fa20f370700 1 -- 172.21.15.159:0/2471384999 wait complete. 2024-03-16T09:08:29.862 DEBUG:teuthology.orchestra.run.smithi159:osd.1> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.1.service 2024-03-16T09:08:29.864 INFO:tasks.cephadm:Deploying osd.2 on smithi194 with /dev/vg_nvme/lv_4... 2024-03-16T09:08:29.865 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-16T09:08:30.110 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:08:30.126 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:29 smithi159 ceph-mon[36739]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:30.127 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:29 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:30.127 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:29 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:30.127 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:29 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:30.127 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:29 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:30.127 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:29 smithi159 ceph-mon[36739]: from='osd.1 [v2:172.21.15.159:6800/140175088,v1:172.21.15.159:6801/140175088]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T09:08:30.127 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:29 smithi159 ceph-mon[36739]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T09:08:30.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:29 smithi194 ceph-mon[36729]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:30.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:30.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:30.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:30.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:30.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:29 smithi194 ceph-mon[36729]: from='osd.1 [v2:172.21.15.159:6800/140175088,v1:172.21.15.159:6801/140175088]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T09:08:30.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:29 smithi194 ceph-mon[36729]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T09:08:30.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:29 smithi112 ceph-mon[32307]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:30.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:30.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:30.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:30.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:30.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:29 smithi112 ceph-mon[32307]: from='osd.1 [v2:172.21.15.159:6800/140175088,v1:172.21.15.159:6801/140175088]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T09:08:30.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:29 smithi112 ceph-mon[32307]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-16T09:08:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:31 smithi194 ceph-mon[36729]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-16T09:08:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:31 smithi194 ceph-mon[36729]: from='osd.1 [v2:172.21.15.159:6800/140175088,v1:172.21.15.159:6801/140175088]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:08:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:31 smithi194 ceph-mon[36729]: osdmap e11: 2 total, 1 up, 2 in 2024-03-16T09:08:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:08:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:31 smithi194 ceph-mon[36729]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:08:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:31 smithi194 ceph-mon[36729]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:31 smithi159 ceph-mon[36739]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-16T09:08:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:31 smithi159 ceph-mon[36739]: from='osd.1 [v2:172.21.15.159:6800/140175088,v1:172.21.15.159:6801/140175088]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:08:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:31 smithi159 ceph-mon[36739]: osdmap e11: 2 total, 1 up, 2 in 2024-03-16T09:08:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:08:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:31 smithi159 ceph-mon[36739]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:08:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:31 smithi159 ceph-mon[36739]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:31.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:31 smithi112 ceph-mon[32307]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-16T09:08:31.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:31 smithi112 ceph-mon[32307]: from='osd.1 [v2:172.21.15.159:6800/140175088,v1:172.21.15.159:6801/140175088]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:08:31.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:31 smithi112 ceph-mon[32307]: osdmap e11: 2 total, 1 up, 2 in 2024-03-16T09:08:31.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:08:31.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:31 smithi112 ceph-mon[32307]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:08:31.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:31 smithi112 ceph-mon[32307]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:31.986 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:08:31 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-1[41333]: 2024-03-16T09:08:31.687+0000 7fdf1f5db700 -1 osd.1 0 waiting for initial osdmap 2024-03-16T09:08:31.986 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:08:31 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-1[41333]: 2024-03-16T09:08:31.690+0000 7fdf173c6700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T09:08:32.075 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-16T09:08:32.075 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-16T09:08:32.075 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2024-03-16T09:08:32.075 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2024-03-16T09:08:32.075 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0186895 s, 561 MB/s 2024-03-16T09:08:32.075 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2024-03-16T09:08:32.289 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:32 smithi194 ceph-mon[36729]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-16T09:08:32.289 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:32 smithi194 ceph-mon[36729]: osdmap e12: 2 total, 1 up, 2 in 2024-03-16T09:08:32.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:32 smithi159 ceph-mon[36739]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-16T09:08:32.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:32 smithi159 ceph-mon[36739]: osdmap e12: 2 total, 1 up, 2 in 2024-03-16T09:08:32.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:32 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:08:32.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:32 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:08:32.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:32 smithi159 ceph-mon[36739]: from='osd.1 ' entity='osd.1' 2024-03-16T09:08:32.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:32 smithi112 ceph-mon[32307]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-16T09:08:32.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:32 smithi112 ceph-mon[32307]: osdmap e12: 2 total, 1 up, 2 in 2024-03-16T09:08:32.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:32 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:08:32.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:32 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:08:32.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:32 smithi112 ceph-mon[32307]: from='osd.1 ' entity='osd.1' 2024-03-16T09:08:32.637 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:32 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:08:32.637 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:32 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:08:32.637 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:32 smithi194 ceph-mon[36729]: from='osd.1 ' entity='osd.1' 2024-03-16T09:08:32.665 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi194:vg_nvme/lv_4 2024-03-16T09:08:32.902 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:08:33.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: purged_snaps scrub starts 2024-03-16T09:08:33.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: purged_snaps scrub ok 2024-03-16T09:08:33.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:08:33.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: osd.1 [v2:172.21.15.159:6800/140175088,v1:172.21.15.159:6801/140175088] boot 2024-03-16T09:08:33.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: osdmap e13: 2 total, 2 up, 2 in 2024-03-16T09:08:33.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:08:33.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: Detected new or changed devices on smithi159 2024-03-16T09:08:33.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:33.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:33.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:08:33.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: Adjusting osd_memory_target on smithi159 to 17080M 2024-03-16T09:08:33.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:33.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:33.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:33.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:33.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:33.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:33.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:33.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:33.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:33.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: purged_snaps scrub starts 2024-03-16T09:08:33.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: purged_snaps scrub ok 2024-03-16T09:08:33.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:08:33.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: osd.1 [v2:172.21.15.159:6800/140175088,v1:172.21.15.159:6801/140175088] boot 2024-03-16T09:08:33.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: osdmap e13: 2 total, 2 up, 2 in 2024-03-16T09:08:33.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:08:33.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: Detected new or changed devices on smithi159 2024-03-16T09:08:33.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:33.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:33.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:08:33.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: Adjusting osd_memory_target on smithi159 to 17080M 2024-03-16T09:08:33.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:33.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:33.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:33.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:33.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:33.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:33.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:33.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:33.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:33.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: purged_snaps scrub starts 2024-03-16T09:08:33.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: purged_snaps scrub ok 2024-03-16T09:08:33.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:08:33.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: osd.1 [v2:172.21.15.159:6800/140175088,v1:172.21.15.159:6801/140175088] boot 2024-03-16T09:08:33.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: osdmap e13: 2 total, 2 up, 2 in 2024-03-16T09:08:33.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-16T09:08:33.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: Detected new or changed devices on smithi159 2024-03-16T09:08:33.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:33.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:33.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:08:33.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: Adjusting osd_memory_target on smithi159 to 17080M 2024-03-16T09:08:33.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:33.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:33.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-16T09:08:33.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:33.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:33.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:33.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:33.738 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:33.738 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:34.402 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.398+0000 7fd503630700 1 -- 172.21.15.194:0/579797076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4fc101200 msgr2=0x7fd4fc101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:34.402 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.398+0000 7fd503630700 1 --2- 172.21.15.194:0/579797076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4fc101200 0x7fd4fc101620 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009990 tx=0x7fd4f8009ca0 comp rx=0 tx=0).stop 2024-03-16T09:08:34.403 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.398+0000 7fd503630700 1 -- 172.21.15.194:0/579797076 shutdown_connections 2024-03-16T09:08:34.403 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.398+0000 7fd503630700 1 --2- 172.21.15.194:0/579797076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4fc102da0 0x7fd4fc109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:34.403 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.398+0000 7fd503630700 1 --2- 172.21.15.194:0/579797076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4fc101200 0x7fd4fc101620 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:34.403 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.398+0000 7fd503630700 1 --2- 172.21.15.194:0/579797076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4fc102400 0x7fd4fc102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:34.403 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.398+0000 7fd503630700 1 -- 172.21.15.194:0/579797076 >> 172.21.15.194:0/579797076 conn(0x7fd4fc0fc7a0 msgr2=0x7fd4fc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:34.403 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.398+0000 7fd503630700 1 -- 172.21.15.194:0/579797076 shutdown_connections 2024-03-16T09:08:34.403 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.398+0000 7fd503630700 1 -- 172.21.15.194:0/579797076 wait complete. 2024-03-16T09:08:34.403 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.399+0000 7fd503630700 1 Processor -- start 2024-03-16T09:08:34.403 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.399+0000 7fd503630700 1 -- start start 2024-03-16T09:08:34.403 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.399+0000 7fd503630700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4fc101200 0x7fd4fc197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:34.403 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.399+0000 7fd503630700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4fc102400 0x7fd4fc197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:34.403 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.399+0000 7fd503630700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4fc102da0 0x7fd4fc19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:34.403 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.399+0000 7fd503630700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc10d000 con 0x7fd4fc102da0 2024-03-16T09:08:34.404 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.399+0000 7fd503630700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc10ce80 con 0x7fd4fc102400 2024-03-16T09:08:34.404 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.399+0000 7fd503630700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd4fc10d180 con 0x7fd4fc101200 2024-03-16T09:08:34.404 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.399+0000 7fd5013cc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4fc101200 0x7fd4fc197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:34.404 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.399+0000 7fd5013cc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4fc101200 0x7fd4fc197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:40144/0 (socket says 172.21.15.194:40144) 2024-03-16T09:08:34.404 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.399+0000 7fd5013cc700 1 -- 172.21.15.194:0/3475208939 learned_addr learned my addr 172.21.15.194:0/3475208939 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T09:08:34.404 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.400+0000 7fd500bcb700 1 --2- 172.21.15.194:0/3475208939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4fc102400 0x7fd4fc197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:34.404 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.400+0000 7fd501bcd700 1 --2- 172.21.15.194:0/3475208939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4fc102da0 0x7fd4fc19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:34.404 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.400+0000 7fd5013cc700 1 -- 172.21.15.194:0/3475208939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4fc102400 msgr2=0x7fd4fc197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:34.404 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.400+0000 7fd5013cc700 1 --2- 172.21.15.194:0/3475208939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4fc102400 0x7fd4fc197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:34.404 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.400+0000 7fd5013cc700 1 -- 172.21.15.194:0/3475208939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4fc102da0 msgr2=0x7fd4fc19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:34.404 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.400+0000 7fd5013cc700 1 --2- 172.21.15.194:0/3475208939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4fc102da0 0x7fd4fc19ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:34.404 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.400+0000 7fd5013cc700 1 -- 172.21.15.194:0/3475208939 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4fc19f270 con 0x7fd4fc101200 2024-03-16T09:08:34.404 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.400+0000 7fd5013cc700 1 --2- 172.21.15.194:0/3475208939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4fc101200 0x7fd4fc197610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80098e0 tx=0x7fd4f800f720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:34.404 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.400+0000 7fd4f27fc700 1 -- 172.21.15.194:0/3475208939 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8027070 con 0x7fd4fc101200 2024-03-16T09:08:34.405 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.400+0000 7fd4f27fc700 1 -- 172.21.15.194:0/3475208939 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd4f800fbd0 con 0x7fd4fc101200 2024-03-16T09:08:34.405 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.400+0000 7fd503630700 1 -- 172.21.15.194:0/3475208939 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4fc069f80 con 0x7fd4fc101200 2024-03-16T09:08:34.405 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.400+0000 7fd4f27fc700 1 -- 172.21.15.194:0/3475208939 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8017770 con 0x7fd4fc101200 2024-03-16T09:08:34.405 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.400+0000 7fd503630700 1 -- 172.21.15.194:0/3475208939 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4fc06a4e0 con 0x7fd4fc101200 2024-03-16T09:08:34.406 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.401+0000 7fd503630700 1 -- 172.21.15.194:0/3475208939 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4fc068f90 con 0x7fd4fc101200 2024-03-16T09:08:34.406 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.403+0000 7fd4f27fc700 1 -- 172.21.15.194:0/3475208939 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 15) ==== 137235+0+0 (secure 0 0 0) 0x7fd4f802a070 con 0x7fd4fc101200 2024-03-16T09:08:34.406 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.403+0000 7fd4f27fc700 1 --2- 172.21.15.194:0/3475208939 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd4e80a2800 0x7fd4e80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:34.406 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.404+0000 7fd4f27fc700 1 -- 172.21.15.194:0/3475208939 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fd4f80ce190 con 0x7fd4fc101200 2024-03-16T09:08:34.406 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.404+0000 7fd500bcb700 1 --2- 172.21.15.194:0/3475208939 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd4e80a2800 0x7fd4e80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:34.406 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.404+0000 7fd500bcb700 1 --2- 172.21.15.194:0/3475208939 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd4e80a2800 0x7fd4e80a4cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec006fd0 tx=0x7fd4ec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:34.409 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.407+0000 7fd4f27fc700 1 -- 172.21.15.194:0/3475208939 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd4f8013070 con 0x7fd4fc101200 2024-03-16T09:08:34.609 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:34.607+0000 7fd503630700 1 -- 172.21.15.194:0/3475208939 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd4fc06a740 con 0x7fd4e80a2800 2024-03-16T09:08:34.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:34 smithi194 ceph-mon[36729]: osdmap e14: 2 total, 2 up, 2 in 2024-03-16T09:08:34.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:34 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:08:34.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:34 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:08:34.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:34 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:34.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:34 smithi159 ceph-mon[36739]: osdmap e14: 2 total, 2 up, 2 in 2024-03-16T09:08:34.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:34 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:08:34.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:34 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:08:34.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:34 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:35.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:34 smithi112 ceph-mon[32307]: osdmap e14: 2 total, 2 up, 2 in 2024-03-16T09:08:35.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:34 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:08:35.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:34 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:08:35.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:34 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:35.825 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:35 smithi194 ceph-mon[36729]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:35.826 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:35 smithi194 ceph-mon[36729]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:35.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:35 smithi159 ceph-mon[36739]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:35.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:35 smithi159 ceph-mon[36739]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:36.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:35 smithi112 ceph-mon[32307]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:36.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:35 smithi112 ceph-mon[32307]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:38.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:37 smithi194 ceph-mon[36729]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:38.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:37 smithi194 ceph-mon[36729]: from='client.? 172.21.15.194:0/282817340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4debd434-e089-4ee8-a944-c22a8405f8be"}]: dispatch 2024-03-16T09:08:38.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:37 smithi194 ceph-mon[36729]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4debd434-e089-4ee8-a944-c22a8405f8be"}]: dispatch 2024-03-16T09:08:38.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:37 smithi194 ceph-mon[36729]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4debd434-e089-4ee8-a944-c22a8405f8be"}]': finished 2024-03-16T09:08:38.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:37 smithi194 ceph-mon[36729]: osdmap e15: 3 total, 2 up, 3 in 2024-03-16T09:08:38.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:37 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:38.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:37 smithi194 ceph-mon[36729]: from='client.? 172.21.15.194:0/1037350563' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:08:38.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:37 smithi159 ceph-mon[36739]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:38.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:37 smithi159 ceph-mon[36739]: from='client.? 172.21.15.194:0/282817340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4debd434-e089-4ee8-a944-c22a8405f8be"}]: dispatch 2024-03-16T09:08:38.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:37 smithi159 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4debd434-e089-4ee8-a944-c22a8405f8be"}]: dispatch 2024-03-16T09:08:38.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:37 smithi159 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4debd434-e089-4ee8-a944-c22a8405f8be"}]': finished 2024-03-16T09:08:38.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:37 smithi159 ceph-mon[36739]: osdmap e15: 3 total, 2 up, 3 in 2024-03-16T09:08:38.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:37 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:38.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:37 smithi159 ceph-mon[36739]: from='client.? 172.21.15.194:0/1037350563' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:08:38.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:37 smithi112 ceph-mon[32307]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:38.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:37 smithi112 ceph-mon[32307]: from='client.? 172.21.15.194:0/282817340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4debd434-e089-4ee8-a944-c22a8405f8be"}]: dispatch 2024-03-16T09:08:38.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:37 smithi112 ceph-mon[32307]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4debd434-e089-4ee8-a944-c22a8405f8be"}]: dispatch 2024-03-16T09:08:38.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:37 smithi112 ceph-mon[32307]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4debd434-e089-4ee8-a944-c22a8405f8be"}]': finished 2024-03-16T09:08:38.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:37 smithi112 ceph-mon[32307]: osdmap e15: 3 total, 2 up, 3 in 2024-03-16T09:08:38.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:37 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:38.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:37 smithi112 ceph-mon[32307]: from='client.? 172.21.15.194:0/1037350563' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:08:40.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:39 smithi194 ceph-mon[36729]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:40.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:39 smithi159 ceph-mon[36739]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:40.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:39 smithi112 ceph-mon[32307]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:42.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:41 smithi194 ceph-mon[36729]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:42.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:41 smithi159 ceph-mon[36739]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:42.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:41 smithi112 ceph-mon[32307]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:44.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:43 smithi194 ceph-mon[36729]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:44.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:43 smithi159 ceph-mon[36739]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:44.341 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:43 smithi112 ceph-mon[32307]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:45.048 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:44 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T09:08:45.048 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:44 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:45.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:44 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T09:08:45.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:44 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:45.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:44 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-16T09:08:45.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:44 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:45.985 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:45 smithi194 ceph-mon[36729]: Deploying daemon osd.2 on smithi194 2024-03-16T09:08:45.985 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:45 smithi194 ceph-mon[36729]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:46.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:45 smithi159 ceph-mon[36739]: Deploying daemon osd.2 on smithi194 2024-03-16T09:08:46.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:45 smithi159 ceph-mon[36739]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:46.341 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:45 smithi112 ceph-mon[32307]: Deploying daemon osd.2 on smithi194 2024-03-16T09:08:46.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:45 smithi112 ceph-mon[32307]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:47.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:47 smithi194 ceph-mon[36729]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:48.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:47 smithi159 ceph-mon[36739]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:47 smithi112 ceph-mon[32307]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:50.103 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:49 smithi194 ceph-mon[36729]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:50.103 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:49 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:50.103 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:49 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:50.103 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:49 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:50.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:49 smithi159 ceph-mon[36739]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:50.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:49 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:50.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:49 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:50.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:49 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:50.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:49 smithi112 ceph-mon[32307]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:50.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:49 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:50.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:49 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:50.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:49 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:52.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:51 smithi194 ceph-mon[36729]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:52.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:51 smithi159 ceph-mon[36739]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:52.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:51 smithi112 ceph-mon[32307]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:52.972 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:52.969+0000 7fd4f27fc700 1 -- 172.21.15.194:0/3475208939 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd4fc06a740 con 0x7fd4e80a2800 2024-03-16T09:08:52.972 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 2 on host 'smithi194' 2024-03-16T09:08:52.974 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:52.971+0000 7fd503630700 1 -- 172.21.15.194:0/3475208939 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd4e80a2800 msgr2=0x7fd4e80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:52.974 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:52.972+0000 7fd503630700 1 --2- 172.21.15.194:0/3475208939 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd4e80a2800 0x7fd4e80a4cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec006fd0 tx=0x7fd4ec008040 comp rx=0 tx=0).stop 2024-03-16T09:08:52.974 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:52.972+0000 7fd503630700 1 -- 172.21.15.194:0/3475208939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4fc101200 msgr2=0x7fd4fc197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:52.974 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:52.972+0000 7fd503630700 1 --2- 172.21.15.194:0/3475208939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4fc101200 0x7fd4fc197610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80098e0 tx=0x7fd4f800f720 comp rx=0 tx=0).stop 2024-03-16T09:08:52.974 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:52.972+0000 7fd503630700 1 -- 172.21.15.194:0/3475208939 shutdown_connections 2024-03-16T09:08:52.974 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:52.972+0000 7fd503630700 1 --2- 172.21.15.194:0/3475208939 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd4e80a2800 0x7fd4e80a4cc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:52.974 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:52.972+0000 7fd503630700 1 --2- 172.21.15.194:0/3475208939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4fc102da0 0x7fd4fc19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:52.974 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:52.972+0000 7fd503630700 1 --2- 172.21.15.194:0/3475208939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4fc101200 0x7fd4fc197610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:52.974 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:52.972+0000 7fd503630700 1 --2- 172.21.15.194:0/3475208939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4fc102400 0x7fd4fc197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:52.975 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:52.972+0000 7fd503630700 1 -- 172.21.15.194:0/3475208939 >> 172.21.15.194:0/3475208939 conn(0x7fd4fc0fc7a0 msgr2=0x7fd4fc105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:52.975 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:52.972+0000 7fd503630700 1 -- 172.21.15.194:0/3475208939 shutdown_connections 2024-03-16T09:08:52.975 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:08:52.972+0000 7fd503630700 1 -- 172.21.15.194:0/3475208939 wait complete. 2024-03-16T09:08:53.277 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:53 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:53.277 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:53 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:53.277 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:53 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:53.277 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:53 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:53.277 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:53 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:53.277 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:53 smithi194 ceph-mon[36729]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:53.277 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:53 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:53.277 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:53 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:53.277 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:53 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:53.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:53 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:53.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:53 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:53.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:53 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:53.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:53 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:53.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:53 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:53.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:53 smithi159 ceph-mon[36739]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:53.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:53 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:53.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:53 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:53.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:53 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:53.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:53 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:53.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:53 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:53.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:53 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:53.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:53 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:53.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:53 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:53.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:53 smithi112 ceph-mon[32307]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:53.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:53 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:53.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:53 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:53.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:53 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:08:53.628 DEBUG:teuthology.orchestra.run.smithi194:osd.2> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.2.service 2024-03-16T09:08:53.631 INFO:tasks.cephadm:Deploying osd.3 on smithi112 with /dev/vg_nvme/lv_3... 2024-03-16T09:08:53.631 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-16T09:08:53.876 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:08:54.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:54 smithi112 ceph-mon[32307]: from='osd.2 [v2:172.21.15.194:6800/4037530979,v1:172.21.15.194:6801/4037530979]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T09:08:54.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:54 smithi112 ceph-mon[32307]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T09:08:54.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:54 smithi194 ceph-mon[36729]: from='osd.2 [v2:172.21.15.194:6800/4037530979,v1:172.21.15.194:6801/4037530979]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T09:08:54.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:54 smithi194 ceph-mon[36729]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T09:08:54.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:54 smithi159 ceph-mon[36739]: from='osd.2 [v2:172.21.15.194:6800/4037530979,v1:172.21.15.194:6801/4037530979]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T09:08:54.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:54 smithi159 ceph-mon[36739]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-16T09:08:55.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:55 smithi112 ceph-mon[32307]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-16T09:08:55.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:55 smithi112 ceph-mon[32307]: osdmap e16: 3 total, 2 up, 3 in 2024-03-16T09:08:55.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:55 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:55.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:55 smithi112 ceph-mon[32307]: from='osd.2 [v2:172.21.15.194:6800/4037530979,v1:172.21.15.194:6801/4037530979]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:08:55.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:55 smithi112 ceph-mon[32307]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:08:55.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:55 smithi112 ceph-mon[32307]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:55.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:55 smithi159 ceph-mon[36739]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-16T09:08:55.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:55 smithi159 ceph-mon[36739]: osdmap e16: 3 total, 2 up, 3 in 2024-03-16T09:08:55.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:55 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:55.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:55 smithi159 ceph-mon[36739]: from='osd.2 [v2:172.21.15.194:6800/4037530979,v1:172.21.15.194:6801/4037530979]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:08:55.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:55 smithi159 ceph-mon[36739]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:08:55.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:55 smithi159 ceph-mon[36739]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:55.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:55 smithi194 ceph-mon[36729]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-16T09:08:55.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:55 smithi194 ceph-mon[36729]: osdmap e16: 3 total, 2 up, 3 in 2024-03-16T09:08:55.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:55 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:55.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:55 smithi194 ceph-mon[36729]: from='osd.2 [v2:172.21.15.194:6800/4037530979,v1:172.21.15.194:6801/4037530979]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:08:55.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:55 smithi194 ceph-mon[36729]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:08:55.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:55 smithi194 ceph-mon[36729]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:55.961 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:08:55 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-2[41896]: 2024-03-16T09:08:55.690+0000 7f39feece700 -1 osd.2 0 waiting for initial osdmap 2024-03-16T09:08:55.962 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:08:55 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-2[41896]: 2024-03-16T09:08:55.694+0000 7f39f6cb9700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T09:08:56.029 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-16T09:08:56.030 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-16T09:08:56.030 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-03-16T09:08:56.030 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-03-16T09:08:56.030 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131102 s, 800 MB/s 2024-03-16T09:08:56.030 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-03-16T09:08:56.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:56 smithi112 ceph-mon[32307]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T09:08:56.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:56 smithi112 ceph-mon[32307]: osdmap e17: 3 total, 2 up, 3 in 2024-03-16T09:08:56.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:56.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:56.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:56 smithi112 ceph-mon[32307]: from='osd.2 ' entity='osd.2' 2024-03-16T09:08:56.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:56 smithi159 ceph-mon[36739]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T09:08:56.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:56 smithi159 ceph-mon[36739]: osdmap e17: 3 total, 2 up, 3 in 2024-03-16T09:08:56.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:56.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:56.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:56 smithi159 ceph-mon[36739]: from='osd.2 ' entity='osd.2' 2024-03-16T09:08:56.616 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:56 smithi194 ceph-mon[36729]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T09:08:56.616 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:56 smithi194 ceph-mon[36729]: osdmap e17: 3 total, 2 up, 3 in 2024-03-16T09:08:56.616 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:56 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:56.616 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:56 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:56.616 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:56 smithi194 ceph-mon[36729]: from='osd.2 ' entity='osd.2' 2024-03-16T09:08:56.736 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi112:vg_nvme/lv_3 2024-03-16T09:08:56.988 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:08:57.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:57 smithi112 ceph-mon[32307]: purged_snaps scrub starts 2024-03-16T09:08:57.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:57 smithi112 ceph-mon[32307]: purged_snaps scrub ok 2024-03-16T09:08:57.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:57.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:57 smithi112 ceph-mon[32307]: Detected new or changed devices on smithi194 2024-03-16T09:08:57.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:57.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:57.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:08:57.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:57 smithi112 ceph-mon[32307]: Adjusting osd_memory_target on smithi194 to 17080M 2024-03-16T09:08:57.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:57.455 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:57.455 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:57.455 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:57.455 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:57 smithi112 ceph-mon[32307]: osd.2 [v2:172.21.15.194:6800/4037530979,v1:172.21.15.194:6801/4037530979] boot 2024-03-16T09:08:57.455 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:57 smithi112 ceph-mon[32307]: osdmap e18: 3 total, 3 up, 3 in 2024-03-16T09:08:57.455 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:57.455 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:57 smithi112 ceph-mon[32307]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:57.455 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-16T09:08:57.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:57 smithi159 ceph-mon[36739]: purged_snaps scrub starts 2024-03-16T09:08:57.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:57 smithi159 ceph-mon[36739]: purged_snaps scrub ok 2024-03-16T09:08:57.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:57.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:57 smithi159 ceph-mon[36739]: Detected new or changed devices on smithi194 2024-03-16T09:08:57.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:57.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:57.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:08:57.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:57 smithi159 ceph-mon[36739]: Adjusting osd_memory_target on smithi194 to 17080M 2024-03-16T09:08:57.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:57.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:57.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:57.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:57.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:57 smithi159 ceph-mon[36739]: osd.2 [v2:172.21.15.194:6800/4037530979,v1:172.21.15.194:6801/4037530979] boot 2024-03-16T09:08:57.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:57 smithi159 ceph-mon[36739]: osdmap e18: 3 total, 3 up, 3 in 2024-03-16T09:08:57.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:57.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:57 smithi159 ceph-mon[36739]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:57.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-16T09:08:57.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:57 smithi194 ceph-mon[36729]: purged_snaps scrub starts 2024-03-16T09:08:57.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:57 smithi194 ceph-mon[36729]: purged_snaps scrub ok 2024-03-16T09:08:57.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:57.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:57 smithi194 ceph-mon[36729]: Detected new or changed devices on smithi194 2024-03-16T09:08:57.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:57.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:57.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:08:57.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:57 smithi194 ceph-mon[36729]: Adjusting osd_memory_target on smithi194 to 17080M 2024-03-16T09:08:57.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:57.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:57.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:08:57.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:08:57.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:57 smithi194 ceph-mon[36729]: osd.2 [v2:172.21.15.194:6800/4037530979,v1:172.21.15.194:6801/4037530979] boot 2024-03-16T09:08:57.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:57 smithi194 ceph-mon[36729]: osdmap e18: 3 total, 3 up, 3 in 2024-03-16T09:08:57.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-16T09:08:57.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:57 smithi194 ceph-mon[36729]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-16T09:08:57.653 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-16T09:08:58.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.678+0000 7f1f9bbca700 1 -- 172.21.15.112:0/3468299897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f94105730 msgr2=0x7f1f94107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:58.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.678+0000 7f1f9bbca700 1 --2- 172.21.15.112:0/3468299897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f94105730 0x7f1f94107b20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009990 tx=0x7f1f84009ca0 comp rx=0 tx=0).stop 2024-03-16T09:08:58.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.678+0000 7f1f9bbca700 1 -- 172.21.15.112:0/3468299897 shutdown_connections 2024-03-16T09:08:58.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.678+0000 7f1f9bbca700 1 --2- 172.21.15.112:0/3468299897 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f94108060 0x7f1f9410a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:58.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.678+0000 7f1f9bbca700 1 --2- 172.21.15.112:0/3468299897 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f94102e00 0x7f1f941051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:58.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.678+0000 7f1f9bbca700 1 --2- 172.21.15.112:0/3468299897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f94105730 0x7f1f94107b20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:58.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.678+0000 7f1f9bbca700 1 -- 172.21.15.112:0/3468299897 >> 172.21.15.112:0/3468299897 conn(0x7f1f940fc830 msgr2=0x7f1f940fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:08:58.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.679+0000 7f1f9bbca700 1 -- 172.21.15.112:0/3468299897 shutdown_connections 2024-03-16T09:08:58.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.679+0000 7f1f9bbca700 1 -- 172.21.15.112:0/3468299897 wait complete. 2024-03-16T09:08:58.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.679+0000 7f1f9bbca700 1 Processor -- start 2024-03-16T09:08:58.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.679+0000 7f1f9bbca700 1 -- start start 2024-03-16T09:08:58.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.679+0000 7f1f9bbca700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f94102e00 0x7f1f94102760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:58.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.679+0000 7f1f9bbca700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f94105730 0x7f1f94100db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.679+0000 7f1f9bbca700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f94108060 0x7f1f941012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.679+0000 7f1f9bbca700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9410e090 con 0x7f1f94105730 2024-03-16T09:08:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.679+0000 7f1f9bbca700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9410df10 con 0x7f1f94102e00 2024-03-16T09:08:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.679+0000 7f1f9bbca700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9410e210 con 0x7f1f94108060 2024-03-16T09:08:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.679+0000 7f1f99165700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f94105730 0x7f1f94100db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.679+0000 7f1f99165700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f94105730 0x7f1f94100db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47098/0 (socket says 172.21.15.112:47098) 2024-03-16T09:08:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.679+0000 7f1f99165700 1 -- 172.21.15.112:0/2355452373 learned_addr learned my addr 172.21.15.112:0/2355452373 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:08:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.680+0000 7f1f99966700 1 --2- 172.21.15.112:0/2355452373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f94102e00 0x7f1f94102760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.680+0000 7f1f9a167700 1 --2- 172.21.15.112:0/2355452373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f94108060 0x7f1f941012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.680+0000 7f1f99165700 1 -- 172.21.15.112:0/2355452373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f94108060 msgr2=0x7f1f941012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.680+0000 7f1f99165700 1 --2- 172.21.15.112:0/2355452373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f94108060 0x7f1f941012f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:58.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.680+0000 7f1f99165700 1 -- 172.21.15.112:0/2355452373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f94102e00 msgr2=0x7f1f94102760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:08:58.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.680+0000 7f1f99165700 1 --2- 172.21.15.112:0/2355452373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f94102e00 0x7f1f94102760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:08:58.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.680+0000 7f1f99165700 1 -- 172.21.15.112:0/2355452373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f94101ba0 con 0x7f1f94105730 2024-03-16T09:08:58.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.680+0000 7f1f9a167700 1 --2- 172.21.15.112:0/2355452373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f94108060 0x7f1f941012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:08:58.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.680+0000 7f1f99165700 1 --2- 172.21.15.112:0/2355452373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f94105730 0x7f1f94100db0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1f84005910 tx=0x7f1f8400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:58.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.680+0000 7f1f8affd700 1 -- 172.21.15.112:0/2355452373 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84027070 con 0x7f1f94105730 2024-03-16T09:08:58.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.680+0000 7f1f8affd700 1 -- 172.21.15.112:0/2355452373 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1f8400fbf0 con 0x7f1f94105730 2024-03-16T09:08:58.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.680+0000 7f1f9bbca700 1 -- 172.21.15.112:0/2355452373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f941a2760 con 0x7f1f94105730 2024-03-16T09:08:58.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.680+0000 7f1f8affd700 1 -- 172.21.15.112:0/2355452373 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84017650 con 0x7f1f94105730 2024-03-16T09:08:58.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.680+0000 7f1f9bbca700 1 -- 172.21.15.112:0/2355452373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f941a2bb0 con 0x7f1f94105730 2024-03-16T09:08:58.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.681+0000 7f1f9bbca700 1 -- 172.21.15.112:0/2355452373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f94068f90 con 0x7f1f94105730 2024-03-16T09:08:58.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.682+0000 7f1f8affd700 1 -- 172.21.15.112:0/2355452373 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 15) ==== 137235+0+0 (secure 0 0 0) 0x7f1f8402a070 con 0x7f1f94105730 2024-03-16T09:08:58.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.683+0000 7f1f8affd700 1 --2- 172.21.15.112:0/2355452373 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1f800a2800 0x7f1f800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:08:58.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.683+0000 7f1f99966700 1 --2- 172.21.15.112:0/2355452373 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1f800a2800 0x7f1f800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:08:58.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.683+0000 7f1f8affd700 1 -- 172.21.15.112:0/2355452373 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f1f840ce830 con 0x7f1f94105730 2024-03-16T09:08:58.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.683+0000 7f1f99966700 1 --2- 172.21.15.112:0/2355452373 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1f800a2800 0x7f1f800a4cc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1f90006fd0 tx=0x7f1f90008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:08:58.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.687+0000 7f1f8affd700 1 -- 172.21.15.112:0/2355452373 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1f84031080 con 0x7f1f94105730 2024-03-16T09:08:58.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:08:58.911+0000 7f1f9bbca700 1 -- 172.21.15.112:0/2355452373 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1f94062710 con 0x7f1f800a2800 2024-03-16T09:08:58.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:58 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-16T09:08:58.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:58 smithi159 ceph-mon[36739]: osdmap e19: 3 total, 3 up, 3 in 2024-03-16T09:08:58.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:58 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T09:08:59.060 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:58 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-16T09:08:59.060 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:58 smithi112 ceph-mon[32307]: osdmap e19: 3 total, 3 up, 3 in 2024-03-16T09:08:59.060 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:58 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T09:08:59.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:58 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-16T09:08:59.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:58 smithi194 ceph-mon[36729]: osdmap e19: 3 total, 3 up, 3 in 2024-03-16T09:08:59.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:58 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T09:08:59.326 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:08:59 smithi112 sudo[51663]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T09:08:59.326 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:08:59 smithi112 sudo[51663]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T09:08:59.326 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:08:59 smithi112 sudo[51663]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T09:08:59.326 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:08:59 smithi112 sudo[51663]: pam_unix(sudo:session): session closed for user root 2024-03-16T09:08:59.594 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:08:59 smithi112 sudo[51666]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T09:08:59.594 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:08:59 smithi112 sudo[51666]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T09:08:59.594 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:08:59 smithi112 sudo[51666]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T09:08:59.595 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:08:59 smithi112 sudo[51666]: pam_unix(sudo:session): session closed for user root 2024-03-16T09:08:59.851 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:08:59 smithi159 sudo[45183]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T09:08:59.851 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:08:59 smithi159 sudo[45183]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T09:08:59.851 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:08:59 smithi159 sudo[45183]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T09:08:59.851 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:08:59 smithi159 sudo[45183]: pam_unix(sudo:session): session closed for user root 2024-03-16T09:08:59.851 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:59 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-16T09:08:59.851 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:59 smithi159 ceph-mon[36739]: osdmap e20: 3 total, 3 up, 3 in 2024-03-16T09:08:59.852 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:59 smithi159 ceph-mon[36739]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:08:59.852 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:59 smithi159 ceph-mon[36739]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:08:59.852 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:59 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:08:59.852 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:59 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:08:59.852 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:59 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:08:59.852 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:08:59 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:00.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-16T09:09:00.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:59 smithi112 ceph-mon[32307]: osdmap e20: 3 total, 3 up, 3 in 2024-03-16T09:09:00.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:59 smithi112 ceph-mon[32307]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:00.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:59 smithi112 ceph-mon[32307]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:00.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:09:00.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:09:00.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:00.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:08:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:00.120 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:59 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-16T09:09:00.121 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:59 smithi194 ceph-mon[36729]: osdmap e20: 3 total, 3 up, 3 in 2024-03-16T09:09:00.121 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:59 smithi194 ceph-mon[36729]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:00.121 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:59 smithi194 ceph-mon[36729]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:00.121 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:59 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:09:00.121 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:59 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:09:00.121 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:59 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:00.121 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:08:59 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:00.236 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:08:59 smithi159 sudo[45186]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T09:09:00.236 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:08:59 smithi159 sudo[45186]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T09:09:00.236 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:08:59 smithi159 sudo[45186]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T09:09:00.236 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:08:59 smithi159 sudo[45186]: pam_unix(sudo:session): session closed for user root 2024-03-16T09:09:00.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:09:00.357+0000 7f1f8affd700 1 -- 172.21.15.112:0/2355452373 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f1f84098d90 con 0x7f1f94105730 2024-03-16T09:09:00.375 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:09:00 smithi194 sudo[45997]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T09:09:00.375 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:09:00 smithi194 sudo[45997]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T09:09:00.375 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:09:00 smithi194 sudo[45997]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T09:09:00.375 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:09:00 smithi194 sudo[45997]: pam_unix(sudo:session): session closed for user root 2024-03-16T09:09:00.652 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:09:00 smithi194 sudo[46000]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T09:09:00.652 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:09:00 smithi194 sudo[46000]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T09:09:00.652 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:09:00 smithi194 sudo[46000]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T09:09:00.652 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:09:00 smithi194 sudo[46000]: pam_unix(sudo:session): session closed for user root 2024-03-16T09:09:00.907 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:00 smithi112 sudo[51769]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T09:09:00.907 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:00 smithi112 sudo[51769]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T09:09:00.908 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:00 smithi112 sudo[51769]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T09:09:00.908 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:00 smithi112 sudo[51769]: pam_unix(sudo:session): session closed for user root 2024-03-16T09:09:01.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:00 smithi194 ceph-mon[36729]: osdmap e21: 3 total, 3 up, 3 in 2024-03-16T09:09:01.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:00 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:01.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:00 smithi194 ceph-mon[36729]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-16T09:09:01.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:00 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:01.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:00 smithi194 ceph-mon[36729]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T09:09:01.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 sudo[46003]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T09:09:01.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 sudo[46003]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T09:09:01.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 sudo[46003]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T09:09:01.166 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 sudo[46003]: pam_unix(sudo:session): session closed for user root 2024-03-16T09:09:01.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:00 smithi159 ceph-mon[36739]: osdmap e21: 3 total, 3 up, 3 in 2024-03-16T09:09:01.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:00 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:01.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:00 smithi159 ceph-mon[36739]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-16T09:09:01.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:00 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:01.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:00 smithi159 ceph-mon[36739]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T09:09:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:00 smithi112 sudo[51776]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T09:09:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:00 smithi112 sudo[51776]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T09:09:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:00 smithi112 sudo[51776]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T09:09:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:00 smithi112 sudo[51776]: pam_unix(sudo:session): session closed for user root 2024-03-16T09:09:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:00 smithi112 ceph-mon[32307]: osdmap e21: 3 total, 3 up, 3 in 2024-03-16T09:09:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:00 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:00 smithi112 ceph-mon[32307]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-16T09:09:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:00 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:01.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:00 smithi112 ceph-mon[32307]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T09:09:01.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 sudo[46006]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T09:09:01.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 sudo[46006]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T09:09:01.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 sudo[46006]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T09:09:01.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 sudo[46006]: pam_unix(sudo:session): session closed for user root 2024-03-16T09:09:01.893 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 sudo[45189]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-16T09:09:01.894 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 sudo[45189]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T09:09:01.894 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 sudo[45189]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T09:09:01.894 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 sudo[45189]: pam_unix(sudo:session): session closed for user root 2024-03-16T09:09:01.894 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 sudo[45192]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-16T09:09:01.894 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 sudo[45192]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-16T09:09:01.894 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 sudo[45192]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-16T09:09:01.894 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 sudo[45192]: pam_unix(sudo:session): session closed for user root 2024-03-16T09:09:02.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 ceph-mon[36739]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:02.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 ceph-mon[36739]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T09:09:02.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:09:02.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:09:02.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:09:02.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 ceph-mon[36739]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T09:09:02.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:09:02.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:09:02.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:09:02.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 ceph-mon[36739]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T09:09:02.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 ceph-mon[36739]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T09:09:02.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:09:02.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:09:02.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:09:02.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3535817874' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f81eddbc-f120-4077-a6f5-2bee53c270e2"}]: dispatch 2024-03-16T09:09:02.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 ceph-mon[36739]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T09:09:02.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3535817874' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f81eddbc-f120-4077-a6f5-2bee53c270e2"}]': finished 2024-03-16T09:09:02.238 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 ceph-mon[36739]: osdmap e22: 4 total, 3 up, 4 in 2024-03-16T09:09:02.238 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:09:02.238 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:01 smithi112 ceph-mon[32307]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:02.238 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:01 smithi112 ceph-mon[32307]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T09:09:02.238 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:09:02.238 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:09:02.238 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:09:02.238 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:01 smithi112 ceph-mon[32307]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T09:09:02.238 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:09:02.238 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:09:02.239 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:09:02.239 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:01 smithi112 ceph-mon[32307]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T09:09:02.239 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:01 smithi112 ceph-mon[32307]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T09:09:02.239 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:09:02.239 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:09:02.239 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:09:02.239 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:01 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3535817874' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f81eddbc-f120-4077-a6f5-2bee53c270e2"}]: dispatch 2024-03-16T09:09:02.239 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:01 smithi112 ceph-mon[32307]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T09:09:02.239 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:01 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3535817874' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f81eddbc-f120-4077-a6f5-2bee53c270e2"}]': finished 2024-03-16T09:09:02.239 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:01 smithi112 ceph-mon[32307]: osdmap e22: 4 total, 3 up, 4 in 2024-03-16T09:09:02.239 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:09:02.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 ceph-mon[36729]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:02.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 ceph-mon[36729]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T09:09:02.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:09:02.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:09:02.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:09:02.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 ceph-mon[36729]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T09:09:02.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:09:02.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:09:02.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:09:02.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 ceph-mon[36729]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T09:09:02.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 ceph-mon[36729]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-16T09:09:02.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-16T09:09:02.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-16T09:09:02.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-16T09:09:02.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3535817874' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f81eddbc-f120-4077-a6f5-2bee53c270e2"}]: dispatch 2024-03-16T09:09:02.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 ceph-mon[36729]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-16T09:09:02.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3535817874' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f81eddbc-f120-4077-a6f5-2bee53c270e2"}]': finished 2024-03-16T09:09:02.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 ceph-mon[36729]: osdmap e22: 4 total, 3 up, 4 in 2024-03-16T09:09:02.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:09:03.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:02 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2094279584' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:09:03.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:02 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2094279584' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:09:03.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:02 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2094279584' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:09:04.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:03 smithi159 ceph-mon[36739]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:04.341 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:03 smithi112 ceph-mon[32307]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:04.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:03 smithi194 ceph-mon[36729]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:06.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:05 smithi112 ceph-mon[32307]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:06.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:05 smithi159 ceph-mon[36739]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:06.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:05 smithi194 ceph-mon[36729]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:08.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:07 smithi112 ceph-mon[32307]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:08.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:07 smithi159 ceph-mon[36739]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:08.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:07 smithi194 ceph-mon[36729]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:10.020 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:09 smithi112 ceph-mon[32307]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:10.020 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:09 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T09:09:10.020 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:09 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:10.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:09 smithi159 ceph-mon[36739]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:10.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:09 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T09:09:10.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:09 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:10.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:09 smithi194 ceph-mon[36729]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:10.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:09 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-16T09:09:10.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:09 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:11.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:10 smithi112 ceph-mon[32307]: Deploying daemon osd.3 on smithi112 2024-03-16T09:09:11.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:10 smithi159 ceph-mon[36739]: Deploying daemon osd.3 on smithi112 2024-03-16T09:09:11.303 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:10 smithi194 ceph-mon[36729]: Deploying daemon osd.3 on smithi112 2024-03-16T09:09:12.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:11 smithi159 ceph-mon[36739]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:12.240 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:11 smithi112 ceph-mon[32307]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:12.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:11 smithi194 ceph-mon[36729]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:14.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:13 smithi159 ceph-mon[36739]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:14.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:13 smithi112 ceph-mon[32307]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:14.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:13 smithi194 ceph-mon[36729]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:16.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:16 smithi112 ceph-mon[32307]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:16.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:16 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:16.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:16 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:16.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:16 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:16.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:16 smithi194 ceph-mon[36729]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:16.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:16 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:16.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:16 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:16.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:16 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:16.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:16 smithi159 ceph-mon[36739]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:16.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:16 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:16.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:16 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:16.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:16 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:18 smithi112 ceph-mon[32307]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:18 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:18 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:18 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:18 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:09:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:18 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:18.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:18 smithi194 ceph-mon[36729]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:18 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:18 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:18 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:18 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:09:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:18 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:18.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:18 smithi159 ceph-mon[36739]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:18.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:18 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:18.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:18 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:18.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:18 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:18.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:18 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:09:18.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:18 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:18.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:09:18.717+0000 7f1f8affd700 1 -- 172.21.15.112:0/2355452373 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1f94062710 con 0x7f1f800a2800 2024-03-16T09:09:18.720 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 3 on host 'smithi112' 2024-03-16T09:09:18.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:09:18.720+0000 7f1f9bbca700 1 -- 172.21.15.112:0/2355452373 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1f800a2800 msgr2=0x7f1f800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:18.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:09:18.720+0000 7f1f9bbca700 1 --2- 172.21.15.112:0/2355452373 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1f800a2800 0x7f1f800a4cc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1f90006fd0 tx=0x7f1f90008040 comp rx=0 tx=0).stop 2024-03-16T09:09:18.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:09:18.720+0000 7f1f9bbca700 1 -- 172.21.15.112:0/2355452373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f94105730 msgr2=0x7f1f94100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:18.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:09:18.720+0000 7f1f9bbca700 1 --2- 172.21.15.112:0/2355452373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f94105730 0x7f1f94100db0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1f84005910 tx=0x7f1f8400f740 comp rx=0 tx=0).stop 2024-03-16T09:09:18.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:09:18.720+0000 7f1f9bbca700 1 -- 172.21.15.112:0/2355452373 shutdown_connections 2024-03-16T09:09:18.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:09:18.720+0000 7f1f9bbca700 1 --2- 172.21.15.112:0/2355452373 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1f800a2800 0x7f1f800a4cc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:18.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:09:18.720+0000 7f1f9bbca700 1 --2- 172.21.15.112:0/2355452373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f94102e00 0x7f1f94102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:18.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:09:18.720+0000 7f1f9bbca700 1 --2- 172.21.15.112:0/2355452373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f94108060 0x7f1f941012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:18.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:09:18.720+0000 7f1f9bbca700 1 --2- 172.21.15.112:0/2355452373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f94105730 0x7f1f94100db0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:18.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:09:18.720+0000 7f1f9bbca700 1 -- 172.21.15.112:0/2355452373 >> 172.21.15.112:0/2355452373 conn(0x7f1f940fc830 msgr2=0x7f1f94103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:18.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:09:18.720+0000 7f1f9bbca700 1 -- 172.21.15.112:0/2355452373 shutdown_connections 2024-03-16T09:09:18.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:09:18.720+0000 7f1f9bbca700 1 -- 172.21.15.112:0/2355452373 wait complete. 2024-03-16T09:09:19.356 DEBUG:teuthology.orchestra.run.smithi112:osd.3> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.3.service 2024-03-16T09:09:19.359 INFO:tasks.cephadm:Deploying osd.4 on smithi159 with /dev/vg_nvme/lv_3... 2024-03-16T09:09:19.359 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-16T09:09:19.604 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T09:09:19.951 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:19 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:19.951 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:19 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:19.951 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:19 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:19.951 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:19 smithi159 ceph-mon[36739]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:19.951 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:19 smithi159 ceph-mon[36739]: from='osd.3 [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T09:09:20.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:19 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:20.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:19 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:20.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:19 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:20.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:19 smithi112 ceph-mon[32307]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:20.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:19 smithi112 ceph-mon[32307]: from='osd.3 [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T09:09:20.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:19 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:20.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:19 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:20.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:19 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:20.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:19 smithi194 ceph-mon[36729]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:20.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:19 smithi194 ceph-mon[36729]: from='osd.3 [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-16T09:09:20.798 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:20 smithi112 ceph-mon[32307]: from='osd.3 [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-16T09:09:20.798 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:20 smithi112 ceph-mon[32307]: osdmap e23: 4 total, 3 up, 4 in 2024-03-16T09:09:20.798 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:20 smithi112 ceph-mon[32307]: from='osd.3 [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-03-16T09:09:20.798 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:09:20.798 INFO:journalctl@ceph.osd.3.smithi112.stdout:Mar 16 09:09:20 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-3[53870]: 2024-03-16T09:09:20.709+0000 7f1094ce7700 -1 osd.3 0 waiting for initial osdmap 2024-03-16T09:09:20.798 INFO:journalctl@ceph.osd.3.smithi112.stdout:Mar 16 09:09:20 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-3[53870]: 2024-03-16T09:09:20.714+0000 7f1090ada700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T09:09:20.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:20 smithi159 ceph-mon[36739]: from='osd.3 [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-16T09:09:20.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:20 smithi159 ceph-mon[36739]: osdmap e23: 4 total, 3 up, 4 in 2024-03-16T09:09:20.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:20 smithi159 ceph-mon[36739]: from='osd.3 [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-03-16T09:09:20.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:20 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:09:21.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:20 smithi194 ceph-mon[36729]: from='osd.3 [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-16T09:09:21.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:20 smithi194 ceph-mon[36729]: osdmap e23: 4 total, 3 up, 4 in 2024-03-16T09:09:21.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:20 smithi194 ceph-mon[36729]: from='osd.3 [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-03-16T09:09:21.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:09:21.593 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-16T09:09:21.593 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-16T09:09:21.593 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2024-03-16T09:09:21.593 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2024-03-16T09:09:21.594 INFO:teuthology.orchestra.run.smithi159.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113378 s, 925 MB/s 2024-03-16T09:09:21.594 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 2024-03-16T09:09:21.594 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2024-03-16T09:09:21.902 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:21 smithi159 ceph-mon[36739]: from='osd.3 [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-03-16T09:09:21.903 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:21 smithi159 ceph-mon[36739]: osdmap e24: 4 total, 3 up, 4 in 2024-03-16T09:09:21.903 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:21 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:09:21.903 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:21 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:09:21.903 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:21 smithi159 ceph-mon[36739]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:21.918 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:21 smithi112 ceph-mon[32307]: from='osd.3 [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-03-16T09:09:21.918 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:21 smithi112 ceph-mon[32307]: osdmap e24: 4 total, 3 up, 4 in 2024-03-16T09:09:21.918 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:21 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:09:21.919 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:21 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:09:21.919 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:21 smithi112 ceph-mon[32307]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:21.999 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:21 smithi194 ceph-mon[36729]: from='osd.3 [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-03-16T09:09:21.999 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:21 smithi194 ceph-mon[36729]: osdmap e24: 4 total, 3 up, 4 in 2024-03-16T09:09:21.999 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:21 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:09:21.999 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:21 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:09:22.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:21 smithi194 ceph-mon[36729]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-16T09:09:22.224 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi159:vg_nvme/lv_3 2024-03-16T09:09:22.478 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T09:09:22.840 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:22 smithi159 ceph-mon[36739]: purged_snaps scrub starts 2024-03-16T09:09:22.840 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:22 smithi159 ceph-mon[36739]: purged_snaps scrub ok 2024-03-16T09:09:22.840 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:22 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:09:22.840 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:22 smithi159 ceph-mon[36739]: osd.3 [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464] boot 2024-03-16T09:09:22.840 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:22 smithi159 ceph-mon[36739]: osdmap e25: 4 total, 4 up, 4 in 2024-03-16T09:09:22.841 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:22 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:09:22.841 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:22 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:22.841 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:22 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:22.841 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:22 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:09:22.841 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:22 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:09:22.841 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:22 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:22.841 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:22 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:22.841 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:22 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:09:22.841 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:22 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:22.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:22 smithi194 ceph-mon[36729]: purged_snaps scrub starts 2024-03-16T09:09:22.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:22 smithi194 ceph-mon[36729]: purged_snaps scrub ok 2024-03-16T09:09:22.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:22 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:09:22.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:22 smithi194 ceph-mon[36729]: osd.3 [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464] boot 2024-03-16T09:09:22.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:22 smithi194 ceph-mon[36729]: osdmap e25: 4 total, 4 up, 4 in 2024-03-16T09:09:22.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:22 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:09:22.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:22 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:22.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:22 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:22.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:22 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:09:22.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:22 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:09:22.903 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:22 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:22.903 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:22 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:22.903 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:22 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:09:22.903 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:22 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:23.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:22 smithi112 ceph-mon[32307]: purged_snaps scrub starts 2024-03-16T09:09:23.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:22 smithi112 ceph-mon[32307]: purged_snaps scrub ok 2024-03-16T09:09:23.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:22 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:09:23.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:22 smithi112 ceph-mon[32307]: osd.3 [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464] boot 2024-03-16T09:09:23.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:22 smithi112 ceph-mon[32307]: osdmap e25: 4 total, 4 up, 4 in 2024-03-16T09:09:23.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:22 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-16T09:09:23.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:22 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:23.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:22 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:23.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:22 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:09:23.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:22 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:09:23.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:22 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:23.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:22 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:23.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:22 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:09:23.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:22 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:23.909 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:23 smithi159 ceph-mon[36739]: Detected new or changed devices on smithi112 2024-03-16T09:09:23.909 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:23 smithi159 ceph-mon[36739]: Adjusting osd_memory_target on smithi112 to 8544M 2024-03-16T09:09:23.909 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:23 smithi159 ceph-mon[36739]: osdmap e26: 4 total, 4 up, 4 in 2024-03-16T09:09:23.909 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:23 smithi159 ceph-mon[36739]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:24.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:23 smithi112 ceph-mon[32307]: Detected new or changed devices on smithi112 2024-03-16T09:09:24.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:23 smithi112 ceph-mon[32307]: Adjusting osd_memory_target on smithi112 to 8544M 2024-03-16T09:09:24.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:23 smithi112 ceph-mon[32307]: osdmap e26: 4 total, 4 up, 4 in 2024-03-16T09:09:24.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:23 smithi112 ceph-mon[32307]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:24.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.122+0000 7f39155d0700 1 -- 172.21.15.159:0/3363773447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3910103ea0 msgr2=0x7f391010a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:24.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.122+0000 7f39155d0700 1 --2- 172.21.15.159:0/3363773447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3910103ea0 0x7f391010a750 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f390800b0d0 tx=0x7f390800b3e0 comp rx=0 tx=0).stop 2024-03-16T09:09:24.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.122+0000 7f39155d0700 1 -- 172.21.15.159:0/3363773447 shutdown_connections 2024-03-16T09:09:24.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.122+0000 7f39155d0700 1 --2- 172.21.15.159:0/3363773447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3910103ea0 0x7f391010a750 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:24.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.122+0000 7f39155d0700 1 --2- 172.21.15.159:0/3363773447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3910102300 0x7f3910102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:24.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.122+0000 7f39155d0700 1 --2- 172.21.15.159:0/3363773447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3910103500 0x7f3910103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:24.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.122+0000 7f39155d0700 1 -- 172.21.15.159:0/3363773447 >> 172.21.15.159:0/3363773447 conn(0x7f39100fd8e0 msgr2=0x7f39100ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:24.125 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.123+0000 7f39155d0700 1 -- 172.21.15.159:0/3363773447 shutdown_connections 2024-03-16T09:09:24.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.123+0000 7f39155d0700 1 -- 172.21.15.159:0/3363773447 wait complete. 2024-03-16T09:09:24.126 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.123+0000 7f39155d0700 1 Processor -- start 2024-03-16T09:09:24.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.123+0000 7f39155d0700 1 -- start start 2024-03-16T09:09:24.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f39155d0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3910102300 0x7f39101984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:24.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f39155d0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3910103500 0x7f3910198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:24.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f39155d0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3910103ea0 0x7f391019fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:24.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f39155d0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f391010e080 con 0x7f3910103ea0 2024-03-16T09:09:24.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f39155d0700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f391010df00 con 0x7f3910102300 2024-03-16T09:09:24.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f39155d0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f391010e200 con 0x7f3910103500 2024-03-16T09:09:24.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f390effd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3910102300 0x7f39101984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:24.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f390effd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3910102300 0x7f39101984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:48020/0 (socket says 172.21.15.159:48020) 2024-03-16T09:09:24.128 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f390effd700 1 -- 172.21.15.159:0/2409148883 learned_addr learned my addr 172.21.15.159:0/2409148883 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-03-16T09:09:24.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f390e7fc700 1 --2- 172.21.15.159:0/2409148883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3910103500 0x7f3910198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:24.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f390f7fe700 1 --2- 172.21.15.159:0/2409148883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3910103ea0 0x7f391019fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:24.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f390effd700 1 -- 172.21.15.159:0/2409148883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3910103500 msgr2=0x7f3910198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:24.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f390effd700 1 --2- 172.21.15.159:0/2409148883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3910103500 0x7f3910198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:24.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f390effd700 1 -- 172.21.15.159:0/2409148883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3910103ea0 msgr2=0x7f391019fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:24.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f390effd700 1 --2- 172.21.15.159:0/2409148883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3910103ea0 0x7f391019fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:24.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f390effd700 1 -- 172.21.15.159:0/2409148883 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39101a0120 con 0x7f3910102300 2024-03-16T09:09:24.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f390f7fe700 1 --2- 172.21.15.159:0/2409148883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3910103ea0 0x7f391019fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:09:24.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f390effd700 1 --2- 172.21.15.159:0/2409148883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3910102300 0x7f39101984c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f390000c970 tx=0x7f390000cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:24.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f38f7fff700 1 -- 172.21.15.159:0/2409148883 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900004220 con 0x7f3910102300 2024-03-16T09:09:24.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f39155d0700 1 -- 172.21.15.159:0/2409148883 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3910074560 con 0x7f3910102300 2024-03-16T09:09:24.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f38f7fff700 1 -- 172.21.15.159:0/2409148883 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3900007da0 con 0x7f3910102300 2024-03-16T09:09:24.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.124+0000 7f38f7fff700 1 -- 172.21.15.159:0/2409148883 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390000f450 con 0x7f3910102300 2024-03-16T09:09:24.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.125+0000 7f39155d0700 1 -- 172.21.15.159:0/2409148883 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3910074af0 con 0x7f3910102300 2024-03-16T09:09:24.129 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.126+0000 7f39155d0700 1 -- 172.21.15.159:0/2409148883 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3910068f90 con 0x7f3910102300 2024-03-16T09:09:24.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.127+0000 7f38f7fff700 1 -- 172.21.15.159:0/2409148883 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f39000078a0 con 0x7f3910102300 2024-03-16T09:09:24.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.127+0000 7f38f7fff700 1 --2- 172.21.15.159:0/2409148883 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f38f80a2890 0x7f38f80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:24.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.127+0000 7f38f7fff700 1 -- 172.21.15.159:0/2409148883 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f39000cd420 con 0x7f3910102300 2024-03-16T09:09:24.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.127+0000 7f390e7fc700 1 --2- 172.21.15.159:0/2409148883 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f38f80a2890 0x7f38f80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:24.130 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.128+0000 7f390e7fc700 1 --2- 172.21.15.159:0/2409148883 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f38f80a2890 0x7f38f80a4d50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f390400ac50 tx=0x7f390400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:24.134 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.132+0000 7f38f7fff700 1 -- 172.21.15.159:0/2409148883 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3900014030 con 0x7f3910102300 2024-03-16T09:09:24.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:23 smithi194 ceph-mon[36729]: Detected new or changed devices on smithi112 2024-03-16T09:09:24.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:23 smithi194 ceph-mon[36729]: Adjusting osd_memory_target on smithi112 to 8544M 2024-03-16T09:09:24.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:23 smithi194 ceph-mon[36729]: osdmap e26: 4 total, 4 up, 4 in 2024-03-16T09:09:24.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:23 smithi194 ceph-mon[36729]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:24.343 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:24.340+0000 7f39155d0700 1 -- 172.21.15.159:0/2409148883 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3910075fa0 con 0x7f38f80a2890 2024-03-16T09:09:24.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:24 smithi159 ceph-mon[36739]: osdmap e27: 4 total, 4 up, 4 in 2024-03-16T09:09:24.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:09:24.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:09:24.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:24 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:25.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:24 smithi112 ceph-mon[32307]: osdmap e27: 4 total, 4 up, 4 in 2024-03-16T09:09:25.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:24 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:09:25.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:24 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:09:25.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:24 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:25.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:24 smithi194 ceph-mon[36729]: osdmap e27: 4 total, 4 up, 4 in 2024-03-16T09:09:25.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:24 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:09:25.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:24 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:09:25.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:24 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:26.087 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:25 smithi159 ceph-mon[36739]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:26.087 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:25 smithi159 ceph-mon[36739]: osdmap e28: 4 total, 4 up, 4 in 2024-03-16T09:09:26.087 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:25 smithi159 ceph-mon[36739]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:26.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:25 smithi112 ceph-mon[32307]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:26.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:25 smithi112 ceph-mon[32307]: osdmap e28: 4 total, 4 up, 4 in 2024-03-16T09:09:26.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:25 smithi112 ceph-mon[32307]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:26.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:25 smithi194 ceph-mon[36729]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:26.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:25 smithi194 ceph-mon[36729]: osdmap e28: 4 total, 4 up, 4 in 2024-03-16T09:09:26.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:25 smithi194 ceph-mon[36729]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:27.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:26 smithi112 ceph-mon[32307]: osdmap e29: 4 total, 4 up, 4 in 2024-03-16T09:09:27.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:26 smithi194 ceph-mon[36729]: osdmap e29: 4 total, 4 up, 4 in 2024-03-16T09:09:27.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:26 smithi159 ceph-mon[36739]: osdmap e29: 4 total, 4 up, 4 in 2024-03-16T09:09:28.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:27 smithi112 ceph-mon[32307]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:28.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:27 smithi112 ceph-mon[32307]: from='client.? 172.21.15.159:0/2985734933' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e27e743-08f5-4722-a903-1adac60ada69"}]: dispatch 2024-03-16T09:09:28.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:27 smithi112 ceph-mon[32307]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e27e743-08f5-4722-a903-1adac60ada69"}]: dispatch 2024-03-16T09:09:28.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:27 smithi112 ceph-mon[32307]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e27e743-08f5-4722-a903-1adac60ada69"}]': finished 2024-03-16T09:09:28.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:27 smithi112 ceph-mon[32307]: osdmap e30: 5 total, 4 up, 5 in 2024-03-16T09:09:28.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:27 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:28.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:27 smithi112 ceph-mon[32307]: from='client.? 172.21.15.159:0/3387439795' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:09:28.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:27 smithi194 ceph-mon[36729]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:28.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:27 smithi194 ceph-mon[36729]: from='client.? 172.21.15.159:0/2985734933' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e27e743-08f5-4722-a903-1adac60ada69"}]: dispatch 2024-03-16T09:09:28.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:27 smithi194 ceph-mon[36729]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e27e743-08f5-4722-a903-1adac60ada69"}]: dispatch 2024-03-16T09:09:28.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:27 smithi194 ceph-mon[36729]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e27e743-08f5-4722-a903-1adac60ada69"}]': finished 2024-03-16T09:09:28.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:27 smithi194 ceph-mon[36729]: osdmap e30: 5 total, 4 up, 5 in 2024-03-16T09:09:28.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:27 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:28.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:27 smithi194 ceph-mon[36729]: from='client.? 172.21.15.159:0/3387439795' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:09:28.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:27 smithi159 ceph-mon[36739]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:28.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:27 smithi159 ceph-mon[36739]: from='client.? 172.21.15.159:0/2985734933' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e27e743-08f5-4722-a903-1adac60ada69"}]: dispatch 2024-03-16T09:09:28.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:27 smithi159 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e27e743-08f5-4722-a903-1adac60ada69"}]: dispatch 2024-03-16T09:09:28.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:27 smithi159 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e27e743-08f5-4722-a903-1adac60ada69"}]': finished 2024-03-16T09:09:28.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:27 smithi159 ceph-mon[36739]: osdmap e30: 5 total, 4 up, 5 in 2024-03-16T09:09:28.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:27 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:28.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:27 smithi159 ceph-mon[36739]: from='client.? 172.21.15.159:0/3387439795' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:09:30.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:29 smithi194 ceph-mon[36729]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-16T09:09:30.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:29 smithi159 ceph-mon[36739]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-16T09:09:30.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:29 smithi112 ceph-mon[32307]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-03-16T09:09:32.139 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:31 smithi159 ceph-mon[36739]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-16T09:09:32.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:31 smithi194 ceph-mon[36729]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-16T09:09:32.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:31 smithi112 ceph-mon[32307]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-03-16T09:09:33.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:33 smithi159 ceph-mon[36739]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-16T09:09:34.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:33 smithi194 ceph-mon[36729]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-16T09:09:34.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:33 smithi112 ceph-mon[32307]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-16T09:09:35.143 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:34 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T09:09:35.143 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:34 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:35.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:34 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T09:09:35.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:34 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:35.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:34 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-16T09:09:35.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:34 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:36.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:35 smithi194 ceph-mon[36729]: Deploying daemon osd.4 on smithi159 2024-03-16T09:09:36.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:35 smithi194 ceph-mon[36729]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-16T09:09:36.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:35 smithi159 ceph-mon[36739]: Deploying daemon osd.4 on smithi159 2024-03-16T09:09:36.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:35 smithi159 ceph-mon[36739]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-16T09:09:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:35 smithi112 ceph-mon[32307]: Deploying daemon osd.4 on smithi159 2024-03-16T09:09:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:35 smithi112 ceph-mon[32307]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-16T09:09:38.095 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:37 smithi159 ceph-mon[36739]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T09:09:38.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:37 smithi194 ceph-mon[36729]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T09:09:38.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:37 smithi112 ceph-mon[32307]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T09:09:39.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:39 smithi159 ceph-mon[36739]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-03-16T09:09:40.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:39 smithi194 ceph-mon[36729]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-03-16T09:09:40.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:39 smithi112 ceph-mon[32307]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-03-16T09:09:41.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:40 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:41.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:40 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:41.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:40 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:41.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:40 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:41.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:40 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:41.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:40 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:41.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:40 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:41.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:40 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:41.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:40 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:42.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:41 smithi159 ceph-mon[36739]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:42.341 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:41 smithi112 ceph-mon[32307]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:42.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:41 smithi194 ceph-mon[36729]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:43.909 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:43.906+0000 7f38f7fff700 1 -- 172.21.15.159:0/2409148883 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3910075fa0 con 0x7f38f80a2890 2024-03-16T09:09:43.909 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 4 on host 'smithi159' 2024-03-16T09:09:43.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:43.908+0000 7f39155d0700 1 -- 172.21.15.159:0/2409148883 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f38f80a2890 msgr2=0x7f38f80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:43.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:43.908+0000 7f39155d0700 1 --2- 172.21.15.159:0/2409148883 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f38f80a2890 0x7f38f80a4d50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f390400ac50 tx=0x7f390400a380 comp rx=0 tx=0).stop 2024-03-16T09:09:43.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:43.908+0000 7f39155d0700 1 -- 172.21.15.159:0/2409148883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3910102300 msgr2=0x7f39101984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:43.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:43.908+0000 7f39155d0700 1 --2- 172.21.15.159:0/2409148883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3910102300 0x7f39101984c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f390000c970 tx=0x7f390000cc80 comp rx=0 tx=0).stop 2024-03-16T09:09:43.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:43.909+0000 7f39155d0700 1 -- 172.21.15.159:0/2409148883 shutdown_connections 2024-03-16T09:09:43.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:43.909+0000 7f39155d0700 1 --2- 172.21.15.159:0/2409148883 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f38f80a2890 0x7f38f80a4d50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:43.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:43.909+0000 7f39155d0700 1 --2- 172.21.15.159:0/2409148883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3910102300 0x7f39101984c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:43.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:43.909+0000 7f39155d0700 1 --2- 172.21.15.159:0/2409148883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3910103ea0 0x7f391019fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:43.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:43.909+0000 7f39155d0700 1 --2- 172.21.15.159:0/2409148883 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3910103500 0x7f3910198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:43.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:43.909+0000 7f39155d0700 1 -- 172.21.15.159:0/2409148883 >> 172.21.15.159:0/2409148883 conn(0x7f39100fd8e0 msgr2=0x7f3910106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:43.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:43.909+0000 7f39155d0700 1 -- 172.21.15.159:0/2409148883 shutdown_connections 2024-03-16T09:09:43.911 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:09:43.909+0000 7f39155d0700 1 -- 172.21.15.159:0/2409148883 wait complete. 2024-03-16T09:09:44.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:43 smithi159 ceph-mon[36739]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:44.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:43 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:44.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:43 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:44.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:43 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:44.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:43 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:09:44.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:43 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:44.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:43 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:44.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:43 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:44.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:43 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:44.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:43 smithi159 ceph-mon[36739]: from='osd.4 [v2:172.21.15.159:6808/4036563014,v1:172.21.15.159:6809/4036563014]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T09:09:44.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:43 smithi159 ceph-mon[36739]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T09:09:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:43 smithi112 ceph-mon[32307]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:09:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:44.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:44.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:44.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:44.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:43 smithi112 ceph-mon[32307]: from='osd.4 [v2:172.21.15.159:6808/4036563014,v1:172.21.15.159:6809/4036563014]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T09:09:44.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:43 smithi112 ceph-mon[32307]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T09:09:44.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:43 smithi194 ceph-mon[36729]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:44.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:43 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:44.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:43 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:44.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:43 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:44.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:43 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:09:44.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:43 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:44.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:43 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:44.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:43 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:44.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:43 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:09:44.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:43 smithi194 ceph-mon[36729]: from='osd.4 [v2:172.21.15.159:6808/4036563014,v1:172.21.15.159:6809/4036563014]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T09:09:44.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:43 smithi194 ceph-mon[36729]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-16T09:09:44.594 DEBUG:teuthology.orchestra.run.smithi159:osd.4> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.4.service 2024-03-16T09:09:44.596 INFO:tasks.cephadm:Deploying osd.5 on smithi194 with /dev/vg_nvme/lv_3... 2024-03-16T09:09:44.596 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-16T09:09:44.843 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:09:46.130 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:45 smithi159 ceph-mon[36739]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:46.130 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:45 smithi159 ceph-mon[36739]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-16T09:09:46.130 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:45 smithi159 ceph-mon[36739]: from='osd.4 [v2:172.21.15.159:6808/4036563014,v1:172.21.15.159:6809/4036563014]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:09:46.130 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:45 smithi159 ceph-mon[36739]: osdmap e31: 5 total, 4 up, 5 in 2024-03-16T09:09:46.130 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:45 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:46.130 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:45 smithi159 ceph-mon[36739]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:09:46.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:45 smithi194 ceph-mon[36729]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:46.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:45 smithi194 ceph-mon[36729]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-16T09:09:46.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:45 smithi194 ceph-mon[36729]: from='osd.4 [v2:172.21.15.159:6808/4036563014,v1:172.21.15.159:6809/4036563014]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:09:46.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:45 smithi194 ceph-mon[36729]: osdmap e31: 5 total, 4 up, 5 in 2024-03-16T09:09:46.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:45 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:46.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:45 smithi194 ceph-mon[36729]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:09:46.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:45 smithi112 ceph-mon[32307]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:46.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:45 smithi112 ceph-mon[32307]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-16T09:09:46.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:45 smithi112 ceph-mon[32307]: from='osd.4 [v2:172.21.15.159:6808/4036563014,v1:172.21.15.159:6809/4036563014]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:09:46.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:45 smithi112 ceph-mon[32307]: osdmap e31: 5 total, 4 up, 5 in 2024-03-16T09:09:46.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:45 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:46.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:45 smithi112 ceph-mon[32307]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:09:46.487 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:09:46 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4[48007]: 2024-03-16T09:09:46.331+0000 7fb2386f0700 -1 osd.4 0 waiting for initial osdmap 2024-03-16T09:09:46.487 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:09:46 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4[48007]: 2024-03-16T09:09:46.339+0000 7fb2304db700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T09:09:46.780 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-16T09:09:46.780 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-16T09:09:46.780 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2024-03-16T09:09:46.780 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2024-03-16T09:09:46.780 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113744 s, 922 MB/s 2024-03-16T09:09:46.780 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2024-03-16T09:09:47.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:46 smithi194 ceph-mon[36729]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-16T09:09:47.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:46 smithi194 ceph-mon[36729]: osdmap e32: 5 total, 4 up, 5 in 2024-03-16T09:09:47.090 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:46 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:47.091 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:46 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:47.091 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:46 smithi194 ceph-mon[36729]: from='osd.4 ' entity='osd.4' 2024-03-16T09:09:47.091 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:46 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:47.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:46 smithi159 ceph-mon[36739]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-16T09:09:47.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:46 smithi159 ceph-mon[36739]: osdmap e32: 5 total, 4 up, 5 in 2024-03-16T09:09:47.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:46 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:47.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:46 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:47.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:46 smithi159 ceph-mon[36739]: from='osd.4 ' entity='osd.4' 2024-03-16T09:09:47.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:46 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:47.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:46 smithi112 ceph-mon[32307]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-16T09:09:47.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:46 smithi112 ceph-mon[32307]: osdmap e32: 5 total, 4 up, 5 in 2024-03-16T09:09:47.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:46 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:47.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:46 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:47.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:46 smithi112 ceph-mon[32307]: from='osd.4 ' entity='osd.4' 2024-03-16T09:09:47.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:46 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:47.410 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi194:vg_nvme/lv_3 2024-03-16T09:09:47.656 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:09:47.893 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:47 smithi159 ceph-mon[36739]: purged_snaps scrub starts 2024-03-16T09:09:47.893 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:47 smithi159 ceph-mon[36739]: purged_snaps scrub ok 2024-03-16T09:09:47.893 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:47 smithi159 ceph-mon[36739]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:47.893 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:47 smithi159 ceph-mon[36739]: osd.4 [v2:172.21.15.159:6808/4036563014,v1:172.21.15.159:6809/4036563014] boot 2024-03-16T09:09:47.893 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:47 smithi159 ceph-mon[36739]: osdmap e33: 5 total, 5 up, 5 in 2024-03-16T09:09:47.894 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:47 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:47.894 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:47 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:47.894 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:47 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:47.894 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:47 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:09:47.894 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:47 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:09:47.894 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:47 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:47.894 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:47 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:47.894 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:47 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:09:47.894 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:47 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:47.995 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:47 smithi194 ceph-mon[36729]: purged_snaps scrub starts 2024-03-16T09:09:47.995 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:47 smithi194 ceph-mon[36729]: purged_snaps scrub ok 2024-03-16T09:09:47.995 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:47 smithi194 ceph-mon[36729]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:47.995 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:47 smithi194 ceph-mon[36729]: osd.4 [v2:172.21.15.159:6808/4036563014,v1:172.21.15.159:6809/4036563014] boot 2024-03-16T09:09:47.995 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:47 smithi194 ceph-mon[36729]: osdmap e33: 5 total, 5 up, 5 in 2024-03-16T09:09:47.995 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:47 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:47.995 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:47 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:47.995 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:47 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:47.995 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:47 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:09:47.995 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:47 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:09:47.995 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:47 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:47.995 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:47 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:47.996 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:47 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:09:47.996 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:47 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:47 smithi112 ceph-mon[32307]: purged_snaps scrub starts 2024-03-16T09:09:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:47 smithi112 ceph-mon[32307]: purged_snaps scrub ok 2024-03-16T09:09:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:47 smithi112 ceph-mon[32307]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-16T09:09:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:47 smithi112 ceph-mon[32307]: osd.4 [v2:172.21.15.159:6808/4036563014,v1:172.21.15.159:6809/4036563014] boot 2024-03-16T09:09:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:47 smithi112 ceph-mon[32307]: osdmap e33: 5 total, 5 up, 5 in 2024-03-16T09:09:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-16T09:09:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:09:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:09:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:09:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:09:49.106 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:48 smithi194 ceph-mon[36729]: Detected new or changed devices on smithi159 2024-03-16T09:09:49.107 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:48 smithi194 ceph-mon[36729]: Adjusting osd_memory_target on smithi159 to 8540M 2024-03-16T09:09:49.107 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:48 smithi194 ceph-mon[36729]: osdmap e34: 5 total, 5 up, 5 in 2024-03-16T09:09:49.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:48 smithi159 ceph-mon[36739]: Detected new or changed devices on smithi159 2024-03-16T09:09:49.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:48 smithi159 ceph-mon[36739]: Adjusting osd_memory_target on smithi159 to 8540M 2024-03-16T09:09:49.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:48 smithi159 ceph-mon[36739]: osdmap e34: 5 total, 5 up, 5 in 2024-03-16T09:09:49.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.328+0000 7fb9040e4700 1 -- 172.21.15.194:0/3570860903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8fc101200 msgr2=0x7fb8fc101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:49.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.328+0000 7fb9040e4700 1 --2- 172.21.15.194:0/3570860903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8fc101200 0x7fb8fc101620 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec009990 tx=0x7fb8ec009ca0 comp rx=0 tx=0).stop 2024-03-16T09:09:49.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.328+0000 7fb9040e4700 1 -- 172.21.15.194:0/3570860903 shutdown_connections 2024-03-16T09:09:49.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.328+0000 7fb9040e4700 1 --2- 172.21.15.194:0/3570860903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8fc102da0 0x7fb8fc109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:49.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.328+0000 7fb9040e4700 1 --2- 172.21.15.194:0/3570860903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8fc101200 0x7fb8fc101620 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:49.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.328+0000 7fb9040e4700 1 --2- 172.21.15.194:0/3570860903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8fc102400 0x7fb8fc102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:49.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.328+0000 7fb9040e4700 1 -- 172.21.15.194:0/3570860903 >> 172.21.15.194:0/3570860903 conn(0x7fb8fc0fc7c0 msgr2=0x7fb8fc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:09:49.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.328+0000 7fb9040e4700 1 -- 172.21.15.194:0/3570860903 shutdown_connections 2024-03-16T09:09:49.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.328+0000 7fb9040e4700 1 -- 172.21.15.194:0/3570860903 wait complete. 2024-03-16T09:09:49.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.329+0000 7fb9040e4700 1 Processor -- start 2024-03-16T09:09:49.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.329+0000 7fb9040e4700 1 -- start start 2024-03-16T09:09:49.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.329+0000 7fb9040e4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8fc101200 0x7fb8fc105190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:49.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.329+0000 7fb9040e4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8fc102400 0x7fb8fc1037e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:49.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.329+0000 7fb9040e4700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8fc102da0 0x7fb8fc103d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:49.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.329+0000 7fb9040e4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc10cf40 con 0x7fb8fc101200 2024-03-16T09:09:49.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.329+0000 7fb9040e4700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc10cdc0 con 0x7fb8fc102da0 2024-03-16T09:09:49.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.329+0000 7fb9040e4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc10d0c0 con 0x7fb8fc102400 2024-03-16T09:09:49.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.329+0000 7fb90167f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8fc102400 0x7fb8fc1037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:49.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.329+0000 7fb90167f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8fc102400 0x7fb8fc1037e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:37302/0 (socket says 172.21.15.194:37302) 2024-03-16T09:09:49.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.329+0000 7fb90167f700 1 -- 172.21.15.194:0/4028963342 learned_addr learned my addr 172.21.15.194:0/4028963342 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T09:09:49.334 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.329+0000 7fb901e80700 1 --2- 172.21.15.194:0/4028963342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8fc101200 0x7fb8fc105190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:49.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.330+0000 7fb902681700 1 --2- 172.21.15.194:0/4028963342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8fc102da0 0x7fb8fc103d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:49.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.330+0000 7fb90167f700 1 -- 172.21.15.194:0/4028963342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8fc102da0 msgr2=0x7fb8fc103d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:49.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.330+0000 7fb90167f700 1 --2- 172.21.15.194:0/4028963342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8fc102da0 0x7fb8fc103d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:49.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.330+0000 7fb90167f700 1 -- 172.21.15.194:0/4028963342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8fc101200 msgr2=0x7fb8fc105190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:09:49.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.330+0000 7fb90167f700 1 --2- 172.21.15.194:0/4028963342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8fc101200 0x7fb8fc105190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:09:49.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.330+0000 7fb90167f700 1 -- 172.21.15.194:0/4028963342 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8fc1045d0 con 0x7fb8fc102400 2024-03-16T09:09:49.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.330+0000 7fb901e80700 1 --2- 172.21.15.194:0/4028963342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8fc101200 0x7fb8fc105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:09:49.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.330+0000 7fb90167f700 1 --2- 172.21.15.194:0/4028963342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8fc102400 0x7fb8fc1037e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800c970 tx=0x7fb8f800cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:49.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.330+0000 7fb8f2ffd700 1 -- 172.21.15.194:0/4028963342 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f80078d0 con 0x7fb8fc102400 2024-03-16T09:09:49.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.330+0000 7fb8f2ffd700 1 -- 172.21.15.194:0/4028963342 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb8f8018480 con 0x7fb8fc102400 2024-03-16T09:09:49.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.330+0000 7fb9040e4700 1 -- 172.21.15.194:0/4028963342 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8fc1a27b0 con 0x7fb8fc102400 2024-03-16T09:09:49.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.330+0000 7fb8f2ffd700 1 -- 172.21.15.194:0/4028963342 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f800f6b0 con 0x7fb8fc102400 2024-03-16T09:09:49.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.330+0000 7fb9040e4700 1 -- 172.21.15.194:0/4028963342 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8fc1a2cc0 con 0x7fb8fc102400 2024-03-16T09:09:49.335 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.331+0000 7fb9040e4700 1 -- 172.21.15.194:0/4028963342 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8fc068f90 con 0x7fb8fc102400 2024-03-16T09:09:49.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.332+0000 7fb8f2ffd700 1 -- 172.21.15.194:0/4028963342 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fb8f8018630 con 0x7fb8fc102400 2024-03-16T09:09:49.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.333+0000 7fb8f2ffd700 1 --2- 172.21.15.194:0/4028963342 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb8e80a2910 0x7fb8e80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:09:49.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.333+0000 7fb8f2ffd700 1 -- 172.21.15.194:0/4028963342 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fb8f80cdd10 con 0x7fb8fc102400 2024-03-16T09:09:49.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.333+0000 7fb901e80700 1 --2- 172.21.15.194:0/4028963342 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb8e80a2910 0x7fb8e80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:09:49.336 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.333+0000 7fb901e80700 1 --2- 172.21.15.194:0/4028963342 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb8e80a2910 0x7fb8e80a4dd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0039a0 tx=0x7fb8ec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:09:49.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.337+0000 7fb8f2ffd700 1 -- 172.21.15.194:0/4028963342 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb8f809bf40 con 0x7fb8fc102400 2024-03-16T09:09:49.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:48 smithi112 ceph-mon[32307]: Detected new or changed devices on smithi159 2024-03-16T09:09:49.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:48 smithi112 ceph-mon[32307]: Adjusting osd_memory_target on smithi159 to 8540M 2024-03-16T09:09:49.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:48 smithi112 ceph-mon[32307]: osdmap e34: 5 total, 5 up, 5 in 2024-03-16T09:09:49.541 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:09:49.539+0000 7fb9040e4700 1 -- 172.21.15.194:0/4028963342 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb8fc1a2f80 con 0x7fb8e80a2910 2024-03-16T09:09:50.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:49 smithi194 ceph-mon[36729]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:09:50.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:49 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:09:50.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:49 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:09:50.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:49 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:50.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:49 smithi159 ceph-mon[36739]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:09:50.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:49 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:09:50.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:49 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:09:50.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:49 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:50.243 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:49 smithi112 ceph-mon[32307]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:09:50.243 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:49 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:09:50.243 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:49 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:09:50.243 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:49 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:09:51.126 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:50 smithi194 ceph-mon[36729]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:51.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:50 smithi159 ceph-mon[36739]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:51.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:50 smithi112 ceph-mon[32307]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:09:52.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:51 smithi159 ceph-mon[36739]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:09:52.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:51 smithi112 ceph-mon[32307]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:09:52.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:51 smithi194 ceph-mon[36729]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:09:53.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:52 smithi159 ceph-mon[36739]: from='client.? 172.21.15.194:0/3119570171' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7440b71-313a-42ff-9d9b-4badc07608cd"}]: dispatch 2024-03-16T09:09:53.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:52 smithi159 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7440b71-313a-42ff-9d9b-4badc07608cd"}]: dispatch 2024-03-16T09:09:53.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:52 smithi159 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7440b71-313a-42ff-9d9b-4badc07608cd"}]': finished 2024-03-16T09:09:53.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:52 smithi159 ceph-mon[36739]: osdmap e35: 6 total, 5 up, 6 in 2024-03-16T09:09:53.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:52 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:09:53.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:52 smithi112 ceph-mon[32307]: from='client.? 172.21.15.194:0/3119570171' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7440b71-313a-42ff-9d9b-4badc07608cd"}]: dispatch 2024-03-16T09:09:53.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:52 smithi112 ceph-mon[32307]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7440b71-313a-42ff-9d9b-4badc07608cd"}]: dispatch 2024-03-16T09:09:53.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:52 smithi112 ceph-mon[32307]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7440b71-313a-42ff-9d9b-4badc07608cd"}]': finished 2024-03-16T09:09:53.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:52 smithi112 ceph-mon[32307]: osdmap e35: 6 total, 5 up, 6 in 2024-03-16T09:09:53.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:52 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:09:53.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:52 smithi194 ceph-mon[36729]: from='client.? 172.21.15.194:0/3119570171' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7440b71-313a-42ff-9d9b-4badc07608cd"}]: dispatch 2024-03-16T09:09:53.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:52 smithi194 ceph-mon[36729]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7440b71-313a-42ff-9d9b-4badc07608cd"}]: dispatch 2024-03-16T09:09:53.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:52 smithi194 ceph-mon[36729]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7440b71-313a-42ff-9d9b-4badc07608cd"}]': finished 2024-03-16T09:09:53.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:52 smithi194 ceph-mon[36729]: osdmap e35: 6 total, 5 up, 6 in 2024-03-16T09:09:53.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:52 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:09:54.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:53 smithi159 ceph-mon[36739]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:09:54.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:53 smithi159 ceph-mon[36739]: from='client.? 172.21.15.194:0/1071914033' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:09:54.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:53 smithi112 ceph-mon[32307]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:09:54.424 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:53 smithi112 ceph-mon[32307]: from='client.? 172.21.15.194:0/1071914033' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:09:54.424 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:53 smithi194 ceph-mon[36729]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:09:54.424 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:53 smithi194 ceph-mon[36729]: from='client.? 172.21.15.194:0/1071914033' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:09:56.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:55 smithi159 ceph-mon[36739]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:09:56.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:55 smithi194 ceph-mon[36729]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:09:56.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:55 smithi112 ceph-mon[32307]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:09:58.171 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:57 smithi194 ceph-mon[36729]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:09:58.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:57 smithi159 ceph-mon[36739]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:09:58.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:57 smithi112 ceph-mon[32307]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:00.205 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:59 smithi194 ceph-mon[36729]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:00.205 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:59 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T09:10:00.205 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:09:59 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:00.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:59 smithi159 ceph-mon[36739]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:00.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:59 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T09:10:00.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:09:59 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:59 smithi112 ceph-mon[32307]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-16T09:10:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:09:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:01.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:00 smithi194 ceph-mon[36729]: Deploying daemon osd.5 on smithi194 2024-03-16T09:10:01.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:00 smithi194 ceph-mon[36729]: overall HEALTH_OK 2024-03-16T09:10:01.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:00 smithi159 ceph-mon[36739]: Deploying daemon osd.5 on smithi194 2024-03-16T09:10:01.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:00 smithi159 ceph-mon[36739]: overall HEALTH_OK 2024-03-16T09:10:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:00 smithi112 ceph-mon[32307]: Deploying daemon osd.5 on smithi194 2024-03-16T09:10:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:00 smithi112 ceph-mon[32307]: overall HEALTH_OK 2024-03-16T09:10:02.208 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:01 smithi194 ceph-mon[36729]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:02.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:01 smithi159 ceph-mon[36739]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:02.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:01 smithi112 ceph-mon[32307]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:04.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:03 smithi194 ceph-mon[36729]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:04.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:03 smithi159 ceph-mon[36739]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:04.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:03 smithi112 ceph-mon[32307]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:06.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:05 smithi194 ceph-mon[36729]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:06.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:06.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:06.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:06.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:05 smithi159 ceph-mon[36739]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:06.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:06.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:06.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:06.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:05 smithi112 ceph-mon[32307]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:06.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:06.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:06.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:08.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:07 smithi194 ceph-mon[36729]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:08.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:07 smithi159 ceph-mon[36739]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:08.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:07 smithi112 ceph-mon[32307]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:08.802 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:10:08.800+0000 7fb8f2ffd700 1 -- 172.21.15.194:0/4028963342 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb8fc1a2f80 con 0x7fb8e80a2910 2024-03-16T09:10:08.802 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 5 on host 'smithi194' 2024-03-16T09:10:08.805 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:10:08.802+0000 7fb9040e4700 1 -- 172.21.15.194:0/4028963342 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb8e80a2910 msgr2=0x7fb8e80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:08.805 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:10:08.802+0000 7fb9040e4700 1 --2- 172.21.15.194:0/4028963342 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb8e80a2910 0x7fb8e80a4dd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0039a0 tx=0x7fb8ec005fd0 comp rx=0 tx=0).stop 2024-03-16T09:10:08.805 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:10:08.802+0000 7fb9040e4700 1 -- 172.21.15.194:0/4028963342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8fc102400 msgr2=0x7fb8fc1037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:08.805 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:10:08.802+0000 7fb9040e4700 1 --2- 172.21.15.194:0/4028963342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8fc102400 0x7fb8fc1037e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800c970 tx=0x7fb8f800cc80 comp rx=0 tx=0).stop 2024-03-16T09:10:08.805 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:10:08.802+0000 7fb9040e4700 1 -- 172.21.15.194:0/4028963342 shutdown_connections 2024-03-16T09:10:08.805 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:10:08.802+0000 7fb9040e4700 1 --2- 172.21.15.194:0/4028963342 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb8e80a2910 0x7fb8e80a4dd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:08.805 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:10:08.802+0000 7fb9040e4700 1 --2- 172.21.15.194:0/4028963342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8fc102da0 0x7fb8fc103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:08.805 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:10:08.802+0000 7fb9040e4700 1 --2- 172.21.15.194:0/4028963342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8fc101200 0x7fb8fc105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:08.805 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:10:08.802+0000 7fb9040e4700 1 --2- 172.21.15.194:0/4028963342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8fc102400 0x7fb8fc1037e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:08.805 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:10:08.802+0000 7fb9040e4700 1 -- 172.21.15.194:0/4028963342 >> 172.21.15.194:0/4028963342 conn(0x7fb8fc0fc7c0 msgr2=0x7fb8fc0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:08.806 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:10:08.802+0000 7fb9040e4700 1 -- 172.21.15.194:0/4028963342 shutdown_connections 2024-03-16T09:10:08.806 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:10:08.802+0000 7fb9040e4700 1 -- 172.21.15.194:0/4028963342 wait complete. 2024-03-16T09:10:09.009 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:09.009 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:09.009 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:09.009 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:10:09.009 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:09.009 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:09.009 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:09.009 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:09.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:09.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:09.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:09.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:10:09.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:09.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:09.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:09.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:09.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:09.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:09.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:09.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:10:09.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:09.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:09.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:09.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:09.449 DEBUG:teuthology.orchestra.run.smithi194:osd.5> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.5.service 2024-03-16T09:10:09.452 INFO:tasks.cephadm:Deploying osd.6 on smithi112 with /dev/vg_nvme/lv_2... 2024-03-16T09:10:09.452 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-16T09:10:09.701 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:10:10.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:09 smithi112 ceph-mon[32307]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:10.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:09 smithi112 ceph-mon[32307]: from='osd.5 [v2:172.21.15.194:6808/1459524918,v1:172.21.15.194:6809/1459524918]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T09:10:10.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:09 smithi112 ceph-mon[32307]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T09:10:10.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:09 smithi194 ceph-mon[36729]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:10.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:09 smithi194 ceph-mon[36729]: from='osd.5 [v2:172.21.15.194:6808/1459524918,v1:172.21.15.194:6809/1459524918]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T09:10:10.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:09 smithi194 ceph-mon[36729]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T09:10:10.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:09 smithi159 ceph-mon[36739]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:10.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:09 smithi159 ceph-mon[36739]: from='osd.5 [v2:172.21.15.194:6808/1459524918,v1:172.21.15.194:6809/1459524918]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T09:10:10.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:09 smithi159 ceph-mon[36739]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-16T09:10:11.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:10 smithi159 ceph-mon[36739]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-16T09:10:11.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:10 smithi159 ceph-mon[36739]: from='osd.5 [v2:172.21.15.194:6808/1459524918,v1:172.21.15.194:6809/1459524918]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:10:11.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:10 smithi159 ceph-mon[36739]: osdmap e36: 6 total, 5 up, 6 in 2024-03-16T09:10:11.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:10 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:10:11.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:10 smithi159 ceph-mon[36739]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:10:11.304 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:10 smithi112 ceph-mon[32307]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-16T09:10:11.304 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:10 smithi112 ceph-mon[32307]: from='osd.5 [v2:172.21.15.194:6808/1459524918,v1:172.21.15.194:6809/1459524918]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:10:11.304 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:10 smithi112 ceph-mon[32307]: osdmap e36: 6 total, 5 up, 6 in 2024-03-16T09:10:11.304 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:10 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:10:11.304 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:10 smithi112 ceph-mon[32307]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:10:11.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:10 smithi194 ceph-mon[36729]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-16T09:10:11.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:10 smithi194 ceph-mon[36729]: from='osd.5 [v2:172.21.15.194:6808/1459524918,v1:172.21.15.194:6809/1459524918]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:10:11.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:10 smithi194 ceph-mon[36729]: osdmap e36: 6 total, 5 up, 6 in 2024-03-16T09:10:11.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:10 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:10:11.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:10 smithi194 ceph-mon[36729]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:10:11.403 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:10:10 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:10:10.980+0000 7feed00a2700 -1 osd.5 0 waiting for initial osdmap 2024-03-16T09:10:11.403 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:10:10 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:10:10.988+0000 7feecbe95700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T09:10:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-16T09:10:11.883 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-16T09:10:11.884 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-03-16T09:10:11.884 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-03-16T09:10:11.884 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113971 s, 920 MB/s 2024-03-16T09:10:11.884 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-03-16T09:10:12.216 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:11 smithi112 ceph-mon[32307]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:12.216 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:11 smithi112 ceph-mon[32307]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T09:10:12.216 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:11 smithi112 ceph-mon[32307]: osdmap e37: 6 total, 5 up, 6 in 2024-03-16T09:10:12.216 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:11 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:10:12.217 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:11 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:10:12.217 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:11 smithi112 ceph-mon[32307]: osd.5 [v2:172.21.15.194:6808/1459524918,v1:172.21.15.194:6809/1459524918] boot 2024-03-16T09:10:12.217 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:11 smithi112 ceph-mon[32307]: osdmap e38: 6 total, 6 up, 6 in 2024-03-16T09:10:12.217 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:11 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:10:12.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:11 smithi159 ceph-mon[36739]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:12.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:11 smithi159 ceph-mon[36739]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T09:10:12.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:11 smithi159 ceph-mon[36739]: osdmap e37: 6 total, 5 up, 6 in 2024-03-16T09:10:12.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:11 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:10:12.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:11 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:10:12.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:11 smithi159 ceph-mon[36739]: osd.5 [v2:172.21.15.194:6808/1459524918,v1:172.21.15.194:6809/1459524918] boot 2024-03-16T09:10:12.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:11 smithi159 ceph-mon[36739]: osdmap e38: 6 total, 6 up, 6 in 2024-03-16T09:10:12.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:11 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:10:12.389 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:11 smithi194 ceph-mon[36729]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-16T09:10:12.389 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:11 smithi194 ceph-mon[36729]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T09:10:12.389 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:11 smithi194 ceph-mon[36729]: osdmap e37: 6 total, 5 up, 6 in 2024-03-16T09:10:12.389 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:11 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:10:12.389 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:11 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:10:12.389 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:11 smithi194 ceph-mon[36729]: osd.5 [v2:172.21.15.194:6808/1459524918,v1:172.21.15.194:6809/1459524918] boot 2024-03-16T09:10:12.389 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:11 smithi194 ceph-mon[36729]: osdmap e38: 6 total, 6 up, 6 in 2024-03-16T09:10:12.389 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:11 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-16T09:10:12.542 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi112:vg_nvme/lv_2 2024-03-16T09:10:12.782 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:10:13.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:12 smithi194 ceph-mon[36729]: purged_snaps scrub starts 2024-03-16T09:10:13.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:12 smithi194 ceph-mon[36729]: purged_snaps scrub ok 2024-03-16T09:10:13.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:12 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:13.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:12 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:13.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:12 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:10:13.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:12 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:10:13.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:12 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:13.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:12 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:13.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:12 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:10:13.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:12 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:13.155 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:12 smithi112 ceph-mon[32307]: purged_snaps scrub starts 2024-03-16T09:10:13.156 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:12 smithi112 ceph-mon[32307]: purged_snaps scrub ok 2024-03-16T09:10:13.156 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:12 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:13.156 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:12 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:13.156 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:12 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:10:13.156 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:12 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:10:13.156 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:12 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:13.157 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:12 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:13.157 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:12 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:10:13.157 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:12 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:13.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:12 smithi159 ceph-mon[36739]: purged_snaps scrub starts 2024-03-16T09:10:13.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:12 smithi159 ceph-mon[36739]: purged_snaps scrub ok 2024-03-16T09:10:13.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:12 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:13.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:12 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:13.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:12 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:10:13.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:12 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:10:13.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:12 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:13.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:12 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:13.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:12 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:10:13.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:12 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:14.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:14 smithi112 ceph-mon[32307]: Detected new or changed devices on smithi194 2024-03-16T09:10:14.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:14 smithi112 ceph-mon[32307]: Adjusting osd_memory_target on smithi194 to 8540M 2024-03-16T09:10:14.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:14 smithi112 ceph-mon[32307]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:14.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:14 smithi112 ceph-mon[32307]: osdmap e39: 6 total, 6 up, 6 in 2024-03-16T09:10:14.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:14 smithi194 ceph-mon[36729]: Detected new or changed devices on smithi194 2024-03-16T09:10:14.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:14 smithi194 ceph-mon[36729]: Adjusting osd_memory_target on smithi194 to 8540M 2024-03-16T09:10:14.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:14 smithi194 ceph-mon[36729]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:14.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:14 smithi194 ceph-mon[36729]: osdmap e39: 6 total, 6 up, 6 in 2024-03-16T09:10:14.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.453+0000 7f5cbc6df700 1 -- 172.21.15.112:0/1820508021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cb4103ea0 msgr2=0x7f5cb410a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:14.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.453+0000 7f5cbc6df700 1 --2- 172.21.15.112:0/1820508021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cb4103ea0 0x7f5cb410a750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00b0d0 tx=0x7f5cac00b3e0 comp rx=0 tx=0).stop 2024-03-16T09:10:14.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.453+0000 7f5cbc6df700 1 -- 172.21.15.112:0/1820508021 shutdown_connections 2024-03-16T09:10:14.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.453+0000 7f5cbc6df700 1 --2- 172.21.15.112:0/1820508021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cb4103ea0 0x7f5cb410a750 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:14.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.453+0000 7f5cbc6df700 1 --2- 172.21.15.112:0/1820508021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cb4102300 0x7f5cb4102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:14.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.453+0000 7f5cbc6df700 1 --2- 172.21.15.112:0/1820508021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cb4103500 0x7f5cb4103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:14.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.453+0000 7f5cbc6df700 1 -- 172.21.15.112:0/1820508021 >> 172.21.15.112:0/1820508021 conn(0x7f5cb40fd8e0 msgr2=0x7f5cb40ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:14.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.453+0000 7f5cbc6df700 1 -- 172.21.15.112:0/1820508021 shutdown_connections 2024-03-16T09:10:14.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.453+0000 7f5cbc6df700 1 -- 172.21.15.112:0/1820508021 wait complete. 2024-03-16T09:10:14.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.453+0000 7f5cbc6df700 1 Processor -- start 2024-03-16T09:10:14.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.453+0000 7f5cbc6df700 1 -- start start 2024-03-16T09:10:14.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.453+0000 7f5cbc6df700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cb4102300 0x7f5cb4076070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:14.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.453+0000 7f5cbc6df700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cb4103500 0x7f5cb40746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:14.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.453+0000 7f5cbc6df700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cb4103ea0 0x7f5cb4074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:14.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.453+0000 7f5cbc6df700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb410e040 con 0x7f5cb4103ea0 2024-03-16T09:10:14.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.453+0000 7f5cbc6df700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb410dec0 con 0x7f5cb4103500 2024-03-16T09:10:14.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.453+0000 7f5cbc6df700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb410e1c0 con 0x7f5cb4102300 2024-03-16T09:10:14.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.454+0000 7f5cbac7c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cb4103ea0 0x7f5cb4074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:14.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.454+0000 7f5cbac7c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cb4103ea0 0x7f5cb4074c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40304/0 (socket says 172.21.15.112:40304) 2024-03-16T09:10:14.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.454+0000 7f5cbac7c700 1 -- 172.21.15.112:0/1072735411 learned_addr learned my addr 172.21.15.112:0/1072735411 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:10:14.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.454+0000 7f5cba47b700 1 --2- 172.21.15.112:0/1072735411 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cb4102300 0x7f5cb4076070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:14.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.454+0000 7f5cb9c7a700 1 --2- 172.21.15.112:0/1072735411 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cb4103500 0x7f5cb40746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:14.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.454+0000 7f5cbac7c700 1 -- 172.21.15.112:0/1072735411 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cb4102300 msgr2=0x7f5cb4076070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:14.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.454+0000 7f5cbac7c700 1 --2- 172.21.15.112:0/1072735411 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cb4102300 0x7f5cb4076070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:14.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.454+0000 7f5cbac7c700 1 -- 172.21.15.112:0/1072735411 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cb4103500 msgr2=0x7f5cb40746c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:14.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.454+0000 7f5cbac7c700 1 --2- 172.21.15.112:0/1072735411 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cb4103500 0x7f5cb40746c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:14.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.454+0000 7f5cbac7c700 1 -- 172.21.15.112:0/1072735411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb40754b0 con 0x7f5cb4103ea0 2024-03-16T09:10:14.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.454+0000 7f5cb9c7a700 1 --2- 172.21.15.112:0/1072735411 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cb4103500 0x7f5cb40746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:10:14.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.454+0000 7f5cbac7c700 1 --2- 172.21.15.112:0/1072735411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cb4103ea0 0x7f5cb4074c00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5cac007dc0 tx=0x7f5cac007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:14.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.454+0000 7f5cab7fe700 1 -- 172.21.15.112:0/1072735411 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac01f020 con 0x7f5cb4103ea0 2024-03-16T09:10:14.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.454+0000 7f5cab7fe700 1 -- 172.21.15.112:0/1072735411 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5cac003e10 con 0x7f5cb4103ea0 2024-03-16T09:10:14.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.454+0000 7f5cbc6df700 1 -- 172.21.15.112:0/1072735411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb41a25d0 con 0x7f5cb4103ea0 2024-03-16T09:10:14.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.454+0000 7f5cab7fe700 1 -- 172.21.15.112:0/1072735411 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac01b880 con 0x7f5cb4103ea0 2024-03-16T09:10:14.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.454+0000 7f5cbc6df700 1 -- 172.21.15.112:0/1072735411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb41a2a20 con 0x7f5cb4103ea0 2024-03-16T09:10:14.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.455+0000 7f5cbc6df700 1 -- 172.21.15.112:0/1072735411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb4068f90 con 0x7f5cb4103ea0 2024-03-16T09:10:14.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.456+0000 7f5cab7fe700 1 -- 172.21.15.112:0/1072735411 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f5cac022020 con 0x7f5cb4103ea0 2024-03-16T09:10:14.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.457+0000 7f5cab7fe700 1 --2- 172.21.15.112:0/1072735411 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5ca00a2890 0x7f5ca00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:14.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.457+0000 7f5cba47b700 1 --2- 172.21.15.112:0/1072735411 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5ca00a2890 0x7f5ca00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:14.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.457+0000 7f5cab7fe700 1 -- 172.21.15.112:0/1072735411 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7f5cac00f020 con 0x7f5cb4103ea0 2024-03-16T09:10:14.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.457+0000 7f5cba47b700 1 --2- 172.21.15.112:0/1072735411 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5ca00a2890 0x7f5ca00a4d50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4103360 tx=0x7f5cb000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:14.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.461+0000 7f5cab7fe700 1 -- 172.21.15.112:0/1072735411 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5cac011390 con 0x7f5cb4103ea0 2024-03-16T09:10:14.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:14 smithi159 ceph-mon[36739]: Detected new or changed devices on smithi194 2024-03-16T09:10:14.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:14 smithi159 ceph-mon[36739]: Adjusting osd_memory_target on smithi194 to 8540M 2024-03-16T09:10:14.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:14 smithi159 ceph-mon[36739]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:14.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:14 smithi159 ceph-mon[36739]: osdmap e39: 6 total, 6 up, 6 in 2024-03-16T09:10:14.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:14.666+0000 7f5cbc6df700 1 -- 172.21.15.112:0/1072735411 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5cb4062710 con 0x7f5ca00a2890 2024-03-16T09:10:15.274 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:15 smithi112 ceph-mon[32307]: osdmap e40: 6 total, 6 up, 6 in 2024-03-16T09:10:15.274 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:15 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:10:15.274 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:15 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:10:15.274 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:15 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:15.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:15 smithi194 ceph-mon[36729]: osdmap e40: 6 total, 6 up, 6 in 2024-03-16T09:10:15.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:15 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:10:15.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:15 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:10:15.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:15 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:15.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:15 smithi159 ceph-mon[36739]: osdmap e40: 6 total, 6 up, 6 in 2024-03-16T09:10:15.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:15 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:10:15.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:15 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:10:15.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:15 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:16.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:16 smithi112 ceph-mon[32307]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:16.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:16 smithi112 ceph-mon[32307]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:16.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:16 smithi112 ceph-mon[32307]: osdmap e41: 6 total, 6 up, 6 in 2024-03-16T09:10:16.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:16 smithi194 ceph-mon[36729]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:16.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:16 smithi194 ceph-mon[36729]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:16.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:16 smithi194 ceph-mon[36729]: osdmap e41: 6 total, 6 up, 6 in 2024-03-16T09:10:16.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:16 smithi159 ceph-mon[36739]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:16.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:16 smithi159 ceph-mon[36739]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:16.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:16 smithi159 ceph-mon[36739]: osdmap e41: 6 total, 6 up, 6 in 2024-03-16T09:10:17.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:17 smithi112 ceph-mon[32307]: osdmap e42: 6 total, 6 up, 6 in 2024-03-16T09:10:17.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:17 smithi194 ceph-mon[36729]: osdmap e42: 6 total, 6 up, 6 in 2024-03-16T09:10:17.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:17 smithi159 ceph-mon[36739]: osdmap e42: 6 total, 6 up, 6 in 2024-03-16T09:10:18.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:18 smithi112 ceph-mon[32307]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:18.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:18 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2016561394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e250eec-8d30-465d-8352-3f7db892c370"}]: dispatch 2024-03-16T09:10:18.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:18 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2016561394' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e250eec-8d30-465d-8352-3f7db892c370"}]': finished 2024-03-16T09:10:18.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:18 smithi112 ceph-mon[32307]: osdmap e43: 7 total, 6 up, 7 in 2024-03-16T09:10:18.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:18 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T09:10:18.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:18 smithi194 ceph-mon[36729]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:18 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2016561394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e250eec-8d30-465d-8352-3f7db892c370"}]: dispatch 2024-03-16T09:10:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:18 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2016561394' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e250eec-8d30-465d-8352-3f7db892c370"}]': finished 2024-03-16T09:10:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:18 smithi194 ceph-mon[36729]: osdmap e43: 7 total, 6 up, 7 in 2024-03-16T09:10:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:18 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T09:10:18.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:18 smithi159 ceph-mon[36739]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:18.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:18 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2016561394' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e250eec-8d30-465d-8352-3f7db892c370"}]: dispatch 2024-03-16T09:10:18.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:18 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2016561394' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e250eec-8d30-465d-8352-3f7db892c370"}]': finished 2024-03-16T09:10:18.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:18 smithi159 ceph-mon[36739]: osdmap e43: 7 total, 6 up, 7 in 2024-03-16T09:10:18.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:18 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T09:10:19.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:19 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1665061486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:10:19.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:19 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1665061486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:10:19.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:19 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1665061486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:10:20.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:20 smithi112 ceph-mon[32307]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-03-16T09:10:20.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:20 smithi194 ceph-mon[36729]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-03-16T09:10:20.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:20 smithi159 ceph-mon[36739]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-03-16T09:10:22.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:22 smithi112 ceph-mon[32307]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T09:10:22.396 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:22 smithi194 ceph-mon[36729]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T09:10:22.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:22 smithi159 ceph-mon[36739]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T09:10:24.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:24 smithi159 ceph-mon[36739]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-03-16T09:10:24.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:24 smithi112 ceph-mon[32307]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-03-16T09:10:24.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:24 smithi194 ceph-mon[36729]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-03-16T09:10:25.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:25 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-16T09:10:25.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:25 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:25.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:25 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-16T09:10:25.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:25 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:25.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:25 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-16T09:10:25.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:25 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:26.325 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:26 smithi112 ceph-mon[32307]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-16T09:10:26.326 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:26 smithi112 ceph-mon[32307]: Deploying daemon osd.6 on smithi112 2024-03-16T09:10:26.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:26 smithi194 ceph-mon[36729]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-16T09:10:26.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:26 smithi194 ceph-mon[36729]: Deploying daemon osd.6 on smithi112 2024-03-16T09:10:26.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:26 smithi159 ceph-mon[36739]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-16T09:10:26.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:26 smithi159 ceph-mon[36739]: Deploying daemon osd.6 on smithi112 2024-03-16T09:10:27.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:27 smithi112 ceph-mon[32307]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T09:10:27.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:27 smithi194 ceph-mon[36729]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T09:10:27.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:27 smithi159 ceph-mon[36739]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-16T09:10:30.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:29 smithi194 ceph-mon[36729]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-03-16T09:10:30.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:29 smithi159 ceph-mon[36739]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-03-16T09:10:30.262 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:29 smithi112 ceph-mon[32307]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-03-16T09:10:32.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:31 smithi194 ceph-mon[36729]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:32.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:32.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:32.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:32.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:31 smithi159 ceph-mon[36739]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:32.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:32.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:32.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:32.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:31 smithi112 ceph-mon[32307]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:32.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:32.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:32.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:34.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:33 smithi112 ceph-mon[32307]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:34.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:33 smithi194 ceph-mon[36729]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:34.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:33 smithi159 ceph-mon[36739]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:34.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:34.977+0000 7f5cab7fe700 1 -- 172.21.15.112:0/1072735411 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5cb4062710 con 0x7f5ca00a2890 2024-03-16T09:10:34.979 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 6 on host 'smithi112' 2024-03-16T09:10:34.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:34.979+0000 7f5cbc6df700 1 -- 172.21.15.112:0/1072735411 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5ca00a2890 msgr2=0x7f5ca00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:34.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:34.979+0000 7f5cbc6df700 1 --2- 172.21.15.112:0/1072735411 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5ca00a2890 0x7f5ca00a4d50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4103360 tx=0x7f5cb000a3b0 comp rx=0 tx=0).stop 2024-03-16T09:10:34.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:34.979+0000 7f5cbc6df700 1 -- 172.21.15.112:0/1072735411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cb4103ea0 msgr2=0x7f5cb4074c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:34.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:34.979+0000 7f5cbc6df700 1 --2- 172.21.15.112:0/1072735411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cb4103ea0 0x7f5cb4074c00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5cac007dc0 tx=0x7f5cac007df0 comp rx=0 tx=0).stop 2024-03-16T09:10:34.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:34.980+0000 7f5cbc6df700 1 -- 172.21.15.112:0/1072735411 shutdown_connections 2024-03-16T09:10:34.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:34.980+0000 7f5cbc6df700 1 --2- 172.21.15.112:0/1072735411 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5ca00a2890 0x7f5ca00a4d50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:34.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:34.980+0000 7f5cbc6df700 1 --2- 172.21.15.112:0/1072735411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cb4103ea0 0x7f5cb4074c00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:34.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:34.980+0000 7f5cbc6df700 1 --2- 172.21.15.112:0/1072735411 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cb4102300 0x7f5cb4076070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:34.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:34.980+0000 7f5cbc6df700 1 --2- 172.21.15.112:0/1072735411 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cb4103500 0x7f5cb40746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:34.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:34.980+0000 7f5cbc6df700 1 -- 172.21.15.112:0/1072735411 >> 172.21.15.112:0/1072735411 conn(0x7f5cb40fd8e0 msgr2=0x7f5cb4106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:34.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:34.980+0000 7f5cbc6df700 1 -- 172.21.15.112:0/1072735411 shutdown_connections 2024-03-16T09:10:34.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:10:34.980+0000 7f5cbc6df700 1 -- 172.21.15.112:0/1072735411 wait complete. 2024-03-16T09:10:35.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:35 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:35.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:35 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:35.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:35 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:35.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:35 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:10:35.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:35 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:35.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:35 smithi112 ceph-mon[32307]: from='osd.6 [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-16T09:10:35.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:35 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:35.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:35 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:35.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:35 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:35.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:35 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:35.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:35 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:35.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:35 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:35.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:35 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:10:35.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:35 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:35.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:35 smithi194 ceph-mon[36729]: from='osd.6 [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-16T09:10:35.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:35 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:35.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:35 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:35.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:35 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:35.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:35 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:35.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:35 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:35.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:35 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:35.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:35 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:10:35.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:35 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:35.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:35 smithi159 ceph-mon[36739]: from='osd.6 [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-16T09:10:35.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:35 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:35.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:35 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:35.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:35 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:35.848 DEBUG:teuthology.orchestra.run.smithi112:osd.6> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.6.service 2024-03-16T09:10:35.850 INFO:tasks.cephadm:Deploying osd.7 on smithi159 with /dev/vg_nvme/lv_2... 2024-03-16T09:10:35.850 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-16T09:10:36.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:36 smithi112 ceph-mon[32307]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:36.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:36 smithi112 ceph-mon[32307]: from='osd.6 [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-16T09:10:36.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:36 smithi112 ceph-mon[32307]: osdmap e44: 7 total, 6 up, 7 in 2024-03-16T09:10:36.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:36 smithi112 ceph-mon[32307]: from='osd.6 [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-03-16T09:10:36.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:36 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T09:10:36.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:10:36 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:10:36.040+0000 7f4735c38700 -1 osd.6 0 waiting for initial osdmap 2024-03-16T09:10:36.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:10:36 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:10:36.047+0000 7f4731a2b700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T09:10:36.097 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T09:10:36.219 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:36 smithi159 ceph-mon[36739]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:36.219 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:36 smithi159 ceph-mon[36739]: from='osd.6 [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-16T09:10:36.219 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:36 smithi159 ceph-mon[36739]: osdmap e44: 7 total, 6 up, 7 in 2024-03-16T09:10:36.219 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:36 smithi159 ceph-mon[36739]: from='osd.6 [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-03-16T09:10:36.220 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:36 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T09:10:36.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:36 smithi194 ceph-mon[36729]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:36.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:36 smithi194 ceph-mon[36729]: from='osd.6 [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-16T09:10:36.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:36 smithi194 ceph-mon[36729]: osdmap e44: 7 total, 6 up, 7 in 2024-03-16T09:10:36.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:36 smithi194 ceph-mon[36729]: from='osd.6 [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-03-16T09:10:36.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:36 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T09:10:37.186 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:37 smithi112 ceph-mon[32307]: from='osd.6 [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-03-16T09:10:37.186 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:37 smithi112 ceph-mon[32307]: osdmap e45: 7 total, 6 up, 7 in 2024-03-16T09:10:37.186 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:37 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T09:10:37.187 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:37 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T09:10:37.398 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:37 smithi159 ceph-mon[36739]: from='osd.6 [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-03-16T09:10:37.398 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:37 smithi159 ceph-mon[36739]: osdmap e45: 7 total, 6 up, 7 in 2024-03-16T09:10:37.398 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:37 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T09:10:37.399 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:37 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T09:10:37.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:37 smithi194 ceph-mon[36729]: from='osd.6 [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-03-16T09:10:37.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:37 smithi194 ceph-mon[36729]: osdmap e45: 7 total, 6 up, 7 in 2024-03-16T09:10:37.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:37 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T09:10:37.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:37 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T09:10:38.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:38 smithi112 ceph-mon[32307]: purged_snaps scrub starts 2024-03-16T09:10:38.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:38 smithi112 ceph-mon[32307]: purged_snaps scrub ok 2024-03-16T09:10:38.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:38 smithi112 ceph-mon[32307]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:38.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:38 smithi112 ceph-mon[32307]: osd.6 [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923] boot 2024-03-16T09:10:38.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:38 smithi112 ceph-mon[32307]: osdmap e46: 7 total, 7 up, 7 in 2024-03-16T09:10:38.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:38 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T09:10:38.325 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:38 smithi159 ceph-mon[36739]: purged_snaps scrub starts 2024-03-16T09:10:38.325 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:38 smithi159 ceph-mon[36739]: purged_snaps scrub ok 2024-03-16T09:10:38.325 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:38 smithi159 ceph-mon[36739]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:38.325 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:38 smithi159 ceph-mon[36739]: osd.6 [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923] boot 2024-03-16T09:10:38.325 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:38 smithi159 ceph-mon[36739]: osdmap e46: 7 total, 7 up, 7 in 2024-03-16T09:10:38.325 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:38 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T09:10:38.325 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-16T09:10:38.325 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-16T09:10:38.326 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2024-03-16T09:10:38.326 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2024-03-16T09:10:38.326 INFO:teuthology.orchestra.run.smithi159.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.01127 s, 930 MB/s 2024-03-16T09:10:38.326 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2024-03-16T09:10:38.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:38 smithi194 ceph-mon[36729]: purged_snaps scrub starts 2024-03-16T09:10:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:38 smithi194 ceph-mon[36729]: purged_snaps scrub ok 2024-03-16T09:10:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:38 smithi194 ceph-mon[36729]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-16T09:10:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:38 smithi194 ceph-mon[36729]: osd.6 [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923] boot 2024-03-16T09:10:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:38 smithi194 ceph-mon[36729]: osdmap e46: 7 total, 7 up, 7 in 2024-03-16T09:10:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:38 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-16T09:10:39.000 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi159:vg_nvme/lv_2 2024-03-16T09:10:39.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:39 smithi159 ceph-mon[36739]: osdmap e47: 7 total, 7 up, 7 in 2024-03-16T09:10:39.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:39 smithi159 ceph-mon[36739]: Detected new or changed devices on smithi112 2024-03-16T09:10:39.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:39 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:39.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:39 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:39.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:39 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:10:39.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:39 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:10:39.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:39 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:10:39.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:39 smithi159 ceph-mon[36739]: Adjusting osd_memory_target on smithi112 to 5696M 2024-03-16T09:10:39.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:39 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:39.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:39 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:39.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:39 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:10:39.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:39 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:39.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:39 smithi159 ceph-mon[36739]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:39.241 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T09:10:39.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:39 smithi112 ceph-mon[32307]: osdmap e47: 7 total, 7 up, 7 in 2024-03-16T09:10:39.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:39 smithi112 ceph-mon[32307]: Detected new or changed devices on smithi112 2024-03-16T09:10:39.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:39 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:39.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:39 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:39.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:39 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:10:39.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:39 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:10:39.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:39 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:10:39.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:39 smithi112 ceph-mon[32307]: Adjusting osd_memory_target on smithi112 to 5696M 2024-03-16T09:10:39.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:39 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:39.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:39 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:39.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:39 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:10:39.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:39 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:39.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:39 smithi112 ceph-mon[32307]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:39.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:39 smithi194 ceph-mon[36729]: osdmap e47: 7 total, 7 up, 7 in 2024-03-16T09:10:39.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:39 smithi194 ceph-mon[36729]: Detected new or changed devices on smithi112 2024-03-16T09:10:39.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:39 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:39.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:39 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:39.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:39 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:10:39.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:39 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:10:39.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:39 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:10:39.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:39 smithi194 ceph-mon[36729]: Adjusting osd_memory_target on smithi112 to 5696M 2024-03-16T09:10:39.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:39 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:39.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:39 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:39.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:39 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:10:39.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:39 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:39.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:39 smithi194 ceph-mon[36729]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:40.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.867+0000 7f597da14700 1 -- 172.21.15.159:0/1072774081 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5978103ea0 msgr2=0x7f597810a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:40.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.867+0000 7f597da14700 1 --2- 172.21.15.159:0/1072774081 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5978103ea0 0x7f597810a750 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f597000b0d0 tx=0x7f597000b3e0 comp rx=0 tx=0).stop 2024-03-16T09:10:40.871 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.867+0000 7f597da14700 1 -- 172.21.15.159:0/1072774081 shutdown_connections 2024-03-16T09:10:40.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.867+0000 7f597da14700 1 --2- 172.21.15.159:0/1072774081 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5978103ea0 0x7f597810a750 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:40.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.867+0000 7f597da14700 1 --2- 172.21.15.159:0/1072774081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5978102300 0x7f5978102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:40.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.867+0000 7f597da14700 1 --2- 172.21.15.159:0/1072774081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5978103500 0x7f5978103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:40.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.867+0000 7f597da14700 1 -- 172.21.15.159:0/1072774081 >> 172.21.15.159:0/1072774081 conn(0x7f59780fd8e0 msgr2=0x7f59780ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:10:40.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.867+0000 7f597da14700 1 -- 172.21.15.159:0/1072774081 shutdown_connections 2024-03-16T09:10:40.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.867+0000 7f597da14700 1 -- 172.21.15.159:0/1072774081 wait complete. 2024-03-16T09:10:40.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.867+0000 7f597da14700 1 Processor -- start 2024-03-16T09:10:40.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.867+0000 7f597da14700 1 -- start start 2024-03-16T09:10:40.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f597da14700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5978102300 0x7f5978198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:40.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f597da14700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5978103500 0x7f59781989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:40.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f597da14700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5978103ea0 0x7f597819fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:40.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f597da14700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f597810e040 con 0x7f5978103500 2024-03-16T09:10:40.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f597da14700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f597810dec0 con 0x7f5978102300 2024-03-16T09:10:40.872 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f597da14700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f597810e1c0 con 0x7f5978103ea0 2024-03-16T09:10:40.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f5976ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5978102300 0x7f5978198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:40.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f5976ffd700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5978102300 0x7f5978198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39326/0 (socket says 172.21.15.159:39326) 2024-03-16T09:10:40.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f5976ffd700 1 -- 172.21.15.159:0/3827687406 learned_addr learned my addr 172.21.15.159:0/3827687406 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-03-16T09:10:40.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f596ffff700 1 --2- 172.21.15.159:0/3827687406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5978103500 0x7f59781989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:40.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f59777fe700 1 --2- 172.21.15.159:0/3827687406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5978103ea0 0x7f597819fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:40.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f5976ffd700 1 -- 172.21.15.159:0/3827687406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5978103ea0 msgr2=0x7f597819fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:40.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f5976ffd700 1 --2- 172.21.15.159:0/3827687406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5978103ea0 0x7f597819fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:40.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f5976ffd700 1 -- 172.21.15.159:0/3827687406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5978103500 msgr2=0x7f59781989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:10:40.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f5976ffd700 1 --2- 172.21.15.159:0/3827687406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5978103500 0x7f59781989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:10:40.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f5976ffd700 1 -- 172.21.15.159:0/3827687406 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59781a00f0 con 0x7f5978102300 2024-03-16T09:10:40.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f5976ffd700 1 --2- 172.21.15.159:0/3827687406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5978102300 0x7f5978198490 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f596800c990 tx=0x7f596800cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:40.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f597ca12700 1 -- 172.21.15.159:0/3827687406 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968007940 con 0x7f5978102300 2024-03-16T09:10:40.873 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f597ca12700 1 -- 172.21.15.159:0/3827687406 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f596800f450 con 0x7f5978102300 2024-03-16T09:10:40.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f597da14700 1 -- 172.21.15.159:0/3827687406 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59781a03e0 con 0x7f5978102300 2024-03-16T09:10:40.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.868+0000 7f597ca12700 1 -- 172.21.15.159:0/3827687406 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968018640 con 0x7f5978102300 2024-03-16T09:10:40.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.869+0000 7f597da14700 1 -- 172.21.15.159:0/3827687406 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59781a0920 con 0x7f5978102300 2024-03-16T09:10:40.874 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.869+0000 7f597da14700 1 -- 172.21.15.159:0/3827687406 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5978068f90 con 0x7f5978102300 2024-03-16T09:10:40.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.870+0000 7f597ca12700 1 -- 172.21.15.159:0/3827687406 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f596800f600 con 0x7f5978102300 2024-03-16T09:10:40.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.871+0000 7f597ca12700 1 --2- 172.21.15.159:0/3827687406 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f59640a2890 0x7f59640a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:10:40.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.871+0000 7f597ca12700 1 -- 172.21.15.159:0/3827687406 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f59680cdd60 con 0x7f5978102300 2024-03-16T09:10:40.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.871+0000 7f596ffff700 1 --2- 172.21.15.159:0/3827687406 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f59640a2890 0x7f59640a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:10:40.875 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.872+0000 7f596ffff700 1 --2- 172.21.15.159:0/3827687406 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f59640a2890 0x7f59640a4d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5960005950 tx=0x7f596000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:10:40.876 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:40.875+0000 7f597ca12700 1 -- 172.21.15.159:0/3827687406 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5968014030 con 0x7f5978102300 2024-03-16T09:10:41.080 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:10:41.078+0000 7f597da14700 1 -- 172.21.15.159:0/3827687406 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5978062710 con 0x7f59640a2890 2024-03-16T09:10:41.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:41 smithi159 ceph-mon[36739]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:41.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:41 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:10:41.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:41 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:10:41.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:41 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:42.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:41 smithi194 ceph-mon[36729]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:42.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:41 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:10:42.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:41 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:10:42.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:41 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:42.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:41 smithi112 ceph-mon[32307]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:42.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:41 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:10:42.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:41 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:10:42.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:41 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:42.957 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:42 smithi159 ceph-mon[36739]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:43.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:42 smithi194 ceph-mon[36729]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:43.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:42 smithi112 ceph-mon[32307]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:10:44.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:43 smithi194 ceph-mon[36729]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:44.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:43 smithi194 ceph-mon[36729]: from='client.? 172.21.15.159:0/1692165227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2eb2220-107d-411e-9572-8fae0faf0a20"}]: dispatch 2024-03-16T09:10:44.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:43 smithi194 ceph-mon[36729]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2eb2220-107d-411e-9572-8fae0faf0a20"}]: dispatch 2024-03-16T09:10:44.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:43 smithi194 ceph-mon[36729]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2eb2220-107d-411e-9572-8fae0faf0a20"}]': finished 2024-03-16T09:10:44.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:43 smithi194 ceph-mon[36729]: osdmap e48: 8 total, 7 up, 8 in 2024-03-16T09:10:44.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:43 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T09:10:44.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:43 smithi159 ceph-mon[36739]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:44.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:43 smithi159 ceph-mon[36739]: from='client.? 172.21.15.159:0/1692165227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2eb2220-107d-411e-9572-8fae0faf0a20"}]: dispatch 2024-03-16T09:10:44.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:43 smithi159 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2eb2220-107d-411e-9572-8fae0faf0a20"}]: dispatch 2024-03-16T09:10:44.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:43 smithi159 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2eb2220-107d-411e-9572-8fae0faf0a20"}]': finished 2024-03-16T09:10:44.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:43 smithi159 ceph-mon[36739]: osdmap e48: 8 total, 7 up, 8 in 2024-03-16T09:10:44.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:43 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T09:10:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:43 smithi112 ceph-mon[32307]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:43 smithi112 ceph-mon[32307]: from='client.? 172.21.15.159:0/1692165227' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2eb2220-107d-411e-9572-8fae0faf0a20"}]: dispatch 2024-03-16T09:10:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:43 smithi112 ceph-mon[32307]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2eb2220-107d-411e-9572-8fae0faf0a20"}]: dispatch 2024-03-16T09:10:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:43 smithi112 ceph-mon[32307]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2eb2220-107d-411e-9572-8fae0faf0a20"}]': finished 2024-03-16T09:10:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:43 smithi112 ceph-mon[32307]: osdmap e48: 8 total, 7 up, 8 in 2024-03-16T09:10:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T09:10:45.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:44 smithi159 ceph-mon[36739]: from='client.? 172.21.15.159:0/1984840311' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:10:45.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:44 smithi112 ceph-mon[32307]: from='client.? 172.21.15.159:0/1984840311' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:10:45.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:44 smithi194 ceph-mon[36729]: from='client.? 172.21.15.159:0/1984840311' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:10:46.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:45 smithi159 ceph-mon[36739]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:46.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:45 smithi112 ceph-mon[32307]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:46.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:45 smithi194 ceph-mon[36729]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:48.117 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:47 smithi159 ceph-mon[36739]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:47 smithi112 ceph-mon[32307]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:48.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:47 smithi194 ceph-mon[36729]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:49.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:49 smithi159 ceph-mon[36739]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:50.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:49 smithi112 ceph-mon[32307]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:50.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:49 smithi194 ceph-mon[36729]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:52.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:51 smithi159 ceph-mon[36739]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:52.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:51 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-16T09:10:52.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:51 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:52.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:51 smithi112 ceph-mon[32307]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:52.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:51 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-16T09:10:52.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:51 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:52.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:51 smithi194 ceph-mon[36729]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:52.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:51 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-16T09:10:52.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:51 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:10:53.218 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:52 smithi159 ceph-mon[36739]: Deploying daemon osd.7 on smithi159 2024-03-16T09:10:53.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:52 smithi112 ceph-mon[32307]: Deploying daemon osd.7 on smithi159 2024-03-16T09:10:53.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:52 smithi194 ceph-mon[36729]: Deploying daemon osd.7 on smithi159 2024-03-16T09:10:54.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:53 smithi194 ceph-mon[36729]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:54.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:53 smithi159 ceph-mon[36739]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:54.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:53 smithi112 ceph-mon[32307]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:56.224 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:55 smithi159 ceph-mon[36739]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:56.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:55 smithi112 ceph-mon[32307]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:56.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:55 smithi194 ceph-mon[36729]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:57.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:57.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:57.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:57.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:57 smithi159 ceph-mon[36739]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:58.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:58.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:58.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:58.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:57 smithi112 ceph-mon[32307]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:10:58.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:58.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:10:58.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:10:58.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:57 smithi194 ceph-mon[36729]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:11:00.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:59 smithi194 ceph-mon[36729]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:11:00.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:59 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:00.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:59 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:00.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:59 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:00.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:59 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:00.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:10:59 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:00.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:59 smithi159 ceph-mon[36739]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:11:00.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:59 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:00.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:59 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:00.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:59 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:00.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:59 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:00.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:10:59 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:59 smithi112 ceph-mon[32307]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:11:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:10:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:00.645 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:00.642+0000 7f597ca12700 1 -- 172.21.15.159:0/3827687406 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5978062710 con 0x7f59640a2890 2024-03-16T09:11:00.645 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 7 on host 'smithi159' 2024-03-16T09:11:00.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:00.645+0000 7f597da14700 1 -- 172.21.15.159:0/3827687406 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f59640a2890 msgr2=0x7f59640a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:00.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:00.645+0000 7f597da14700 1 --2- 172.21.15.159:0/3827687406 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f59640a2890 0x7f59640a4d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5960005950 tx=0x7f596000b410 comp rx=0 tx=0).stop 2024-03-16T09:11:00.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:00.645+0000 7f597da14700 1 -- 172.21.15.159:0/3827687406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5978102300 msgr2=0x7f5978198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:00.647 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:00.645+0000 7f597da14700 1 --2- 172.21.15.159:0/3827687406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5978102300 0x7f5978198490 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f596800c990 tx=0x7f596800cca0 comp rx=0 tx=0).stop 2024-03-16T09:11:00.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:00.645+0000 7f597da14700 1 -- 172.21.15.159:0/3827687406 shutdown_connections 2024-03-16T09:11:00.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:00.645+0000 7f597da14700 1 --2- 172.21.15.159:0/3827687406 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f59640a2890 0x7f59640a4d50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:00.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:00.645+0000 7f597da14700 1 --2- 172.21.15.159:0/3827687406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5978102300 0x7f5978198490 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:00.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:00.645+0000 7f597da14700 1 --2- 172.21.15.159:0/3827687406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5978103ea0 0x7f597819fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:00.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:00.645+0000 7f597da14700 1 --2- 172.21.15.159:0/3827687406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5978103500 0x7f59781989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:00.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:00.645+0000 7f597da14700 1 -- 172.21.15.159:0/3827687406 >> 172.21.15.159:0/3827687406 conn(0x7f59780fd8e0 msgr2=0x7f59781067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:11:00.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:00.645+0000 7f597da14700 1 -- 172.21.15.159:0/3827687406 shutdown_connections 2024-03-16T09:11:00.648 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:00.645+0000 7f597da14700 1 -- 172.21.15.159:0/3827687406 wait complete. 2024-03-16T09:11:01.317 DEBUG:teuthology.orchestra.run.smithi159:osd.7> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.7.service 2024-03-16T09:11:01.319 INFO:tasks.cephadm:Deploying osd.8 on smithi194 with /dev/vg_nvme/lv_2... 2024-03-16T09:11:01.320 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-16T09:11:01.565 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:11:01.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:01.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:01.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:01.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:01 smithi112 ceph-mon[32307]: from='osd.7 [v2:172.21.15.159:6816/2484095468,v1:172.21.15.159:6817/2484095468]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-16T09:11:01.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:01 smithi112 ceph-mon[32307]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-16T09:11:01.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:01 smithi112 ceph-mon[32307]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:11:01.901 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:01.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:01.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:01.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:01 smithi194 ceph-mon[36729]: from='osd.7 [v2:172.21.15.159:6816/2484095468,v1:172.21.15.159:6817/2484095468]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-16T09:11:01.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:01 smithi194 ceph-mon[36729]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-16T09:11:01.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:01 smithi194 ceph-mon[36729]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:11:01.912 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:01.912 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:01.912 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:01.912 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:01 smithi159 ceph-mon[36739]: from='osd.7 [v2:172.21.15.159:6816/2484095468,v1:172.21.15.159:6817/2484095468]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-16T09:11:01.912 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:01 smithi159 ceph-mon[36739]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-16T09:11:01.912 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:01 smithi159 ceph-mon[36739]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:11:02.901 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:02 smithi194 ceph-mon[36729]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-16T09:11:02.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:02 smithi194 ceph-mon[36729]: osdmap e49: 8 total, 7 up, 8 in 2024-03-16T09:11:02.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:02 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T09:11:02.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:02 smithi194 ceph-mon[36729]: from='osd.7 [v2:172.21.15.159:6816/2484095468,v1:172.21.15.159:6817/2484095468]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:11:02.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:02 smithi194 ceph-mon[36729]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:11:02.916 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:02 smithi159 ceph-mon[36739]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-16T09:11:02.916 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:02 smithi159 ceph-mon[36739]: osdmap e49: 8 total, 7 up, 8 in 2024-03-16T09:11:02.916 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:02 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T09:11:02.916 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:02 smithi159 ceph-mon[36739]: from='osd.7 [v2:172.21.15.159:6816/2484095468,v1:172.21.15.159:6817/2484095468]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:11:02.916 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:02 smithi159 ceph-mon[36739]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:11:03.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:02 smithi112 ceph-mon[32307]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-16T09:11:03.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:02 smithi112 ceph-mon[32307]: osdmap e49: 8 total, 7 up, 8 in 2024-03-16T09:11:03.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:02 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T09:11:03.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:02 smithi112 ceph-mon[32307]: from='osd.7 [v2:172.21.15.159:6816/2484095468,v1:172.21.15.159:6817/2484095468]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:11:03.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:02 smithi112 ceph-mon[32307]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:11:03.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:11:03 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:11:03.126+0000 7f36761e0700 -1 osd.7 0 waiting for initial osdmap 2024-03-16T09:11:03.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:11:03 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:11:03.138+0000 7f366dfcb700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T09:11:03.607 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-16T09:11:03.607 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-16T09:11:03.607 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2024-03-16T09:11:03.607 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2024-03-16T09:11:03.607 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0146442 s, 716 MB/s 2024-03-16T09:11:03.607 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2024-03-16T09:11:03.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:03 smithi194 ceph-mon[36729]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-16T09:11:03.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:03 smithi194 ceph-mon[36729]: osdmap e50: 8 total, 7 up, 8 in 2024-03-16T09:11:03.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:03 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T09:11:03.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:03 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T09:11:03.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:03 smithi194 ceph-mon[36729]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:11:03.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:03 smithi194 ceph-mon[36729]: from='osd.7 ' entity='osd.7' 2024-03-16T09:11:03.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:03 smithi159 ceph-mon[36739]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-16T09:11:03.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:03 smithi159 ceph-mon[36739]: osdmap e50: 8 total, 7 up, 8 in 2024-03-16T09:11:03.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:03 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T09:11:03.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:03 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T09:11:03.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:03 smithi159 ceph-mon[36739]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:11:03.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:03 smithi159 ceph-mon[36739]: from='osd.7 ' entity='osd.7' 2024-03-16T09:11:04.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:03 smithi112 ceph-mon[32307]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-16T09:11:04.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:03 smithi112 ceph-mon[32307]: osdmap e50: 8 total, 7 up, 8 in 2024-03-16T09:11:04.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:03 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T09:11:04.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:03 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T09:11:04.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:03 smithi112 ceph-mon[32307]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-16T09:11:04.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:03 smithi112 ceph-mon[32307]: from='osd.7 ' entity='osd.7' 2024-03-16T09:11:04.260 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi194:vg_nvme/lv_2 2024-03-16T09:11:04.501 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:11:04.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:04 smithi194 ceph-mon[36729]: purged_snaps scrub starts 2024-03-16T09:11:04.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:04 smithi194 ceph-mon[36729]: purged_snaps scrub ok 2024-03-16T09:11:04.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:04 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T09:11:04.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:04 smithi194 ceph-mon[36729]: osd.7 [v2:172.21.15.159:6816/2484095468,v1:172.21.15.159:6817/2484095468] boot 2024-03-16T09:11:04.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:04 smithi194 ceph-mon[36729]: osdmap e51: 8 total, 8 up, 8 in 2024-03-16T09:11:04.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:04 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T09:11:04.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:04 smithi159 ceph-mon[36739]: purged_snaps scrub starts 2024-03-16T09:11:04.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:04 smithi159 ceph-mon[36739]: purged_snaps scrub ok 2024-03-16T09:11:04.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:04 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T09:11:04.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:04 smithi159 ceph-mon[36739]: osd.7 [v2:172.21.15.159:6816/2484095468,v1:172.21.15.159:6817/2484095468] boot 2024-03-16T09:11:04.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:04 smithi159 ceph-mon[36739]: osdmap e51: 8 total, 8 up, 8 in 2024-03-16T09:11:04.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:04 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T09:11:05.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:04 smithi112 ceph-mon[32307]: purged_snaps scrub starts 2024-03-16T09:11:05.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:04 smithi112 ceph-mon[32307]: purged_snaps scrub ok 2024-03-16T09:11:05.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:04 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T09:11:05.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:04 smithi112 ceph-mon[32307]: osd.7 [v2:172.21.15.159:6816/2484095468,v1:172.21.15.159:6817/2484095468] boot 2024-03-16T09:11:05.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:04 smithi112 ceph-mon[32307]: osdmap e51: 8 total, 8 up, 8 in 2024-03-16T09:11:05.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:04 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-16T09:11:05.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:05 smithi194 ceph-mon[36729]: osdmap e52: 8 total, 8 up, 8 in 2024-03-16T09:11:05.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:05 smithi194 ceph-mon[36729]: Detected new or changed devices on smithi159 2024-03-16T09:11:05.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:05.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:05.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:05.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:05.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:05.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:05 smithi194 ceph-mon[36729]: Adjusting osd_memory_target on smithi159 to 5693M 2024-03-16T09:11:05.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:05.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:05.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:05.903 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:05.903 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:05 smithi194 ceph-mon[36729]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:05.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:05 smithi159 ceph-mon[36739]: osdmap e52: 8 total, 8 up, 8 in 2024-03-16T09:11:05.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:05 smithi159 ceph-mon[36739]: Detected new or changed devices on smithi159 2024-03-16T09:11:05.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:05.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:05.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:05.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:05.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:05.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:05 smithi159 ceph-mon[36739]: Adjusting osd_memory_target on smithi159 to 5693M 2024-03-16T09:11:05.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:05.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:05.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:05.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:05.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:05 smithi159 ceph-mon[36739]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:06.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:05 smithi112 ceph-mon[32307]: osdmap e52: 8 total, 8 up, 8 in 2024-03-16T09:11:06.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:05 smithi112 ceph-mon[32307]: Detected new or changed devices on smithi159 2024-03-16T09:11:06.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:06.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:06.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:06.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:06.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:06.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:05 smithi112 ceph-mon[32307]: Adjusting osd_memory_target on smithi159 to 5693M 2024-03-16T09:11:06.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:06.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:06.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:06.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:06.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:05 smithi112 ceph-mon[32307]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:06.190 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.186+0000 7f11a6daf700 1 -- 172.21.15.194:0/590424570 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11a0102da0 msgr2=0x7f11a0109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:06.190 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.186+0000 7f11a6daf700 1 --2- 172.21.15.194:0/590424570 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11a0102da0 0x7f11a0109650 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f119800b0a0 tx=0x7f119800b3b0 comp rx=0 tx=0).stop 2024-03-16T09:11:06.190 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.186+0000 7f11a6daf700 1 -- 172.21.15.194:0/590424570 shutdown_connections 2024-03-16T09:11:06.190 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.186+0000 7f11a6daf700 1 --2- 172.21.15.194:0/590424570 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11a0102da0 0x7f11a0109650 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:06.190 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.186+0000 7f11a6daf700 1 --2- 172.21.15.194:0/590424570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a0101200 0x7f11a0101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:06.190 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.186+0000 7f11a6daf700 1 --2- 172.21.15.194:0/590424570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11a0102400 0x7f11a0102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:06.191 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.186+0000 7f11a6daf700 1 -- 172.21.15.194:0/590424570 >> 172.21.15.194:0/590424570 conn(0x7f11a00fc7a0 msgr2=0x7f11a00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:11:06.191 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.186+0000 7f11a6daf700 1 -- 172.21.15.194:0/590424570 shutdown_connections 2024-03-16T09:11:06.191 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.186+0000 7f11a6daf700 1 -- 172.21.15.194:0/590424570 wait complete. 2024-03-16T09:11:06.191 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.187+0000 7f11a6daf700 1 Processor -- start 2024-03-16T09:11:06.191 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.187+0000 7f11a6daf700 1 -- start start 2024-03-16T09:11:06.191 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.187+0000 7f11a6daf700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11a0101200 0x7f11a01985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:06.191 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.187+0000 7f11a6daf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a0102400 0x7f11a0198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:06.191 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.187+0000 7f11a6daf700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11a0102da0 0x7f11a019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:06.191 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.187+0000 7f11a6daf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11a010cf80 con 0x7f11a0102400 2024-03-16T09:11:06.191 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.187+0000 7f11a6daf700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f11a010ce00 con 0x7f11a0102da0 2024-03-16T09:11:06.191 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.187+0000 7f11a6daf700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f11a010d100 con 0x7f11a0101200 2024-03-16T09:11:06.191 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.187+0000 7f11a4b4b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11a0101200 0x7f11a01985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:06.191 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.187+0000 7f11a4b4b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11a0101200 0x7f11a01985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:53958/0 (socket says 172.21.15.194:53958) 2024-03-16T09:11:06.191 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.187+0000 7f11a4b4b700 1 -- 172.21.15.194:0/4108312348 learned_addr learned my addr 172.21.15.194:0/4108312348 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T09:11:06.192 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.187+0000 7f11a534c700 1 --2- 172.21.15.194:0/4108312348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11a0102da0 0x7f11a019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:06.192 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.187+0000 7f119ffff700 1 --2- 172.21.15.194:0/4108312348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a0102400 0x7f11a0198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:06.192 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.187+0000 7f11a4b4b700 1 -- 172.21.15.194:0/4108312348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11a0102da0 msgr2=0x7f11a019fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:06.192 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.187+0000 7f11a4b4b700 1 --2- 172.21.15.194:0/4108312348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11a0102da0 0x7f11a019fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:06.192 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.187+0000 7f11a4b4b700 1 -- 172.21.15.194:0/4108312348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a0102400 msgr2=0x7f11a0198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:06.192 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.187+0000 7f11a4b4b700 1 --2- 172.21.15.194:0/4108312348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a0102400 0x7f11a0198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:06.192 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.187+0000 7f11a4b4b700 1 -- 172.21.15.194:0/4108312348 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a01a0230 con 0x7f11a0101200 2024-03-16T09:11:06.192 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.187+0000 7f11a4b4b700 1 --2- 172.21.15.194:0/4108312348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11a0101200 0x7f11a01985d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f119400d9b0 tx=0x7f119400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:11:06.192 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.188+0000 7f119dffb700 1 -- 172.21.15.194:0/4108312348 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194017030 con 0x7f11a0101200 2024-03-16T09:11:06.192 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.188+0000 7f11a6daf700 1 -- 172.21.15.194:0/4108312348 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a01a0520 con 0x7f11a0101200 2024-03-16T09:11:06.192 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.188+0000 7f119dffb700 1 -- 172.21.15.194:0/4108312348 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1194004520 con 0x7f11a0101200 2024-03-16T09:11:06.192 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.188+0000 7f119dffb700 1 -- 172.21.15.194:0/4108312348 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11940039e0 con 0x7f11a0101200 2024-03-16T09:11:06.192 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.188+0000 7f11a6daf700 1 -- 172.21.15.194:0/4108312348 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a01a0a60 con 0x7f11a0101200 2024-03-16T09:11:06.193 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.189+0000 7f11a6daf700 1 -- 172.21.15.194:0/4108312348 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a0068f90 con 0x7f11a0101200 2024-03-16T09:11:06.193 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.190+0000 7f119dffb700 1 -- 172.21.15.194:0/4108312348 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f119401a070 con 0x7f11a0101200 2024-03-16T09:11:06.193 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.191+0000 7f119dffb700 1 --2- 172.21.15.194:0/4108312348 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f11900a2890 0x7f11900a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:06.193 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.191+0000 7f119dffb700 1 -- 172.21.15.194:0/4108312348 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7f11940cdbd0 con 0x7f11a0101200 2024-03-16T09:11:06.193 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.191+0000 7f119ffff700 1 --2- 172.21.15.194:0/4108312348 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f11900a2890 0x7f11900a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:06.194 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.192+0000 7f119ffff700 1 --2- 172.21.15.194:0/4108312348 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f11900a2890 0x7f11900a4d50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f118c004630 tx=0x7f118c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:11:06.196 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.195+0000 7f119dffb700 1 -- 172.21.15.194:0/4108312348 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f119400e350 con 0x7f11a0101200 2024-03-16T09:11:06.402 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:06.400+0000 7f11a6daf700 1 -- 172.21.15.194:0/4108312348 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f11a00fe8e0 con 0x7f11900a2890 2024-03-16T09:11:06.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:06 smithi159 ceph-mon[36739]: osdmap e53: 8 total, 8 up, 8 in 2024-03-16T09:11:06.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:06 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:11:06.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:06 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:11:06.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:06 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:07.002 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:06 smithi194 ceph-mon[36729]: osdmap e53: 8 total, 8 up, 8 in 2024-03-16T09:11:07.002 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:06 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:11:07.002 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:06 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:11:07.002 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:06 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:07.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:06 smithi112 ceph-mon[32307]: osdmap e53: 8 total, 8 up, 8 in 2024-03-16T09:11:07.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:06 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:11:07.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:06 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:11:07.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:06 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:08.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:07 smithi194 ceph-mon[36729]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:11:08.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:07 smithi194 ceph-mon[36729]: osdmap e54: 8 total, 8 up, 8 in 2024-03-16T09:11:08.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:07 smithi194 ceph-mon[36729]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:08.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:07 smithi159 ceph-mon[36739]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:11:08.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:07 smithi159 ceph-mon[36739]: osdmap e54: 8 total, 8 up, 8 in 2024-03-16T09:11:08.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:07 smithi159 ceph-mon[36739]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:08.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:07 smithi112 ceph-mon[32307]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:11:08.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:07 smithi112 ceph-mon[32307]: osdmap e54: 8 total, 8 up, 8 in 2024-03-16T09:11:08.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:07 smithi112 ceph-mon[32307]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:09.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:08 smithi194 ceph-mon[36729]: osdmap e55: 8 total, 8 up, 8 in 2024-03-16T09:11:09.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:08 smithi159 ceph-mon[36739]: osdmap e55: 8 total, 8 up, 8 in 2024-03-16T09:11:09.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:08 smithi112 ceph-mon[32307]: osdmap e55: 8 total, 8 up, 8 in 2024-03-16T09:11:10.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:09 smithi194 ceph-mon[36729]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:10.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:09 smithi194 ceph-mon[36729]: from='client.? 172.21.15.194:0/2364847222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9ce2ec1-b2db-4ce1-b306-ea6e333b02de"}]: dispatch 2024-03-16T09:11:10.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:09 smithi194 ceph-mon[36729]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9ce2ec1-b2db-4ce1-b306-ea6e333b02de"}]: dispatch 2024-03-16T09:11:10.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:09 smithi194 ceph-mon[36729]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9ce2ec1-b2db-4ce1-b306-ea6e333b02de"}]': finished 2024-03-16T09:11:10.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:09 smithi194 ceph-mon[36729]: osdmap e56: 9 total, 8 up, 9 in 2024-03-16T09:11:10.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:09 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T09:11:10.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:09 smithi194 ceph-mon[36729]: from='client.? 172.21.15.194:0/2406605869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:11:10.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:09 smithi159 ceph-mon[36739]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:10.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:09 smithi159 ceph-mon[36739]: from='client.? 172.21.15.194:0/2364847222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9ce2ec1-b2db-4ce1-b306-ea6e333b02de"}]: dispatch 2024-03-16T09:11:10.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:09 smithi159 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9ce2ec1-b2db-4ce1-b306-ea6e333b02de"}]: dispatch 2024-03-16T09:11:10.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:09 smithi159 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9ce2ec1-b2db-4ce1-b306-ea6e333b02de"}]': finished 2024-03-16T09:11:10.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:09 smithi159 ceph-mon[36739]: osdmap e56: 9 total, 8 up, 9 in 2024-03-16T09:11:10.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:09 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T09:11:10.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:09 smithi159 ceph-mon[36739]: from='client.? 172.21.15.194:0/2406605869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:11:10.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:09 smithi112 ceph-mon[32307]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:10.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:09 smithi112 ceph-mon[32307]: from='client.? 172.21.15.194:0/2364847222' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9ce2ec1-b2db-4ce1-b306-ea6e333b02de"}]: dispatch 2024-03-16T09:11:10.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:09 smithi112 ceph-mon[32307]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c9ce2ec1-b2db-4ce1-b306-ea6e333b02de"}]: dispatch 2024-03-16T09:11:10.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:09 smithi112 ceph-mon[32307]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9ce2ec1-b2db-4ce1-b306-ea6e333b02de"}]': finished 2024-03-16T09:11:10.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:09 smithi112 ceph-mon[32307]: osdmap e56: 9 total, 8 up, 9 in 2024-03-16T09:11:10.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:09 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T09:11:10.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:09 smithi112 ceph-mon[32307]: from='client.? 172.21.15.194:0/2406605869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:11:12.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:11 smithi159 ceph-mon[36739]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:12.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:11 smithi112 ceph-mon[32307]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:12.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:11 smithi194 ceph-mon[36729]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:14.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:13 smithi194 ceph-mon[36729]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:14.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:13 smithi159 ceph-mon[36739]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:14.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:13 smithi112 ceph-mon[32307]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:16.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:15 smithi194 ceph-mon[36729]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:16.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:15 smithi159 ceph-mon[36739]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:16.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:15 smithi112 ceph-mon[32307]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:17.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:16 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-16T09:11:17.206 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:16 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:17.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:16 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-16T09:11:17.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:16 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:17.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:16 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-16T09:11:17.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:16 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:18.079 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:17 smithi194 ceph-mon[36729]: Deploying daemon osd.8 on smithi194 2024-03-16T09:11:18.079 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:17 smithi194 ceph-mon[36729]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:18.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:17 smithi159 ceph-mon[36739]: Deploying daemon osd.8 on smithi194 2024-03-16T09:11:18.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:17 smithi159 ceph-mon[36739]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:17 smithi112 ceph-mon[32307]: Deploying daemon osd.8 on smithi194 2024-03-16T09:11:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:17 smithi112 ceph-mon[32307]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:20.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:19 smithi159 ceph-mon[36739]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:20.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:19 smithi112 ceph-mon[32307]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:20.380 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:19 smithi194 ceph-mon[36729]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:22.063 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:21 smithi194 ceph-mon[36729]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:22.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:21 smithi159 ceph-mon[36739]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:22.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:21 smithi112 ceph-mon[32307]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:23.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:23 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:23.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:23 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:23.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:23 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:23.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:23 smithi112 ceph-mon[32307]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:23.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:23 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:23.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:23 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:23.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:23 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:23.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:23 smithi194 ceph-mon[36729]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:23.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:23 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:23.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:23 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:23.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:23 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:23.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:23 smithi159 ceph-mon[36739]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:25.572 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:25.570+0000 7f119dffb700 1 -- 172.21.15.194:0/4108312348 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f11a00fe8e0 con 0x7f11900a2890 2024-03-16T09:11:25.573 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 8 on host 'smithi194' 2024-03-16T09:11:25.574 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:25.572+0000 7f11a6daf700 1 -- 172.21.15.194:0/4108312348 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f11900a2890 msgr2=0x7f11900a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:25.575 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:25.572+0000 7f11a6daf700 1 --2- 172.21.15.194:0/4108312348 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f11900a2890 0x7f11900a4d50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f118c004630 tx=0x7f118c00b3c0 comp rx=0 tx=0).stop 2024-03-16T09:11:25.575 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:25.572+0000 7f11a6daf700 1 -- 172.21.15.194:0/4108312348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11a0101200 msgr2=0x7f11a01985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:25.575 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:25.572+0000 7f11a6daf700 1 --2- 172.21.15.194:0/4108312348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11a0101200 0x7f11a01985d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f119400d9b0 tx=0x7f119400dcc0 comp rx=0 tx=0).stop 2024-03-16T09:11:25.575 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:25.572+0000 7f11a6daf700 1 -- 172.21.15.194:0/4108312348 shutdown_connections 2024-03-16T09:11:25.575 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:25.572+0000 7f11a6daf700 1 --2- 172.21.15.194:0/4108312348 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f11900a2890 0x7f11900a4d50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:25.575 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:25.572+0000 7f11a6daf700 1 --2- 172.21.15.194:0/4108312348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11a0102da0 0x7f11a019fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:25.575 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:25.572+0000 7f11a6daf700 1 --2- 172.21.15.194:0/4108312348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f11a0101200 0x7f11a01985d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:25.575 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:25.572+0000 7f11a6daf700 1 --2- 172.21.15.194:0/4108312348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11a0102400 0x7f11a0198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:25.575 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:25.572+0000 7f11a6daf700 1 -- 172.21.15.194:0/4108312348 >> 172.21.15.194:0/4108312348 conn(0x7f11a00fc7a0 msgr2=0x7f11a00fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:11:25.575 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:25.572+0000 7f11a6daf700 1 -- 172.21.15.194:0/4108312348 shutdown_connections 2024-03-16T09:11:25.575 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:11:25.572+0000 7f11a6daf700 1 -- 172.21.15.194:0/4108312348 wait complete. 2024-03-16T09:11:26.036 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:25 smithi194 ceph-mon[36729]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:26.036 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:25 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:26.036 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:25 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:26.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:25 smithi159 ceph-mon[36739]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:26.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:25 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:26.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:25 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:26.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:25 smithi112 ceph-mon[32307]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:26.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:25 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:26.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:25 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:26.362 DEBUG:teuthology.orchestra.run.smithi194:osd.8> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.8.service 2024-03-16T09:11:26.364 INFO:tasks.cephadm:Deploying osd.9 on smithi112 with /dev/vg_nvme/lv_1... 2024-03-16T09:11:26.364 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-16T09:11:26.615 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:11:26.989 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:26 smithi112 ceph-mon[32307]: from='osd.8 [v2:172.21.15.194:6816/3334422298,v1:172.21.15.194:6817/3334422298]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-16T09:11:26.990 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:26 smithi112 ceph-mon[32307]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-16T09:11:26.990 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:26 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:26.990 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:26 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:27.191 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:26 smithi194 ceph-mon[36729]: from='osd.8 [v2:172.21.15.194:6816/3334422298,v1:172.21.15.194:6817/3334422298]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-16T09:11:27.192 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:26 smithi194 ceph-mon[36729]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-16T09:11:27.192 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:26 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:27.192 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:26 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:27.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:26 smithi159 ceph-mon[36739]: from='osd.8 [v2:172.21.15.194:6816/3334422298,v1:172.21.15.194:6817/3334422298]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-16T09:11:27.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:26 smithi159 ceph-mon[36739]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-16T09:11:27.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:26 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:27.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:26 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:28.059 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:27 smithi112 ceph-mon[32307]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:28.059 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:27 smithi112 ceph-mon[32307]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-16T09:11:28.059 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:27 smithi112 ceph-mon[32307]: from='osd.8 [v2:172.21.15.194:6816/3334422298,v1:172.21.15.194:6817/3334422298]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:11:28.059 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:27 smithi112 ceph-mon[32307]: osdmap e57: 9 total, 8 up, 9 in 2024-03-16T09:11:28.059 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:27 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T09:11:28.059 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:27 smithi112 ceph-mon[32307]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:11:28.203 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:27 smithi194 ceph-mon[36729]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:28.203 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:27 smithi194 ceph-mon[36729]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-16T09:11:28.203 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:27 smithi194 ceph-mon[36729]: from='osd.8 [v2:172.21.15.194:6816/3334422298,v1:172.21.15.194:6817/3334422298]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:11:28.203 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:27 smithi194 ceph-mon[36729]: osdmap e57: 9 total, 8 up, 9 in 2024-03-16T09:11:28.203 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:27 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T09:11:28.203 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:27 smithi194 ceph-mon[36729]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:11:28.203 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:11:27 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:11:27.921+0000 7fb79ad4f700 -1 osd.8 0 waiting for initial osdmap 2024-03-16T09:11:28.203 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:11:27 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:11:27.932+0000 7fb796b42700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T09:11:28.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:27 smithi159 ceph-mon[36739]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:28.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:27 smithi159 ceph-mon[36739]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-16T09:11:28.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:27 smithi159 ceph-mon[36739]: from='osd.8 [v2:172.21.15.194:6816/3334422298,v1:172.21.15.194:6817/3334422298]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:11:28.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:27 smithi159 ceph-mon[36739]: osdmap e57: 9 total, 8 up, 9 in 2024-03-16T09:11:28.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:27 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T09:11:28.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:27 smithi159 ceph-mon[36739]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:11:28.643 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-16T09:11:28.643 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-16T09:11:28.643 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-03-16T09:11:28.643 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-03-16T09:11:28.643 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011497 s, 912 MB/s 2024-03-16T09:11:28.643 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-03-16T09:11:29.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:28 smithi112 ceph-mon[32307]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T09:11:29.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:28 smithi112 ceph-mon[32307]: osdmap e58: 9 total, 8 up, 9 in 2024-03-16T09:11:29.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:28 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T09:11:29.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:28 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T09:11:29.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:28 smithi112 ceph-mon[32307]: osd.8 [v2:172.21.15.194:6816/3334422298,v1:172.21.15.194:6817/3334422298] boot 2024-03-16T09:11:29.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:28 smithi112 ceph-mon[32307]: osdmap e59: 9 total, 9 up, 9 in 2024-03-16T09:11:29.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:28 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T09:11:29.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:28 smithi159 ceph-mon[36739]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T09:11:29.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:28 smithi159 ceph-mon[36739]: osdmap e58: 9 total, 8 up, 9 in 2024-03-16T09:11:29.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:28 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T09:11:29.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:28 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T09:11:29.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:28 smithi159 ceph-mon[36739]: osd.8 [v2:172.21.15.194:6816/3334422298,v1:172.21.15.194:6817/3334422298] boot 2024-03-16T09:11:29.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:28 smithi159 ceph-mon[36739]: osdmap e59: 9 total, 9 up, 9 in 2024-03-16T09:11:29.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:28 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T09:11:29.248 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi112:vg_nvme/lv_1 2024-03-16T09:11:29.360 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:28 smithi194 ceph-mon[36729]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T09:11:29.360 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:28 smithi194 ceph-mon[36729]: osdmap e58: 9 total, 8 up, 9 in 2024-03-16T09:11:29.360 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:28 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T09:11:29.360 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:28 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T09:11:29.360 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:28 smithi194 ceph-mon[36729]: osd.8 [v2:172.21.15.194:6816/3334422298,v1:172.21.15.194:6817/3334422298] boot 2024-03-16T09:11:29.360 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:28 smithi194 ceph-mon[36729]: osdmap e59: 9 total, 9 up, 9 in 2024-03-16T09:11:29.360 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:28 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-16T09:11:29.496 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:11:30.076 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:29 smithi194 ceph-mon[36729]: purged_snaps scrub starts 2024-03-16T09:11:30.076 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:29 smithi194 ceph-mon[36729]: purged_snaps scrub ok 2024-03-16T09:11:30.077 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:29 smithi194 ceph-mon[36729]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:30.077 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:29 smithi194 ceph-mon[36729]: osdmap e60: 9 total, 9 up, 9 in 2024-03-16T09:11:30.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:29 smithi159 ceph-mon[36739]: purged_snaps scrub starts 2024-03-16T09:11:30.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:29 smithi159 ceph-mon[36739]: purged_snaps scrub ok 2024-03-16T09:11:30.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:29 smithi159 ceph-mon[36739]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:30.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:29 smithi159 ceph-mon[36739]: osdmap e60: 9 total, 9 up, 9 in 2024-03-16T09:11:30.336 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:29 smithi112 ceph-mon[32307]: purged_snaps scrub starts 2024-03-16T09:11:30.336 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:29 smithi112 ceph-mon[32307]: purged_snaps scrub ok 2024-03-16T09:11:30.336 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:29 smithi112 ceph-mon[32307]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-16T09:11:30.336 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:29 smithi112 ceph-mon[32307]: osdmap e60: 9 total, 9 up, 9 in 2024-03-16T09:11:31.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.210+0000 7f3a46190700 1 -- 172.21.15.112:0/61221691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a40104000 msgr2=0x7f3a4010a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:31.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.210+0000 7f3a46190700 1 --2- 172.21.15.112:0/61221691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a40104000 0x7f3a4010a8b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3a380099c0 tx=0x7f3a38009cd0 comp rx=0 tx=0).stop 2024-03-16T09:11:31.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.210+0000 7f3a46190700 1 -- 172.21.15.112:0/61221691 shutdown_connections 2024-03-16T09:11:31.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.210+0000 7f3a46190700 1 --2- 172.21.15.112:0/61221691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a401005e0 0x7f3a40100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:31.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.210+0000 7f3a46190700 1 --2- 172.21.15.112:0/61221691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a40104000 0x7f3a4010a8b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:31.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.210+0000 7f3a46190700 1 --2- 172.21.15.112:0/61221691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a40103660 0x7f3a40103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:31.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.210+0000 7f3a46190700 1 -- 172.21.15.112:0/61221691 >> 172.21.15.112:0/61221691 conn(0x7f3a400fbf40 msgr2=0x7f3a400fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:11:31.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.210+0000 7f3a46190700 1 -- 172.21.15.112:0/61221691 shutdown_connections 2024-03-16T09:11:31.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.210+0000 7f3a46190700 1 -- 172.21.15.112:0/61221691 wait complete. 2024-03-16T09:11:31.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.210+0000 7f3a46190700 1 Processor -- start 2024-03-16T09:11:31.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.211+0000 7f3a46190700 1 -- start start 2024-03-16T09:11:31.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.211+0000 7f3a46190700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a401005e0 0x7f3a40102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:31.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.211+0000 7f3a46190700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a40103660 0x7f3a40101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:31.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.211+0000 7f3a46190700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a40104000 0x7f3a40101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:31.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.211+0000 7f3a46190700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4010e020 con 0x7f3a40104000 2024-03-16T09:11:31.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.211+0000 7f3a46190700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4010dea0 con 0x7f3a401005e0 2024-03-16T09:11:31.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.211+0000 7f3a46190700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4010e1a0 con 0x7f3a40103660 2024-03-16T09:11:31.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.211+0000 7f3a3ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a40104000 0x7f3a40101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:31.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.211+0000 7f3a3ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a40104000 0x7f3a40101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59824/0 (socket says 172.21.15.112:59824) 2024-03-16T09:11:31.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.211+0000 7f3a3ffff700 1 -- 172.21.15.112:0/3716243178 learned_addr learned my addr 172.21.15.112:0/3716243178 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:11:31.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.211+0000 7f3a3effd700 1 --2- 172.21.15.112:0/3716243178 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a40103660 0x7f3a40101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:31.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.211+0000 7f3a3f7fe700 1 --2- 172.21.15.112:0/3716243178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a401005e0 0x7f3a40102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:31.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.211+0000 7f3a3ffff700 1 -- 172.21.15.112:0/3716243178 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a40103660 msgr2=0x7f3a40101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:31.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.211+0000 7f3a3ffff700 1 --2- 172.21.15.112:0/3716243178 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a40103660 0x7f3a40101610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:31.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.211+0000 7f3a3ffff700 1 -- 172.21.15.112:0/3716243178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a401005e0 msgr2=0x7f3a40102fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:31.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.211+0000 7f3a3ffff700 1 --2- 172.21.15.112:0/3716243178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a401005e0 0x7f3a40102fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:31.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.211+0000 7f3a3ffff700 1 -- 172.21.15.112:0/3716243178 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a40102400 con 0x7f3a40104000 2024-03-16T09:11:31.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.211+0000 7f3a3f7fe700 1 --2- 172.21.15.112:0/3716243178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a401005e0 0x7f3a40102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:11:31.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.211+0000 7f3a3ffff700 1 --2- 172.21.15.112:0/3716243178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a40104000 0x7f3a40101b50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009990 tx=0x7f3a38009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:11:31.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.212+0000 7f3a3cff9700 1 -- 172.21.15.112:0/3716243178 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a38027070 con 0x7f3a40104000 2024-03-16T09:11:31.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.212+0000 7f3a46190700 1 -- 172.21.15.112:0/3716243178 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a401a2710 con 0x7f3a40104000 2024-03-16T09:11:31.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.212+0000 7f3a3cff9700 1 -- 172.21.15.112:0/3716243178 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3a38004200 con 0x7f3a40104000 2024-03-16T09:11:31.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.212+0000 7f3a3cff9700 1 -- 172.21.15.112:0/3716243178 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3800f460 con 0x7f3a40104000 2024-03-16T09:11:31.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.212+0000 7f3a46190700 1 -- 172.21.15.112:0/3716243178 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a401a2b60 con 0x7f3a40104000 2024-03-16T09:11:31.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.213+0000 7f3a46190700 1 -- 172.21.15.112:0/3716243178 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a40068f90 con 0x7f3a40104000 2024-03-16T09:11:31.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.214+0000 7f3a3cff9700 1 -- 172.21.15.112:0/3716243178 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f3a3802a070 con 0x7f3a40104000 2024-03-16T09:11:31.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.214+0000 7f3a3cff9700 1 --2- 172.21.15.112:0/3716243178 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3a280a2890 0x7f3a280a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:31.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.215+0000 7f3a3f7fe700 1 --2- 172.21.15.112:0/3716243178 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3a280a2890 0x7f3a280a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:31.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.215+0000 7f3a3cff9700 1 -- 172.21.15.112:0/3716243178 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5858+0+0 (secure 0 0 0) 0x7f3a380cf620 con 0x7f3a40104000 2024-03-16T09:11:31.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.215+0000 7f3a3f7fe700 1 --2- 172.21.15.112:0/3716243178 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3a280a2890 0x7f3a280a4d50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3a40069cc0 tx=0x7f3a30007c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:11:31.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.218+0000 7f3a3cff9700 1 -- 172.21.15.112:0/3716243178 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3a38031070 con 0x7f3a40104000 2024-03-16T09:11:31.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:31 smithi112 ceph-mon[32307]: Detected new or changed devices on smithi194 2024-03-16T09:11:31.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:31.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:31.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:31.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:31.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:31.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:31 smithi112 ceph-mon[32307]: Adjusting osd_memory_target on smithi194 to 5693M 2024-03-16T09:11:31.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:31.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:31.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:31.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:31.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:31.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:31.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:31.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:31.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:31 smithi112 ceph-mon[32307]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:31.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:31 smithi112 ceph-mon[32307]: osdmap e61: 9 total, 9 up, 9 in 2024-03-16T09:11:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:31 smithi194 ceph-mon[36729]: Detected new or changed devices on smithi194 2024-03-16T09:11:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:31 smithi194 ceph-mon[36729]: Adjusting osd_memory_target on smithi194 to 5693M 2024-03-16T09:11:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:31.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:31.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:31.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:31 smithi194 ceph-mon[36729]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:31.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:31 smithi194 ceph-mon[36729]: osdmap e61: 9 total, 9 up, 9 in 2024-03-16T09:11:31.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:31.423+0000 7f3a46190700 1 -- 172.21.15.112:0/3716243178 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3a40062710 con 0x7f3a280a2890 2024-03-16T09:11:31.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:31 smithi159 ceph-mon[36739]: Detected new or changed devices on smithi194 2024-03-16T09:11:31.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:31.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:31 smithi159 ceph-mon[36739]: Adjusting osd_memory_target on smithi194 to 5693M 2024-03-16T09:11:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:31 smithi159 ceph-mon[36739]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:31 smithi159 ceph-mon[36739]: osdmap e61: 9 total, 9 up, 9 in 2024-03-16T09:11:32.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:32 smithi112 ceph-mon[32307]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:11:32.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:32 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:11:32.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:32 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:11:32.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:32 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:32.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:32 smithi112 ceph-mon[32307]: osdmap e62: 9 total, 9 up, 9 in 2024-03-16T09:11:32.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:32 smithi194 ceph-mon[36729]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:11:32.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:32 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:11:32.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:32 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:11:32.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:32 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:32.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:32 smithi194 ceph-mon[36729]: osdmap e62: 9 total, 9 up, 9 in 2024-03-16T09:11:32.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:32 smithi159 ceph-mon[36739]: from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:11:32.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:32 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:11:32.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:32 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:11:32.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:32 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:32.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:32 smithi159 ceph-mon[36739]: osdmap e62: 9 total, 9 up, 9 in 2024-03-16T09:11:33.466 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:33 smithi112 ceph-mon[32307]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:33.466 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:33 smithi112 ceph-mon[32307]: osdmap e63: 9 total, 9 up, 9 in 2024-03-16T09:11:33.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:33 smithi194 ceph-mon[36729]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:33.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:33 smithi194 ceph-mon[36729]: osdmap e63: 9 total, 9 up, 9 in 2024-03-16T09:11:33.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:33 smithi159 ceph-mon[36739]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:33.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:33 smithi159 ceph-mon[36739]: osdmap e63: 9 total, 9 up, 9 in 2024-03-16T09:11:34.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:34 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2796452962' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2b22c20-177b-41ec-b579-f18bc619d79c"}]: dispatch 2024-03-16T09:11:34.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:34 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2796452962' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2b22c20-177b-41ec-b579-f18bc619d79c"}]': finished 2024-03-16T09:11:34.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:34 smithi112 ceph-mon[32307]: osdmap e64: 10 total, 9 up, 10 in 2024-03-16T09:11:34.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:34 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T09:11:34.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:34 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2796452962' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2b22c20-177b-41ec-b579-f18bc619d79c"}]: dispatch 2024-03-16T09:11:34.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:34 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2796452962' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2b22c20-177b-41ec-b579-f18bc619d79c"}]': finished 2024-03-16T09:11:34.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:34 smithi194 ceph-mon[36729]: osdmap e64: 10 total, 9 up, 10 in 2024-03-16T09:11:34.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:34 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T09:11:34.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:34 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2796452962' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e2b22c20-177b-41ec-b579-f18bc619d79c"}]: dispatch 2024-03-16T09:11:34.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:34 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2796452962' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e2b22c20-177b-41ec-b579-f18bc619d79c"}]': finished 2024-03-16T09:11:34.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:34 smithi159 ceph-mon[36739]: osdmap e64: 10 total, 9 up, 10 in 2024-03-16T09:11:34.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:34 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T09:11:35.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:35 smithi112 ceph-mon[32307]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-16T09:11:35.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:35 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/597812053' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:11:35.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:35 smithi194 ceph-mon[36729]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-16T09:11:35.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:35 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/597812053' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:11:35.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:35 smithi159 ceph-mon[36739]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-16T09:11:35.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:35 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/597812053' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:11:38.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:37 smithi159 ceph-mon[36739]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-16T09:11:38.283 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:37 smithi112 ceph-mon[32307]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-16T09:11:38.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:37 smithi194 ceph-mon[36729]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-16T09:11:40.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:39 smithi112 ceph-mon[32307]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-16T09:11:40.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:39 smithi159 ceph-mon[36739]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-16T09:11:40.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:39 smithi194 ceph-mon[36729]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-03-16T09:11:42.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:41 smithi112 ceph-mon[32307]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-16T09:11:42.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:41 smithi159 ceph-mon[36739]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-16T09:11:42.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:41 smithi194 ceph-mon[36729]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-16T09:11:43.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:42 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-16T09:11:43.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:42 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:43.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:42 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-16T09:11:43.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:42 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:43.303 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:42 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-16T09:11:43.303 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:42 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:44.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:43 smithi112 ceph-mon[32307]: Deploying daemon osd.9 on smithi112 2024-03-16T09:11:44.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:43 smithi112 ceph-mon[32307]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-16T09:11:44.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:43 smithi159 ceph-mon[36739]: Deploying daemon osd.9 on smithi112 2024-03-16T09:11:44.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:43 smithi159 ceph-mon[36739]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-16T09:11:44.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:43 smithi194 ceph-mon[36729]: Deploying daemon osd.9 on smithi112 2024-03-16T09:11:44.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:43 smithi194 ceph-mon[36729]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-16T09:11:46.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:45 smithi112 ceph-mon[32307]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-03-16T09:11:46.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:45 smithi159 ceph-mon[36739]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-03-16T09:11:46.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:45 smithi194 ceph-mon[36729]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-03-16T09:11:48.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:47 smithi159 ceph-mon[36739]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:47 smithi112 ceph-mon[32307]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:48.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:47 smithi194 ceph-mon[36729]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:49.194 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:48 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:49.194 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:48 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:49.194 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:48 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:49.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:48 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:49.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:48 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:49.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:48 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:49.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:48 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:49.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:48 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:49.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:48 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:50.202 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:49 smithi112 ceph-mon[32307]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:50.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:49 smithi159 ceph-mon[36739]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:50.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:49 smithi194 ceph-mon[36729]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:52.185 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:51 smithi112 ceph-mon[32307]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:52.186 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:51 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:52.186 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:51 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:52.186 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:51 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:52.186 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:51 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:52.186 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:51 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:52.183+0000 7f3a3cff9700 1 -- 172.21.15.112:0/3716243178 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3a40062710 con 0x7f3a280a2890 2024-03-16T09:11:52.186 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 9 on host 'smithi112' 2024-03-16T09:11:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:52.186+0000 7f3a46190700 1 -- 172.21.15.112:0/3716243178 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3a280a2890 msgr2=0x7f3a280a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:52.186+0000 7f3a46190700 1 --2- 172.21.15.112:0/3716243178 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3a280a2890 0x7f3a280a4d50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3a40069cc0 tx=0x7f3a30007c20 comp rx=0 tx=0).stop 2024-03-16T09:11:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:52.186+0000 7f3a46190700 1 -- 172.21.15.112:0/3716243178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a40104000 msgr2=0x7f3a40101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:52.186+0000 7f3a46190700 1 --2- 172.21.15.112:0/3716243178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a40104000 0x7f3a40101b50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009990 tx=0x7f3a38009580 comp rx=0 tx=0).stop 2024-03-16T09:11:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:52.186+0000 7f3a46190700 1 -- 172.21.15.112:0/3716243178 shutdown_connections 2024-03-16T09:11:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:52.186+0000 7f3a46190700 1 --2- 172.21.15.112:0/3716243178 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3a280a2890 0x7f3a280a4d50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:52.186+0000 7f3a46190700 1 --2- 172.21.15.112:0/3716243178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a401005e0 0x7f3a40102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:52.186+0000 7f3a46190700 1 --2- 172.21.15.112:0/3716243178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a40104000 0x7f3a40101b50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:52.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:52.186+0000 7f3a46190700 1 --2- 172.21.15.112:0/3716243178 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a40103660 0x7f3a40101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:52.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:52.186+0000 7f3a46190700 1 -- 172.21.15.112:0/3716243178 >> 172.21.15.112:0/3716243178 conn(0x7f3a400fbf40 msgr2=0x7f3a40106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:11:52.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:52.186+0000 7f3a46190700 1 -- 172.21.15.112:0/3716243178 shutdown_connections 2024-03-16T09:11:52.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:11:52.186+0000 7f3a46190700 1 -- 172.21.15.112:0/3716243178 wait complete. 2024-03-16T09:11:52.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:51 smithi159 ceph-mon[36739]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:52.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:51 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:52.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:51 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:52.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:51 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:52.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:51 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:52.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:51 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:52.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:51 smithi194 ceph-mon[36729]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:52.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:51 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:52.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:51 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:52.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:51 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:52.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:51 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:52.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:51 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:52.837 DEBUG:teuthology.orchestra.run.smithi112:osd.9> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.9.service 2024-03-16T09:11:52.840 INFO:tasks.cephadm:Deploying osd.10 on smithi159 with /dev/vg_nvme/lv_1... 2024-03-16T09:11:52.840 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-16T09:11:53.082 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T09:11:53.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:52 smithi112 ceph-mon[32307]: from='osd.9 [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-16T09:11:53.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:52 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:53.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:52 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:53.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:52 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:53.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:52 smithi159 ceph-mon[36739]: from='osd.9 [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-16T09:11:53.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:52 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:53.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:52 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:53.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:52 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:53.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:52 smithi194 ceph-mon[36729]: from='osd.9 [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-16T09:11:53.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:52 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:53.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:52 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:53.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:52 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:11:54.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:53 smithi112 ceph-mon[32307]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:54.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:53 smithi112 ceph-mon[32307]: from='osd.9 [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-16T09:11:54.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:53 smithi112 ceph-mon[32307]: osdmap e65: 10 total, 9 up, 10 in 2024-03-16T09:11:54.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:53 smithi112 ceph-mon[32307]: from='osd.9 [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-03-16T09:11:54.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:53 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T09:11:54.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:11:53 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:11:53.938+0000 7fb118e56700 -1 osd.9 0 waiting for initial osdmap 2024-03-16T09:11:54.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:11:53 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:11:53.956+0000 7fb114c49700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T09:11:54.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:53 smithi159 ceph-mon[36739]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:54.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:53 smithi159 ceph-mon[36739]: from='osd.9 [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-16T09:11:54.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:53 smithi159 ceph-mon[36739]: osdmap e65: 10 total, 9 up, 10 in 2024-03-16T09:11:54.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:53 smithi159 ceph-mon[36739]: from='osd.9 [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-03-16T09:11:54.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:53 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T09:11:54.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:53 smithi194 ceph-mon[36729]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:54.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:53 smithi194 ceph-mon[36729]: from='osd.9 [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-16T09:11:54.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:53 smithi194 ceph-mon[36729]: osdmap e65: 10 total, 9 up, 10 in 2024-03-16T09:11:54.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:53 smithi194 ceph-mon[36729]: from='osd.9 [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-03-16T09:11:54.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:53 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T09:11:55.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:54 smithi159 ceph-mon[36739]: from='osd.9 [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-03-16T09:11:55.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:54 smithi159 ceph-mon[36739]: osdmap e66: 10 total, 9 up, 10 in 2024-03-16T09:11:55.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:54 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T09:11:55.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:54 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T09:11:55.247 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-16T09:11:55.247 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-16T09:11:55.247 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2024-03-16T09:11:55.247 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2024-03-16T09:11:55.247 INFO:teuthology.orchestra.run.smithi159.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114308 s, 917 MB/s 2024-03-16T09:11:55.247 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2024-03-16T09:11:55.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:54 smithi112 ceph-mon[32307]: from='osd.9 [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-03-16T09:11:55.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:54 smithi112 ceph-mon[32307]: osdmap e66: 10 total, 9 up, 10 in 2024-03-16T09:11:55.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:54 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T09:11:55.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:54 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T09:11:55.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:54 smithi194 ceph-mon[36729]: from='osd.9 [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-03-16T09:11:55.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:54 smithi194 ceph-mon[36729]: osdmap e66: 10 total, 9 up, 10 in 2024-03-16T09:11:55.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:54 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T09:11:55.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:54 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T09:11:55.866 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi159:vg_nvme/lv_1 2024-03-16T09:11:56.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:55 smithi112 ceph-mon[32307]: purged_snaps scrub starts 2024-03-16T09:11:56.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:55 smithi112 ceph-mon[32307]: purged_snaps scrub ok 2024-03-16T09:11:56.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:55 smithi112 ceph-mon[32307]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:56.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:55 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T09:11:56.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:55 smithi112 ceph-mon[32307]: osd.9 [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860] boot 2024-03-16T09:11:56.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:55 smithi112 ceph-mon[32307]: osdmap e67: 10 total, 10 up, 10 in 2024-03-16T09:11:56.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:55 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T09:11:56.112 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T09:11:56.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:55 smithi159 ceph-mon[36739]: purged_snaps scrub starts 2024-03-16T09:11:56.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:55 smithi159 ceph-mon[36739]: purged_snaps scrub ok 2024-03-16T09:11:56.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:55 smithi159 ceph-mon[36739]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:56.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:55 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T09:11:56.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:55 smithi159 ceph-mon[36739]: osd.9 [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860] boot 2024-03-16T09:11:56.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:55 smithi159 ceph-mon[36739]: osdmap e67: 10 total, 10 up, 10 in 2024-03-16T09:11:56.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:55 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T09:11:56.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:55 smithi194 ceph-mon[36729]: purged_snaps scrub starts 2024-03-16T09:11:56.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:55 smithi194 ceph-mon[36729]: purged_snaps scrub ok 2024-03-16T09:11:56.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:55 smithi194 ceph-mon[36729]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-16T09:11:56.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:55 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T09:11:56.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:55 smithi194 ceph-mon[36729]: osd.9 [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860] boot 2024-03-16T09:11:56.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:55 smithi194 ceph-mon[36729]: osdmap e67: 10 total, 10 up, 10 in 2024-03-16T09:11:56.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:55 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-16T09:11:57.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:56 smithi159 ceph-mon[36739]: osdmap e68: 10 total, 10 up, 10 in 2024-03-16T09:11:57.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:57.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:57.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:57.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:57.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:57.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:57.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:57.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:57.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:57.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:57.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:56 smithi112 ceph-mon[32307]: osdmap e68: 10 total, 10 up, 10 in 2024-03-16T09:11:57.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:57.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:57.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:57.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:57.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:57.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:57.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:57.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:57.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:57.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:57.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:56 smithi194 ceph-mon[36729]: osdmap e68: 10 total, 10 up, 10 in 2024-03-16T09:11:57.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:56 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:57.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:56 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:57.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:56 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:57.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:56 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:57.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:56 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:57.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:56 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:11:57.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:56 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:57.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:56 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:57.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:56 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:11:57.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:56 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:11:57.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.936+0000 7fb1f87b0700 1 -- 172.21.15.159:0/3515372962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1f0102e30 msgr2=0x7fb1f0105220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:57.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.936+0000 7fb1f87b0700 1 --2- 172.21.15.159:0/3515372962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1f0102e30 0x7fb1f0105220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009990 tx=0x7fb1ec009ca0 comp rx=0 tx=0).stop 2024-03-16T09:11:57.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.936+0000 7fb1f87b0700 1 -- 172.21.15.159:0/3515372962 shutdown_connections 2024-03-16T09:11:57.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.936+0000 7fb1f87b0700 1 --2- 172.21.15.159:0/3515372962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1f0102e30 0x7fb1f0105220 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:57.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.936+0000 7fb1f87b0700 1 --2- 172.21.15.159:0/3515372962 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1f0108090 0x7fb1f010a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:57.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.936+0000 7fb1f87b0700 1 --2- 172.21.15.159:0/3515372962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f0105760 0x7fb1f0107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:57.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.936+0000 7fb1f87b0700 1 -- 172.21.15.159:0/3515372962 >> 172.21.15.159:0/3515372962 conn(0x7fb1f00fc7c0 msgr2=0x7fb1f00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:11:57.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.936+0000 7fb1f87b0700 1 -- 172.21.15.159:0/3515372962 shutdown_connections 2024-03-16T09:11:57.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.936+0000 7fb1f87b0700 1 -- 172.21.15.159:0/3515372962 wait complete. 2024-03-16T09:11:57.941 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.936+0000 7fb1f87b0700 1 Processor -- start 2024-03-16T09:11:57.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.936+0000 7fb1f87b0700 1 -- start start 2024-03-16T09:11:57.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.936+0000 7fb1f87b0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1f0102e30 0x7fb1f0102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:57.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.936+0000 7fb1f87b0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f0105760 0x7fb1f0100de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:57.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.936+0000 7fb1f87b0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1f0108090 0x7fb1f0101320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:57.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.937+0000 7fb1f87b0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f010e110 con 0x7fb1f0105760 2024-03-16T09:11:57.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.937+0000 7fb1f87b0700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f010df90 con 0x7fb1f0102e30 2024-03-16T09:11:57.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.937+0000 7fb1f87b0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f010e290 con 0x7fb1f0108090 2024-03-16T09:11:57.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.937+0000 7fb1f654c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1f0102e30 0x7fb1f0102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:57.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.937+0000 7fb1f654c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1f0102e30 0x7fb1f0102790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:35546/0 (socket says 172.21.15.159:35546) 2024-03-16T09:11:57.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.937+0000 7fb1f654c700 1 -- 172.21.15.159:0/544684466 learned_addr learned my addr 172.21.15.159:0/544684466 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-03-16T09:11:57.942 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.937+0000 7fb1f5d4b700 1 --2- 172.21.15.159:0/544684466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f0105760 0x7fb1f0100de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:57.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.937+0000 7fb1f6d4d700 1 --2- 172.21.15.159:0/544684466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1f0108090 0x7fb1f0101320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:57.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.937+0000 7fb1f654c700 1 -- 172.21.15.159:0/544684466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1f0108090 msgr2=0x7fb1f0101320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:57.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.937+0000 7fb1f654c700 1 --2- 172.21.15.159:0/544684466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1f0108090 0x7fb1f0101320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:57.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.937+0000 7fb1f654c700 1 -- 172.21.15.159:0/544684466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f0105760 msgr2=0x7fb1f0100de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:11:57.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.937+0000 7fb1f654c700 1 --2- 172.21.15.159:0/544684466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f0105760 0x7fb1f0100de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:11:57.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.937+0000 7fb1f654c700 1 -- 172.21.15.159:0/544684466 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f0101bd0 con 0x7fb1f0102e30 2024-03-16T09:11:57.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.937+0000 7fb1f654c700 1 --2- 172.21.15.159:0/544684466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1f0102e30 0x7fb1f0102790 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec005910 tx=0x7fb1ec00f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:11:57.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.937+0000 7fb1e77fe700 1 -- 172.21.15.159:0/544684466 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec027070 con 0x7fb1f0102e30 2024-03-16T09:11:57.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.937+0000 7fb1f87b0700 1 -- 172.21.15.159:0/544684466 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f01a27c0 con 0x7fb1f0102e30 2024-03-16T09:11:57.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.937+0000 7fb1e77fe700 1 -- 172.21.15.159:0/544684466 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb1ec00fce0 con 0x7fb1f0102e30 2024-03-16T09:11:57.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.937+0000 7fb1e77fe700 1 -- 172.21.15.159:0/544684466 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec017750 con 0x7fb1f0102e30 2024-03-16T09:11:57.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.937+0000 7fb1f87b0700 1 -- 172.21.15.159:0/544684466 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f01a2c90 con 0x7fb1f0102e30 2024-03-16T09:11:57.943 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.938+0000 7fb1f87b0700 1 -- 172.21.15.159:0/544684466 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1f0068f90 con 0x7fb1f0102e30 2024-03-16T09:11:57.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.939+0000 7fb1e77fe700 1 -- 172.21.15.159:0/544684466 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fb1ec02a070 con 0x7fb1f0102e30 2024-03-16T09:11:57.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.940+0000 7fb1e77fe700 1 --2- 172.21.15.159:0/544684466 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb1dc0a2890 0x7fb1dc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:11:57.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.940+0000 7fb1e77fe700 1 -- 172.21.15.159:0/544684466 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7fb1ec0cf2b0 con 0x7fb1f0102e30 2024-03-16T09:11:57.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.940+0000 7fb1f5d4b700 1 --2- 172.21.15.159:0/544684466 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb1dc0a2890 0x7fb1dc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:11:57.944 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.941+0000 7fb1f5d4b700 1 --2- 172.21.15.159:0/544684466 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb1dc0a2890 0x7fb1dc0a4d50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0006fd0 tx=0x7fb1e0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:11:57.947 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:57.945+0000 7fb1e77fe700 1 -- 172.21.15.159:0/544684466 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb1ec013070 con 0x7fb1f0102e30 2024-03-16T09:11:58.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:11:58.147+0000 7fb1f87b0700 1 -- 172.21.15.159:0/544684466 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb1f0062710 con 0x7fb1dc0a2890 2024-03-16T09:11:58.149 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:57 smithi159 ceph-mon[36739]: Detected new or changed devices on smithi112 2024-03-16T09:11:58.149 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:57 smithi159 ceph-mon[36739]: Adjusting osd_memory_target on smithi112 to 4272M 2024-03-16T09:11:58.149 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:57 smithi159 ceph-mon[36739]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:11:58.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:57 smithi112 ceph-mon[32307]: Detected new or changed devices on smithi112 2024-03-16T09:11:58.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:57 smithi112 ceph-mon[32307]: Adjusting osd_memory_target on smithi112 to 4272M 2024-03-16T09:11:58.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:57 smithi112 ceph-mon[32307]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:11:58.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:57 smithi194 ceph-mon[36729]: Detected new or changed devices on smithi112 2024-03-16T09:11:58.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:57 smithi194 ceph-mon[36729]: Adjusting osd_memory_target on smithi112 to 4272M 2024-03-16T09:11:58.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:57 smithi194 ceph-mon[36729]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:11:59.088 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:58 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:11:59.089 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:58 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:11:59.089 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:58 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:59.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:58 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:11:59.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:58 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:11:59.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:58 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:11:59.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:58 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:11:59.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:58 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:11:59.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:58 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:00.065 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:59 smithi159 ceph-mon[36739]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:12:00.065 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:11:59 smithi159 ceph-mon[36739]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:59 smithi112 ceph-mon[32307]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:12:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:11:59 smithi112 ceph-mon[32307]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:00.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:59 smithi194 ceph-mon[36729]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:12:00.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:11:59 smithi194 ceph-mon[36729]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:01.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:00 smithi159 ceph-mon[36739]: from='client.? 172.21.15.159:0/2567250131' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1922eaad-cff8-46cc-b837-238133613e28"}]: dispatch 2024-03-16T09:12:01.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:00 smithi159 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1922eaad-cff8-46cc-b837-238133613e28"}]: dispatch 2024-03-16T09:12:01.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:00 smithi159 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1922eaad-cff8-46cc-b837-238133613e28"}]': finished 2024-03-16T09:12:01.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:00 smithi159 ceph-mon[36739]: osdmap e69: 11 total, 10 up, 11 in 2024-03-16T09:12:01.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:00 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T09:12:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:00 smithi112 ceph-mon[32307]: from='client.? 172.21.15.159:0/2567250131' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1922eaad-cff8-46cc-b837-238133613e28"}]: dispatch 2024-03-16T09:12:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:00 smithi112 ceph-mon[32307]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1922eaad-cff8-46cc-b837-238133613e28"}]: dispatch 2024-03-16T09:12:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:00 smithi112 ceph-mon[32307]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1922eaad-cff8-46cc-b837-238133613e28"}]': finished 2024-03-16T09:12:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:00 smithi112 ceph-mon[32307]: osdmap e69: 11 total, 10 up, 11 in 2024-03-16T09:12:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:00 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T09:12:01.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:00 smithi194 ceph-mon[36729]: from='client.? 172.21.15.159:0/2567250131' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1922eaad-cff8-46cc-b837-238133613e28"}]: dispatch 2024-03-16T09:12:01.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:00 smithi194 ceph-mon[36729]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1922eaad-cff8-46cc-b837-238133613e28"}]: dispatch 2024-03-16T09:12:01.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:00 smithi194 ceph-mon[36729]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1922eaad-cff8-46cc-b837-238133613e28"}]': finished 2024-03-16T09:12:01.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:00 smithi194 ceph-mon[36729]: osdmap e69: 11 total, 10 up, 11 in 2024-03-16T09:12:01.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:00 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T09:12:02.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:01 smithi159 ceph-mon[36739]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:02.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:01 smithi159 ceph-mon[36739]: from='client.? 172.21.15.159:0/3765679392' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:12:02.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:01 smithi112 ceph-mon[32307]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:02.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:01 smithi112 ceph-mon[32307]: from='client.? 172.21.15.159:0/3765679392' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:12:02.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:01 smithi194 ceph-mon[36729]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:02.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:01 smithi194 ceph-mon[36729]: from='client.? 172.21.15.159:0/3765679392' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:12:04.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:03 smithi159 ceph-mon[36739]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:04.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:03 smithi112 ceph-mon[32307]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:04.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:03 smithi194 ceph-mon[36729]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:06.224 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:05 smithi159 ceph-mon[36739]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:06.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:05 smithi112 ceph-mon[32307]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:06.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:05 smithi194 ceph-mon[36729]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:08.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:07 smithi159 ceph-mon[36739]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:08.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:07 smithi112 ceph-mon[32307]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:08.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:07 smithi194 ceph-mon[36729]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:09.190 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-16T09:12:09.190 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:09.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-16T09:12:09.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:09.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-16T09:12:09.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:10.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:09 smithi159 ceph-mon[36739]: Deploying daemon osd.10 on smithi159 2024-03-16T09:12:10.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:09 smithi159 ceph-mon[36739]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:10.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:09 smithi112 ceph-mon[32307]: Deploying daemon osd.10 on smithi159 2024-03-16T09:12:10.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:09 smithi112 ceph-mon[32307]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:10.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:09 smithi194 ceph-mon[36729]: Deploying daemon osd.10 on smithi159 2024-03-16T09:12:10.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:09 smithi194 ceph-mon[36729]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:12.123 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:11 smithi159 ceph-mon[36739]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:12.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:11 smithi112 ceph-mon[32307]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:12.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:11 smithi194 ceph-mon[36729]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:14.019 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:13 smithi159 ceph-mon[36739]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:14.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:13 smithi112 ceph-mon[32307]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:14.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:13 smithi194 ceph-mon[36729]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:15.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:15 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:15.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:15 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:15.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:15 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:12:15.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:15 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:15.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:15 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:15.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:15 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:12:15.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:15 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:15.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:15 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:15.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:15 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:12:16.255 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:16 smithi159 ceph-mon[36739]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:16.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:16 smithi112 ceph-mon[32307]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:16.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:16 smithi194 ceph-mon[36729]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:17.915 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:12:17.913+0000 7fb1e77fe700 1 -- 172.21.15.159:0/544684466 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb1f0062710 con 0x7fb1dc0a2890 2024-03-16T09:12:17.915 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 10 on host 'smithi159' 2024-03-16T09:12:17.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:12:17.915+0000 7fb1f87b0700 1 -- 172.21.15.159:0/544684466 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb1dc0a2890 msgr2=0x7fb1dc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:17.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:12:17.915+0000 7fb1f87b0700 1 --2- 172.21.15.159:0/544684466 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb1dc0a2890 0x7fb1dc0a4d50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0006fd0 tx=0x7fb1e0008040 comp rx=0 tx=0).stop 2024-03-16T09:12:17.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:12:17.915+0000 7fb1f87b0700 1 -- 172.21.15.159:0/544684466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1f0102e30 msgr2=0x7fb1f0102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:17.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:12:17.915+0000 7fb1f87b0700 1 --2- 172.21.15.159:0/544684466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1f0102e30 0x7fb1f0102790 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec005910 tx=0x7fb1ec00f740 comp rx=0 tx=0).stop 2024-03-16T09:12:17.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:12:17.915+0000 7fb1f87b0700 1 -- 172.21.15.159:0/544684466 shutdown_connections 2024-03-16T09:12:17.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:12:17.915+0000 7fb1f87b0700 1 --2- 172.21.15.159:0/544684466 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb1dc0a2890 0x7fb1dc0a4d50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:17.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:12:17.915+0000 7fb1f87b0700 1 --2- 172.21.15.159:0/544684466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1f0102e30 0x7fb1f0102790 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:17.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:12:17.915+0000 7fb1f87b0700 1 --2- 172.21.15.159:0/544684466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb1f0108090 0x7fb1f0101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:17.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:12:17.915+0000 7fb1f87b0700 1 --2- 172.21.15.159:0/544684466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f0105760 0x7fb1f0100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:17.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:12:17.915+0000 7fb1f87b0700 1 -- 172.21.15.159:0/544684466 >> 172.21.15.159:0/544684466 conn(0x7fb1f00fc7c0 msgr2=0x7fb1f0103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:17.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:12:17.915+0000 7fb1f87b0700 1 -- 172.21.15.159:0/544684466 shutdown_connections 2024-03-16T09:12:17.918 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-16T09:12:17.915+0000 7fb1f87b0700 1 -- 172.21.15.159:0/544684466 wait complete. 2024-03-16T09:12:18.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:18 smithi159 ceph-mon[36739]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:18.270 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:18 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:18.270 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:18 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:18.270 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:18 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:18.270 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:18 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:12:18.270 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:18 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:18.271 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:18 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:18.271 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:18 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:18.271 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:18 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:12:18.271 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:18 smithi159 ceph-mon[36739]: from='osd.10 [v2:172.21.15.159:6824/1906671120,v1:172.21.15.159:6825/1906671120]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-16T09:12:18.271 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:18 smithi159 ceph-mon[36739]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-16T09:12:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:18 smithi112 ceph-mon[32307]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:18 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:18 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:18 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:18 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:12:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:18 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:18 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:18 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:18 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:12:18.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:18 smithi112 ceph-mon[32307]: from='osd.10 [v2:172.21.15.159:6824/1906671120,v1:172.21.15.159:6825/1906671120]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-16T09:12:18.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:18 smithi112 ceph-mon[32307]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-16T09:12:18.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:18 smithi194 ceph-mon[36729]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:18.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:18 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:18 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:18 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:18 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:12:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:18 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:18 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:18 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:18 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:12:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:18 smithi194 ceph-mon[36729]: from='osd.10 [v2:172.21.15.159:6824/1906671120,v1:172.21.15.159:6825/1906671120]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-16T09:12:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:18 smithi194 ceph-mon[36729]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-16T09:12:18.592 DEBUG:teuthology.orchestra.run.smithi159:osd.10> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.10.service 2024-03-16T09:12:18.594 INFO:tasks.cephadm:Deploying osd.11 on smithi194 with /dev/vg_nvme/lv_1... 2024-03-16T09:12:18.595 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-16T09:12:18.842 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:12:20.048 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:19 smithi159 ceph-mon[36739]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:20.049 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:19 smithi159 ceph-mon[36739]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-16T09:12:20.049 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:19 smithi159 ceph-mon[36739]: osdmap e70: 11 total, 10 up, 11 in 2024-03-16T09:12:20.049 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:19 smithi159 ceph-mon[36739]: from='osd.10 [v2:172.21.15.159:6824/1906671120,v1:172.21.15.159:6825/1906671120]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:12:20.049 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:19 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T09:12:20.049 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:19 smithi159 ceph-mon[36739]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:12:20.049 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:12:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:12:19.918+0000 7fe44402a700 -1 osd.10 0 waiting for initial osdmap 2024-03-16T09:12:20.049 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:12:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:12:19.934+0000 7fe43fe1d700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T09:12:20.213 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:19 smithi194 ceph-mon[36729]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:20.213 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:19 smithi194 ceph-mon[36729]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-16T09:12:20.213 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:19 smithi194 ceph-mon[36729]: osdmap e70: 11 total, 10 up, 11 in 2024-03-16T09:12:20.213 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:19 smithi194 ceph-mon[36729]: from='osd.10 [v2:172.21.15.159:6824/1906671120,v1:172.21.15.159:6825/1906671120]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:12:20.213 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:19 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T09:12:20.213 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:19 smithi194 ceph-mon[36729]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:12:20.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:19 smithi112 ceph-mon[32307]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:20.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:19 smithi112 ceph-mon[32307]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-16T09:12:20.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:19 smithi112 ceph-mon[32307]: osdmap e70: 11 total, 10 up, 11 in 2024-03-16T09:12:20.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:19 smithi112 ceph-mon[32307]: from='osd.10 [v2:172.21.15.159:6824/1906671120,v1:172.21.15.159:6825/1906671120]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:12:20.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:19 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T09:12:20.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:19 smithi112 ceph-mon[32307]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-16T09:12:20.925 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-16T09:12:20.925 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-16T09:12:20.925 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2024-03-16T09:12:20.925 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2024-03-16T09:12:20.925 INFO:teuthology.orchestra.run.smithi194.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115524 s, 908 MB/s 2024-03-16T09:12:20.926 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 2024-03-16T09:12:20.926 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2024-03-16T09:12:21.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:20 smithi159 ceph-mon[36739]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-16T09:12:21.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:20 smithi159 ceph-mon[36739]: osdmap e71: 11 total, 10 up, 11 in 2024-03-16T09:12:21.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:20 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T09:12:21.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:20 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T09:12:21.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:20 smithi159 ceph-mon[36739]: osd.10 [v2:172.21.15.159:6824/1906671120,v1:172.21.15.159:6825/1906671120] boot 2024-03-16T09:12:21.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:20 smithi159 ceph-mon[36739]: osdmap e72: 11 total, 11 up, 11 in 2024-03-16T09:12:21.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:20 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T09:12:21.254 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:20 smithi194 ceph-mon[36729]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-16T09:12:21.254 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:20 smithi194 ceph-mon[36729]: osdmap e71: 11 total, 10 up, 11 in 2024-03-16T09:12:21.254 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T09:12:21.254 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T09:12:21.254 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:20 smithi194 ceph-mon[36729]: osd.10 [v2:172.21.15.159:6824/1906671120,v1:172.21.15.159:6825/1906671120] boot 2024-03-16T09:12:21.254 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:20 smithi194 ceph-mon[36729]: osdmap e72: 11 total, 11 up, 11 in 2024-03-16T09:12:21.255 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T09:12:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:20 smithi112 ceph-mon[32307]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-16T09:12:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:20 smithi112 ceph-mon[32307]: osdmap e71: 11 total, 10 up, 11 in 2024-03-16T09:12:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T09:12:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T09:12:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:20 smithi112 ceph-mon[32307]: osd.10 [v2:172.21.15.159:6824/1906671120,v1:172.21.15.159:6825/1906671120] boot 2024-03-16T09:12:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:20 smithi112 ceph-mon[32307]: osdmap e72: 11 total, 11 up, 11 in 2024-03-16T09:12:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-16T09:12:21.673 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi194:vg_nvme/lv_1 2024-03-16T09:12:21.931 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:12:21.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:21 smithi159 ceph-mon[36739]: purged_snaps scrub starts 2024-03-16T09:12:21.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:21 smithi159 ceph-mon[36739]: purged_snaps scrub ok 2024-03-16T09:12:21.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:21 smithi159 ceph-mon[36739]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:21.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:21 smithi159 ceph-mon[36739]: osdmap e73: 11 total, 11 up, 11 in 2024-03-16T09:12:21.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:21 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:21.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:21 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:21.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:21 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:21.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:21 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:21.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:21 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:21.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:21 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:21.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:21 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:21.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:21 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:21.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:21 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:12:21.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:21 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:22.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:21 smithi194 ceph-mon[36729]: purged_snaps scrub starts 2024-03-16T09:12:22.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:21 smithi194 ceph-mon[36729]: purged_snaps scrub ok 2024-03-16T09:12:22.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:21 smithi194 ceph-mon[36729]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:22.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:21 smithi194 ceph-mon[36729]: osdmap e73: 11 total, 11 up, 11 in 2024-03-16T09:12:22.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:21 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:22.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:21 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:22.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:21 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:22.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:21 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:22.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:21 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:22.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:21 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:22.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:21 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:22.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:21 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:22.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:21 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:12:22.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:21 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:22.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:21 smithi112 ceph-mon[32307]: purged_snaps scrub starts 2024-03-16T09:12:22.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:21 smithi112 ceph-mon[32307]: purged_snaps scrub ok 2024-03-16T09:12:22.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:21 smithi112 ceph-mon[32307]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-03-16T09:12:22.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:21 smithi112 ceph-mon[32307]: osdmap e73: 11 total, 11 up, 11 in 2024-03-16T09:12:22.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:21 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:22.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:21 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:22.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:21 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:22.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:21 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:22.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:21 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:22.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:21 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:22.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:21 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:22.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:21 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:22.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:21 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:12:22.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:21 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:23.080 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:22 smithi194 ceph-mon[36729]: Detected new or changed devices on smithi159 2024-03-16T09:12:23.080 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:22 smithi194 ceph-mon[36729]: Adjusting osd_memory_target on smithi159 to 4270M 2024-03-16T09:12:23.080 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:22 smithi194 ceph-mon[36729]: osdmap e74: 11 total, 11 up, 11 in 2024-03-16T09:12:23.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:22 smithi159 ceph-mon[36739]: Detected new or changed devices on smithi159 2024-03-16T09:12:23.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:22 smithi159 ceph-mon[36739]: Adjusting osd_memory_target on smithi159 to 4270M 2024-03-16T09:12:23.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:22 smithi159 ceph-mon[36739]: osdmap e74: 11 total, 11 up, 11 in 2024-03-16T09:12:23.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:22 smithi112 ceph-mon[32307]: Detected new or changed devices on smithi159 2024-03-16T09:12:23.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:22 smithi112 ceph-mon[32307]: Adjusting osd_memory_target on smithi159 to 4270M 2024-03-16T09:12:23.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:22 smithi112 ceph-mon[32307]: osdmap e74: 11 total, 11 up, 11 in 2024-03-16T09:12:23.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.594+0000 7fba7bfd6700 1 -- 172.21.15.194:0/2529468756 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba74102300 msgr2=0x7fba74102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:23.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.594+0000 7fba7bfd6700 1 --2- 172.21.15.194:0/2529468756 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba74102300 0x7fba74102720 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fba70009990 tx=0x7fba70009ca0 comp rx=0 tx=0).stop 2024-03-16T09:12:23.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.594+0000 7fba7bfd6700 1 -- 172.21.15.194:0/2529468756 shutdown_connections 2024-03-16T09:12:23.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.594+0000 7fba7bfd6700 1 --2- 172.21.15.194:0/2529468756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba74103ea0 0x7fba7410a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:23.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.594+0000 7fba7bfd6700 1 --2- 172.21.15.194:0/2529468756 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba74102300 0x7fba74102720 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:23.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.594+0000 7fba7bfd6700 1 --2- 172.21.15.194:0/2529468756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba74103500 0x7fba74103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:23.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.594+0000 7fba7bfd6700 1 -- 172.21.15.194:0/2529468756 >> 172.21.15.194:0/2529468756 conn(0x7fba740fd8e0 msgr2=0x7fba740ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:23.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.594+0000 7fba7bfd6700 1 -- 172.21.15.194:0/2529468756 shutdown_connections 2024-03-16T09:12:23.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.594+0000 7fba7bfd6700 1 -- 172.21.15.194:0/2529468756 wait complete. 2024-03-16T09:12:23.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.595+0000 7fba7bfd6700 1 Processor -- start 2024-03-16T09:12:23.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.595+0000 7fba7bfd6700 1 -- start start 2024-03-16T09:12:23.599 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.595+0000 7fba7bfd6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba74102300 0x7fba74076020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:23.600 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.595+0000 7fba7bfd6700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba74103500 0x7fba740746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:23.600 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.595+0000 7fba7bfd6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba74103ea0 0x7fba74074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:23.600 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.595+0000 7fba7bfd6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fba7410e080 con 0x7fba74102300 2024-03-16T09:12:23.600 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.595+0000 7fba7bfd6700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fba7410df00 con 0x7fba74103500 2024-03-16T09:12:23.600 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.595+0000 7fba7bfd6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fba7410e200 con 0x7fba74103ea0 2024-03-16T09:12:23.600 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.595+0000 7fba7a573700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba74103ea0 0x7fba74074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:23.600 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.595+0000 7fba7a573700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba74103ea0 0x7fba74074c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:46510/0 (socket says 172.21.15.194:46510) 2024-03-16T09:12:23.600 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.595+0000 7fba7a573700 1 -- 172.21.15.194:0/54911274 learned_addr learned my addr 172.21.15.194:0/54911274 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T09:12:23.600 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.595+0000 7fba79d72700 1 --2- 172.21.15.194:0/54911274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba74102300 0x7fba74076020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:23.600 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.595+0000 7fba79571700 1 --2- 172.21.15.194:0/54911274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba74103500 0x7fba740746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:23.600 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.595+0000 7fba7a573700 1 -- 172.21.15.194:0/54911274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba74103500 msgr2=0x7fba740746c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:23.600 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.595+0000 7fba7a573700 1 --2- 172.21.15.194:0/54911274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba74103500 0x7fba740746c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:23.600 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.595+0000 7fba7a573700 1 -- 172.21.15.194:0/54911274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba74102300 msgr2=0x7fba74076020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:23.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.595+0000 7fba7a573700 1 --2- 172.21.15.194:0/54911274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba74102300 0x7fba74076020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:23.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.595+0000 7fba7a573700 1 -- 172.21.15.194:0/54911274 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba740754d0 con 0x7fba74103ea0 2024-03-16T09:12:23.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.595+0000 7fba79d72700 1 --2- 172.21.15.194:0/54911274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba74102300 0x7fba74076020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:12:23.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.596+0000 7fba7a573700 1 --2- 172.21.15.194:0/54911274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba74103ea0 0x7fba74074c00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00ada0 tx=0x7fba6c00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:23.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.596+0000 7fba6affd700 1 -- 172.21.15.194:0/54911274 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c017020 con 0x7fba74103ea0 2024-03-16T09:12:23.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.596+0000 7fba6affd700 1 -- 172.21.15.194:0/54911274 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fba6c004bb0 con 0x7fba74103ea0 2024-03-16T09:12:23.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.596+0000 7fba6affd700 1 -- 172.21.15.194:0/54911274 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c005630 con 0x7fba74103ea0 2024-03-16T09:12:23.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.596+0000 7fba7bfd6700 1 -- 172.21.15.194:0/54911274 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba741a27f0 con 0x7fba74103ea0 2024-03-16T09:12:23.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.596+0000 7fba7bfd6700 1 -- 172.21.15.194:0/54911274 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba741a2d00 con 0x7fba74103ea0 2024-03-16T09:12:23.601 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.597+0000 7fba7bfd6700 1 -- 172.21.15.194:0/54911274 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba74068f90 con 0x7fba74103ea0 2024-03-16T09:12:23.602 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.598+0000 7fba6affd700 1 -- 172.21.15.194:0/54911274 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fba6c004d60 con 0x7fba74103ea0 2024-03-16T09:12:23.602 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.598+0000 7fba6affd700 1 --2- 172.21.15.194:0/54911274 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fba600a2910 0x7fba600a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:23.602 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.599+0000 7fba6affd700 1 -- 172.21.15.194:0/54911274 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6694+0+0 (secure 0 0 0) 0x7fba6c0ce120 con 0x7fba74103ea0 2024-03-16T09:12:23.602 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.599+0000 7fba79d72700 1 --2- 172.21.15.194:0/54911274 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fba600a2910 0x7fba600a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:23.602 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.599+0000 7fba79d72700 1 --2- 172.21.15.194:0/54911274 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fba600a2910 0x7fba600a4dd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fba70003ad0 tx=0x7fba70005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:23.604 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.602+0000 7fba6affd700 1 -- 172.21.15.194:0/54911274 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fba6c021020 con 0x7fba74103ea0 2024-03-16T09:12:23.811 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:23.809+0000 7fba7bfd6700 1 -- 172.21.15.194:0/54911274 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fba74062710 con 0x7fba600a2910 2024-03-16T09:12:24.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:23 smithi194 ceph-mon[36729]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:24.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:23 smithi194 ceph-mon[36729]: osdmap e75: 11 total, 11 up, 11 in 2024-03-16T09:12:24.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:23 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:12:24.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:23 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:12:24.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:23 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:24.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:23 smithi159 ceph-mon[36739]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:24.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:23 smithi159 ceph-mon[36739]: osdmap e75: 11 total, 11 up, 11 in 2024-03-16T09:12:24.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:23 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:12:24.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:23 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:12:24.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:23 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:24.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:23 smithi112 ceph-mon[32307]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:24.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:23 smithi112 ceph-mon[32307]: osdmap e75: 11 total, 11 up, 11 in 2024-03-16T09:12:24.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:23 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-16T09:12:24.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:23 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-16T09:12:24.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:23 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:25.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:24 smithi159 ceph-mon[36739]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:12:25.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:24 smithi159 ceph-mon[36739]: osdmap e76: 11 total, 11 up, 11 in 2024-03-16T09:12:25.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:24 smithi112 ceph-mon[32307]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:12:25.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:24 smithi112 ceph-mon[32307]: osdmap e76: 11 total, 11 up, 11 in 2024-03-16T09:12:25.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:24 smithi194 ceph-mon[36729]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:12:25.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:24 smithi194 ceph-mon[36729]: osdmap e76: 11 total, 11 up, 11 in 2024-03-16T09:12:26.122 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:25 smithi194 ceph-mon[36729]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-16T09:12:26.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:25 smithi159 ceph-mon[36739]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-16T09:12:26.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:25 smithi112 ceph-mon[32307]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-16T09:12:27.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:26 smithi159 ceph-mon[36739]: from='client.? 172.21.15.194:0/2557920631' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb0e58e0-7d51-4a00-9004-03607251189f"}]: dispatch 2024-03-16T09:12:27.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:26 smithi159 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb0e58e0-7d51-4a00-9004-03607251189f"}]: dispatch 2024-03-16T09:12:27.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:26 smithi159 ceph-mon[36739]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb0e58e0-7d51-4a00-9004-03607251189f"}]': finished 2024-03-16T09:12:27.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:26 smithi159 ceph-mon[36739]: osdmap e77: 12 total, 11 up, 12 in 2024-03-16T09:12:27.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:26 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T09:12:27.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:26 smithi112 ceph-mon[32307]: from='client.? 172.21.15.194:0/2557920631' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb0e58e0-7d51-4a00-9004-03607251189f"}]: dispatch 2024-03-16T09:12:27.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:26 smithi112 ceph-mon[32307]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb0e58e0-7d51-4a00-9004-03607251189f"}]: dispatch 2024-03-16T09:12:27.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:26 smithi112 ceph-mon[32307]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb0e58e0-7d51-4a00-9004-03607251189f"}]': finished 2024-03-16T09:12:27.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:26 smithi112 ceph-mon[32307]: osdmap e77: 12 total, 11 up, 12 in 2024-03-16T09:12:27.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:26 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T09:12:27.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:26 smithi194 ceph-mon[36729]: from='client.? 172.21.15.194:0/2557920631' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb0e58e0-7d51-4a00-9004-03607251189f"}]: dispatch 2024-03-16T09:12:27.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:26 smithi194 ceph-mon[36729]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb0e58e0-7d51-4a00-9004-03607251189f"}]: dispatch 2024-03-16T09:12:27.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:26 smithi194 ceph-mon[36729]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb0e58e0-7d51-4a00-9004-03607251189f"}]': finished 2024-03-16T09:12:27.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:26 smithi194 ceph-mon[36729]: osdmap e77: 12 total, 11 up, 12 in 2024-03-16T09:12:27.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:26 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T09:12:28.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:27 smithi159 ceph-mon[36739]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-03-16T09:12:28.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:27 smithi159 ceph-mon[36739]: from='client.? 172.21.15.194:0/3613245024' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:12:28.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:27 smithi112 ceph-mon[32307]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-03-16T09:12:28.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:27 smithi112 ceph-mon[32307]: from='client.? 172.21.15.194:0/3613245024' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:12:28.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:27 smithi194 ceph-mon[36729]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-03-16T09:12:28.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:27 smithi194 ceph-mon[36729]: from='client.? 172.21.15.194:0/3613245024' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-16T09:12:30.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:29 smithi159 ceph-mon[36739]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T09:12:30.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:29 smithi112 ceph-mon[32307]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T09:12:30.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:29 smithi194 ceph-mon[36729]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-16T09:12:32.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:31 smithi194 ceph-mon[36729]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-16T09:12:32.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:31 smithi159 ceph-mon[36739]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-16T09:12:32.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:31 smithi112 ceph-mon[32307]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-16T09:12:34.218 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:33 smithi194 ceph-mon[36729]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-16T09:12:34.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:33 smithi159 ceph-mon[36739]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-16T09:12:34.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:33 smithi112 ceph-mon[32307]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-16T09:12:35.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:34 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-16T09:12:35.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:34 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:35.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:34 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-16T09:12:35.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:34 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:35.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:34 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-16T09:12:35.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:34 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:36.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:35 smithi194 ceph-mon[36729]: Deploying daemon osd.11 on smithi194 2024-03-16T09:12:36.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:35 smithi194 ceph-mon[36729]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:36.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:35 smithi159 ceph-mon[36739]: Deploying daemon osd.11 on smithi194 2024-03-16T09:12:36.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:35 smithi159 ceph-mon[36739]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:35 smithi112 ceph-mon[32307]: Deploying daemon osd.11 on smithi194 2024-03-16T09:12:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:35 smithi112 ceph-mon[32307]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:38.118 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:37 smithi194 ceph-mon[36729]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:38.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:37 smithi159 ceph-mon[36739]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:38.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:37 smithi112 ceph-mon[32307]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:40.127 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:39 smithi194 ceph-mon[36729]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:40.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:39 smithi159 ceph-mon[36739]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:40.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:39 smithi112 ceph-mon[32307]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:41.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:41 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:41.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:41 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:41.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:41 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:12:41.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:41 smithi112 ceph-mon[32307]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:41.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:41 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:41.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:41 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:41.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:41 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:12:41.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:41 smithi194 ceph-mon[36729]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:41.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:41 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:41.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:41 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:41.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:41 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:12:41.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:41 smithi159 ceph-mon[36739]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:44.049 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:43 smithi194 ceph-mon[36729]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:44.049 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:43 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:44.049 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:43 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:44.049 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:43 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:44.049 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:43 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:12:44.049 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:43 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:44.130 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 11 on host 'smithi194' 2024-03-16T09:12:44.130 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:44.127+0000 7fba6affd700 1 -- 172.21.15.194:0/54911274 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fba74062710 con 0x7fba600a2910 2024-03-16T09:12:44.132 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:44.130+0000 7fba7bfd6700 1 -- 172.21.15.194:0/54911274 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fba600a2910 msgr2=0x7fba600a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:44.133 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:44.130+0000 7fba7bfd6700 1 --2- 172.21.15.194:0/54911274 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fba600a2910 0x7fba600a4dd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fba70003ad0 tx=0x7fba70005fb0 comp rx=0 tx=0).stop 2024-03-16T09:12:44.133 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:44.130+0000 7fba7bfd6700 1 -- 172.21.15.194:0/54911274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba74103ea0 msgr2=0x7fba74074c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:44.133 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:44.130+0000 7fba7bfd6700 1 --2- 172.21.15.194:0/54911274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba74103ea0 0x7fba74074c00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00ada0 tx=0x7fba6c00c330 comp rx=0 tx=0).stop 2024-03-16T09:12:44.133 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:44.130+0000 7fba7bfd6700 1 -- 172.21.15.194:0/54911274 shutdown_connections 2024-03-16T09:12:44.133 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:44.130+0000 7fba7bfd6700 1 --2- 172.21.15.194:0/54911274 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fba600a2910 0x7fba600a4dd0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:44.133 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:44.130+0000 7fba7bfd6700 1 --2- 172.21.15.194:0/54911274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fba74103ea0 0x7fba74074c00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:44.133 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:44.130+0000 7fba7bfd6700 1 --2- 172.21.15.194:0/54911274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba74102300 0x7fba74076020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:44.133 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:44.130+0000 7fba7bfd6700 1 --2- 172.21.15.194:0/54911274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba74103500 0x7fba740746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:44.134 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:44.130+0000 7fba7bfd6700 1 -- 172.21.15.194:0/54911274 >> 172.21.15.194:0/54911274 conn(0x7fba740fd8e0 msgr2=0x7fba74106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:44.134 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:44.130+0000 7fba7bfd6700 1 -- 172.21.15.194:0/54911274 shutdown_connections 2024-03-16T09:12:44.134 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:44.130+0000 7fba7bfd6700 1 -- 172.21.15.194:0/54911274 wait complete. 2024-03-16T09:12:44.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:43 smithi159 ceph-mon[36739]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:44.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:43 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:44.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:43 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:44.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:43 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:44.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:43 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:12:44.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:43 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:43 smithi112 ceph-mon[32307]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:44.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:44.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:12:44.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:43 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:44.821 DEBUG:teuthology.orchestra.run.smithi194:osd.11> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.11.service 2024-03-16T09:12:44.824 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-16T09:12:44.824 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-16T09:12:45.076 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:12:45.199 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:45 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:45.199 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:45 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:45.199 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:45 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:12:45.199 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:45 smithi112 ceph-mon[32307]: from='osd.11 [v2:172.21.15.194:6824/3230458513,v1:172.21.15.194:6825/3230458513]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-16T09:12:45.199 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:45 smithi112 ceph-mon[32307]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-16T09:12:45.199 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:45 smithi112 ceph-mon[32307]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:45.315 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:45 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:45.315 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:45 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:45.315 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:45 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:12:45.315 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:45 smithi194 ceph-mon[36729]: from='osd.11 [v2:172.21.15.194:6824/3230458513,v1:172.21.15.194:6825/3230458513]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-16T09:12:45.316 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:45 smithi194 ceph-mon[36729]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-16T09:12:45.316 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:45 smithi194 ceph-mon[36729]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:45.442 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:45 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:45.442 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:45 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:45.442 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:45 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:12:45.442 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:45 smithi159 ceph-mon[36739]: from='osd.11 [v2:172.21.15.194:6824/3230458513,v1:172.21.15.194:6825/3230458513]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-16T09:12:45.442 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:45 smithi159 ceph-mon[36739]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-16T09:12:45.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:45 smithi159 ceph-mon[36739]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:46.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:46 smithi112 ceph-mon[32307]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-16T09:12:46.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:46 smithi112 ceph-mon[32307]: from='osd.11 [v2:172.21.15.194:6824/3230458513,v1:172.21.15.194:6825/3230458513]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:12:46.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:46 smithi112 ceph-mon[32307]: osdmap e78: 12 total, 11 up, 12 in 2024-03-16T09:12:46.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:46 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T09:12:46.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:46 smithi112 ceph-mon[32307]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:12:46.379 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:46 smithi194 ceph-mon[36729]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-16T09:12:46.379 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:46 smithi194 ceph-mon[36729]: from='osd.11 [v2:172.21.15.194:6824/3230458513,v1:172.21.15.194:6825/3230458513]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:12:46.379 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:46 smithi194 ceph-mon[36729]: osdmap e78: 12 total, 11 up, 12 in 2024-03-16T09:12:46.379 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:46 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T09:12:46.380 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:46 smithi194 ceph-mon[36729]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:12:46.380 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:12:46 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:12:46.101+0000 7f2892b88700 -1 osd.11 0 waiting for initial osdmap 2024-03-16T09:12:46.380 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:12:46 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:12:46.117+0000 7f288e97b700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T09:12:46.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:46 smithi159 ceph-mon[36739]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-16T09:12:46.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:46 smithi159 ceph-mon[36739]: from='osd.11 [v2:172.21.15.194:6824/3230458513,v1:172.21.15.194:6825/3230458513]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:12:46.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:46 smithi159 ceph-mon[36739]: osdmap e78: 12 total, 11 up, 12 in 2024-03-16T09:12:46.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:46 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T09:12:46.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:46 smithi159 ceph-mon[36739]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-03-16T09:12:47.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.040+0000 7fa8d0f89700 1 -- 172.21.15.112:0/2588003113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8cc102450 msgr2=0x7fa8cc1028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:47.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.040+0000 7fa8d0f89700 1 --2- 172.21.15.112:0/2588003113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8cc102450 0x7fa8cc1028b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40099c0 tx=0x7fa8b4009cd0 comp rx=0 tx=0).stop 2024-03-16T09:12:47.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.040+0000 7fa8d0f89700 1 -- 172.21.15.112:0/2588003113 shutdown_connections 2024-03-16T09:12:47.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.040+0000 7fa8d0f89700 1 --2- 172.21.15.112:0/2588003113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8cc102df0 0x7fa8cc1096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:47.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.040+0000 7fa8d0f89700 1 --2- 172.21.15.112:0/2588003113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8cc101250 0x7fa8cc101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:47.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.040+0000 7fa8d0f89700 1 --2- 172.21.15.112:0/2588003113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8cc102450 0x7fa8cc1028b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:47.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.040+0000 7fa8d0f89700 1 -- 172.21.15.112:0/2588003113 >> 172.21.15.112:0/2588003113 conn(0x7fa8cc0fc7d0 msgr2=0x7fa8cc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:47.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.040+0000 7fa8d0f89700 1 -- 172.21.15.112:0/2588003113 shutdown_connections 2024-03-16T09:12:47.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.040+0000 7fa8d0f89700 1 -- 172.21.15.112:0/2588003113 wait complete. 2024-03-16T09:12:47.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.041+0000 7fa8d0f89700 1 Processor -- start 2024-03-16T09:12:47.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.041+0000 7fa8d0f89700 1 -- start start 2024-03-16T09:12:47.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.041+0000 7fa8d0f89700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8cc101250 0x7fa8cc06b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:47.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.041+0000 7fa8d0f89700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8cc102450 0x7fa8cc19d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:47.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.041+0000 7fa8d0f89700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8cc102df0 0x7fa8cc19f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:47.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.041+0000 7fa8d0f89700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc10cff0 con 0x7fa8cc102df0 2024-03-16T09:12:47.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.041+0000 7fa8d0f89700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc10ce70 con 0x7fa8cc101250 2024-03-16T09:12:47.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.041+0000 7fa8d0f89700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc10d170 con 0x7fa8cc102450 2024-03-16T09:12:47.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.041+0000 7fa8c1d9b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8cc102450 0x7fa8cc19d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:47.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.041+0000 7fa8c1d9b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8cc102450 0x7fa8cc19d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:40002/0 (socket says 172.21.15.112:40002) 2024-03-16T09:12:47.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.041+0000 7fa8cad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8cc102df0 0x7fa8cc19f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:47.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.041+0000 7fa8c1d9b700 1 -- 172.21.15.112:0/2200745343 learned_addr learned my addr 172.21.15.112:0/2200745343 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:12:47.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.041+0000 7fa8ca59c700 1 --2- 172.21.15.112:0/2200745343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8cc101250 0x7fa8cc06b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:47.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.042+0000 7fa8c1d9b700 1 -- 172.21.15.112:0/2200745343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8cc101250 msgr2=0x7fa8cc06b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:47.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.042+0000 7fa8c1d9b700 1 --2- 172.21.15.112:0/2200745343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8cc101250 0x7fa8cc06b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:47.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.042+0000 7fa8c1d9b700 1 -- 172.21.15.112:0/2200745343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8cc102df0 msgr2=0x7fa8cc19f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:47.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.042+0000 7fa8c1d9b700 1 --2- 172.21.15.112:0/2200745343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8cc102df0 0x7fa8cc19f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:47.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.042+0000 7fa8c1d9b700 1 -- 172.21.15.112:0/2200745343 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8cc1a0240 con 0x7fa8cc102450 2024-03-16T09:12:47.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.042+0000 7fa8c1d9b700 1 --2- 172.21.15.112:0/2200745343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8cc102450 0x7fa8cc19d2e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4005910 tx=0x7fa8b4005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:47.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.042+0000 7fa8c3fff700 1 -- 172.21.15.112:0/2200745343 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4027070 con 0x7fa8cc102450 2024-03-16T09:12:47.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.042+0000 7fa8c3fff700 1 -- 172.21.15.112:0/2200745343 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa8b4004140 con 0x7fa8cc102450 2024-03-16T09:12:47.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.042+0000 7fa8d0f89700 1 -- 172.21.15.112:0/2200745343 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8cc1a04d0 con 0x7fa8cc102450 2024-03-16T09:12:47.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.042+0000 7fa8c3fff700 1 -- 172.21.15.112:0/2200745343 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4022e40 con 0x7fa8cc102450 2024-03-16T09:12:47.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.042+0000 7fa8d0f89700 1 -- 172.21.15.112:0/2200745343 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8cc1a09b0 con 0x7fa8cc102450 2024-03-16T09:12:47.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.043+0000 7fa8d0f89700 1 -- 172.21.15.112:0/2200745343 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8cc068f90 con 0x7fa8cc102450 2024-03-16T09:12:47.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.044+0000 7fa8c3fff700 1 -- 172.21.15.112:0/2200745343 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fa8b402a070 con 0x7fa8cc102450 2024-03-16T09:12:47.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.045+0000 7fa8c3fff700 1 --2- 172.21.15.112:0/2200745343 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa8b80a2890 0x7fa8b80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:47.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.045+0000 7fa8ca59c700 1 --2- 172.21.15.112:0/2200745343 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa8b80a2890 0x7fa8b80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:47.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.045+0000 7fa8c3fff700 1 -- 172.21.15.112:0/2200745343 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6847+0+0 (secure 0 0 0) 0x7fa8b40cf1f0 con 0x7fa8cc102450 2024-03-16T09:12:47.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.045+0000 7fa8ca59c700 1 --2- 172.21.15.112:0/2200745343 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa8b80a2890 0x7fa8b80a4d50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc1022b0 tx=0x7fa8bc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:47.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.048+0000 7fa8c3fff700 1 -- 172.21.15.112:0/2200745343 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa8b4013070 con 0x7fa8cc102450 2024-03-16T09:12:47.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.250+0000 7fa8d0f89700 1 -- 172.21.15.112:0/2200745343 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fa8cc062ee0 con 0x7fa8cc102450 2024-03-16T09:12:47.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.251+0000 7fa8c3fff700 1 -- 172.21.15.112:0/2200745343 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7fa8b40a2070 con 0x7fa8cc102450 2024-03-16T09:12:47.254 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:12:47.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.254+0000 7fa8d0f89700 1 -- 172.21.15.112:0/2200745343 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa8b80a2890 msgr2=0x7fa8b80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:47.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.254+0000 7fa8d0f89700 1 --2- 172.21.15.112:0/2200745343 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa8b80a2890 0x7fa8b80a4d50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc1022b0 tx=0x7fa8bc00b410 comp rx=0 tx=0).stop 2024-03-16T09:12:47.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.254+0000 7fa8d0f89700 1 -- 172.21.15.112:0/2200745343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8cc102450 msgr2=0x7fa8cc19d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:47.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.254+0000 7fa8d0f89700 1 --2- 172.21.15.112:0/2200745343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8cc102450 0x7fa8cc19d2e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4005910 tx=0x7fa8b4005e80 comp rx=0 tx=0).stop 2024-03-16T09:12:47.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.254+0000 7fa8d0f89700 1 -- 172.21.15.112:0/2200745343 shutdown_connections 2024-03-16T09:12:47.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.254+0000 7fa8d0f89700 1 --2- 172.21.15.112:0/2200745343 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa8b80a2890 0x7fa8b80a4d50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:47.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.254+0000 7fa8d0f89700 1 --2- 172.21.15.112:0/2200745343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8cc101250 0x7fa8cc06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:47.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.254+0000 7fa8d0f89700 1 --2- 172.21.15.112:0/2200745343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8cc102df0 0x7fa8cc19f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:47.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.254+0000 7fa8d0f89700 1 --2- 172.21.15.112:0/2200745343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8cc102450 0x7fa8cc19d2e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:47.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.254+0000 7fa8d0f89700 1 -- 172.21.15.112:0/2200745343 >> 172.21.15.112:0/2200745343 conn(0x7fa8cc0fc7d0 msgr2=0x7fa8cc105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:47.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.254+0000 7fa8d0f89700 1 -- 172.21.15.112:0/2200745343 shutdown_connections 2024-03-16T09:12:47.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:47.254+0000 7fa8d0f89700 1 -- 172.21.15.112:0/2200745343 wait complete. 2024-03-16T09:12:47.314 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:47 smithi112 ceph-mon[32307]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T09:12:47.314 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:47 smithi112 ceph-mon[32307]: osdmap e79: 12 total, 11 up, 12 in 2024-03-16T09:12:47.314 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T09:12:47.314 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:47 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T09:12:47.314 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:47 smithi112 ceph-mon[32307]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:47.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:47 smithi194 ceph-mon[36729]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T09:12:47.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:47 smithi194 ceph-mon[36729]: osdmap e79: 12 total, 11 up, 12 in 2024-03-16T09:12:47.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:47 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T09:12:47.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:47 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T09:12:47.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:47 smithi194 ceph-mon[36729]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:47.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:47 smithi159 ceph-mon[36739]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-03-16T09:12:47.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:47 smithi159 ceph-mon[36739]: osdmap e79: 12 total, 11 up, 12 in 2024-03-16T09:12:47.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:47 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T09:12:47.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:47 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T09:12:47.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:47 smithi159 ceph-mon[36739]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-03-16T09:12:48.096 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1710580367,"num_in_osds":12,"osd_in_since":1710580346,"num_remapped_pgs":0} 2024-03-16T09:12:48.096 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-16T09:12:48.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:48 smithi194 ceph-mon[36729]: purged_snaps scrub starts 2024-03-16T09:12:48.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:48 smithi194 ceph-mon[36729]: purged_snaps scrub ok 2024-03-16T09:12:48.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:48 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T09:12:48.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:48 smithi194 ceph-mon[36729]: osd.11 [v2:172.21.15.194:6824/3230458513,v1:172.21.15.194:6825/3230458513] boot 2024-03-16T09:12:48.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:48 smithi194 ceph-mon[36729]: osdmap e80: 12 total, 12 up, 12 in 2024-03-16T09:12:48.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:48 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T09:12:48.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:48 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2200745343' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-16T09:12:48.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:48 smithi194 ceph-mon[36729]: Detected new or changed devices on smithi194 2024-03-16T09:12:48.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:48 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:48.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:48 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:48.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:48 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:48.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:48 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:48.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:48 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:48.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:48 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:48.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:48 smithi194 ceph-mon[36729]: Adjusting osd_memory_target on smithi194 to 4270M 2024-03-16T09:12:48.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:48 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:48.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:48 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:48.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:48 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:12:48.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:48 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:48.341 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:12:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:48 smithi112 ceph-mon[32307]: purged_snaps scrub starts 2024-03-16T09:12:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:48 smithi112 ceph-mon[32307]: purged_snaps scrub ok 2024-03-16T09:12:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:48 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T09:12:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:48 smithi112 ceph-mon[32307]: osd.11 [v2:172.21.15.194:6824/3230458513,v1:172.21.15.194:6825/3230458513] boot 2024-03-16T09:12:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:48 smithi112 ceph-mon[32307]: osdmap e80: 12 total, 12 up, 12 in 2024-03-16T09:12:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:48 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T09:12:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:48 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2200745343' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-16T09:12:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:48 smithi112 ceph-mon[32307]: Detected new or changed devices on smithi194 2024-03-16T09:12:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:48 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:48 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:48 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:48 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:48 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:48 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:48 smithi112 ceph-mon[32307]: Adjusting osd_memory_target on smithi194 to 4270M 2024-03-16T09:12:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:48 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:48 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:48 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:12:48.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:48 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:48.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:48 smithi159 ceph-mon[36739]: purged_snaps scrub starts 2024-03-16T09:12:48.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:48 smithi159 ceph-mon[36739]: purged_snaps scrub ok 2024-03-16T09:12:48.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:48 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T09:12:48.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:48 smithi159 ceph-mon[36739]: osd.11 [v2:172.21.15.194:6824/3230458513,v1:172.21.15.194:6825/3230458513] boot 2024-03-16T09:12:48.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:48 smithi159 ceph-mon[36739]: osdmap e80: 12 total, 12 up, 12 in 2024-03-16T09:12:48.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:48 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-16T09:12:48.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:48 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2200745343' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-16T09:12:48.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:48 smithi159 ceph-mon[36739]: Detected new or changed devices on smithi194 2024-03-16T09:12:48.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:48 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:48.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:48 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:48.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:48 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:48.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:48 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:48.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:48 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:48.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:48 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-16T09:12:48.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:48 smithi159 ceph-mon[36739]: Adjusting osd_memory_target on smithi194 to 4270M 2024-03-16T09:12:48.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:48 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:48.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:48 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:48.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:48 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:12:48.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:48 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:49.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:49 smithi194 ceph-mon[36729]: osdmap e81: 12 total, 12 up, 12 in 2024-03-16T09:12:49.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:49 smithi194 ceph-mon[36729]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:12:49.451 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:49 smithi112 ceph-mon[32307]: osdmap e81: 12 total, 12 up, 12 in 2024-03-16T09:12:49.451 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:49 smithi112 ceph-mon[32307]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:12:49.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:49 smithi159 ceph-mon[36739]: osdmap e81: 12 total, 12 up, 12 in 2024-03-16T09:12:49.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:49 smithi159 ceph-mon[36739]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:12:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.993+0000 7f7dfa29b700 1 -- 172.21.15.112:0/1374164741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7df4107210 msgr2=0x7f7df41075f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.993+0000 7f7dfa29b700 1 --2- 172.21.15.112:0/1374164741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7df4107210 0x7f7df41075f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7de8009990 tx=0x7f7de8009ca0 comp rx=0 tx=0).stop 2024-03-16T09:12:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.993+0000 7f7dfa29b700 1 -- 172.21.15.112:0/1374164741 shutdown_connections 2024-03-16T09:12:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.993+0000 7f7dfa29b700 1 --2- 172.21.15.112:0/1374164741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7df4101b70 0x7f7df410e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.993+0000 7f7dfa29b700 1 --2- 172.21.15.112:0/1374164741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7df4107210 0x7f7df41075f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.993+0000 7f7dfa29b700 1 --2- 172.21.15.112:0/1374164741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7df41011d0 0x7f7df4101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.993+0000 7f7dfa29b700 1 -- 172.21.15.112:0/1374164741 >> 172.21.15.112:0/1374164741 conn(0x7f7df4078100 msgr2=0x7f7df4078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.993+0000 7f7dfa29b700 1 -- 172.21.15.112:0/1374164741 shutdown_connections 2024-03-16T09:12:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.993+0000 7f7dfa29b700 1 -- 172.21.15.112:0/1374164741 wait complete. 2024-03-16T09:12:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.994+0000 7f7dfa29b700 1 Processor -- start 2024-03-16T09:12:49.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.994+0000 7f7dfa29b700 1 -- start start 2024-03-16T09:12:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.994+0000 7f7dfa29b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7df41011d0 0x7f7df41999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.994+0000 7f7dfa29b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7df4101b70 0x7f7df4199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.994+0000 7f7dfa29b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7df4107210 0x7f7df41a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.994+0000 7f7dfa29b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7df4111630 con 0x7f7df4101b70 2024-03-16T09:12:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.994+0000 7f7dfa29b700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41114b0 con 0x7f7df41011d0 2024-03-16T09:12:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.994+0000 7f7dfa29b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41117b0 con 0x7f7df4107210 2024-03-16T09:12:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.994+0000 7f7df37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7df4101b70 0x7f7df4199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.995+0000 7f7df37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7df4101b70 0x7f7df4199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52834/0 (socket says 172.21.15.112:52834) 2024-03-16T09:12:50.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.995+0000 7f7df37fe700 1 -- 172.21.15.112:0/3909001066 learned_addr learned my addr 172.21.15.112:0/3909001066 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:12:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.995+0000 7f7df3fff700 1 --2- 172.21.15.112:0/3909001066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7df41011d0 0x7f7df41999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.995+0000 7f7df8838700 1 --2- 172.21.15.112:0/3909001066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7df4107210 0x7f7df41a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.995+0000 7f7df37fe700 1 -- 172.21.15.112:0/3909001066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7df4107210 msgr2=0x7f7df41a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.995+0000 7f7df37fe700 1 --2- 172.21.15.112:0/3909001066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7df4107210 0x7f7df41a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.995+0000 7f7df37fe700 1 -- 172.21.15.112:0/3909001066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7df41011d0 msgr2=0x7f7df41999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.995+0000 7f7df37fe700 1 --2- 172.21.15.112:0/3909001066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7df41011d0 0x7f7df41999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.995+0000 7f7df37fe700 1 -- 172.21.15.112:0/3909001066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df41a4fb0 con 0x7f7df4101b70 2024-03-16T09:12:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.995+0000 7f7df8838700 1 --2- 172.21.15.112:0/3909001066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7df4107210 0x7f7df41a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:12:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.995+0000 7f7df37fe700 1 --2- 172.21.15.112:0/3909001066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7df4101b70 0x7f7df4199f30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7de400c970 tx=0x7f7de400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.995+0000 7f7df17fa700 1 -- 172.21.15.112:0/3909001066 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4007aa0 con 0x7f7df4101b70 2024-03-16T09:12:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.995+0000 7f7dfa29b700 1 -- 172.21.15.112:0/3909001066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df41a51e0 con 0x7f7df4101b70 2024-03-16T09:12:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.995+0000 7f7df17fa700 1 -- 172.21.15.112:0/3909001066 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7de4004d30 con 0x7f7df4101b70 2024-03-16T09:12:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.995+0000 7f7dfa29b700 1 -- 172.21.15.112:0/3909001066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df41a56f0 con 0x7f7df4101b70 2024-03-16T09:12:50.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.995+0000 7f7df17fa700 1 -- 172.21.15.112:0/3909001066 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4005690 con 0x7f7df4101b70 2024-03-16T09:12:50.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.996+0000 7f7dfa29b700 1 -- 172.21.15.112:0/3909001066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df41028c0 con 0x7f7df4101b70 2024-03-16T09:12:50.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.997+0000 7f7df17fa700 1 -- 172.21.15.112:0/3909001066 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f7de4004750 con 0x7f7df4101b70 2024-03-16T09:12:50.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.998+0000 7f7df17fa700 1 --2- 172.21.15.112:0/3909001066 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7ddc0a2890 0x7f7ddc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:50.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.998+0000 7f7df3fff700 1 --2- 172.21.15.112:0/3909001066 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7ddc0a2890 0x7f7ddc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:50.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.998+0000 7f7df17fa700 1 -- 172.21.15.112:0/3909001066 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f7de40ce340 con 0x7f7df4101b70 2024-03-16T09:12:50.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:49.998+0000 7f7df3fff700 1 --2- 172.21.15.112:0/3909001066 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7ddc0a2890 0x7f7ddc0a4d50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7de800b5c0 tx=0x7f7de8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:50.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:50.002+0000 7f7df17fa700 1 -- 172.21.15.112:0/3909001066 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7de409b910 con 0x7f7df4101b70 2024-03-16T09:12:50.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:50.199+0000 7f7dfa29b700 1 -- 172.21.15.112:0/3909001066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7df402db70 con 0x7f7df4101b70 2024-03-16T09:12:50.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:50.200+0000 7f7df17fa700 1 -- 172.21.15.112:0/3909001066 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18528 (secure 0 0 0) 0x7f7de409b210 con 0x7f7df4101b70 2024-03-16T09:12:50.203 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:12:50.203 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":81,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:12:48.106465+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:12:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:50.202+0000 7f7dfa29b700 1 -- 172.21.15.112:0/3909001066 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7ddc0a2890 msgr2=0x7f7ddc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:50.202+0000 7f7dfa29b700 1 --2- 172.21.15.112:0/3909001066 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7ddc0a2890 0x7f7ddc0a4d50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7de800b5c0 tx=0x7f7de8005960 comp rx=0 tx=0).stop 2024-03-16T09:12:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:50.202+0000 7f7dfa29b700 1 -- 172.21.15.112:0/3909001066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7df4101b70 msgr2=0x7f7df4199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:50.202+0000 7f7dfa29b700 1 --2- 172.21.15.112:0/3909001066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7df4101b70 0x7f7df4199f30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7de400c970 tx=0x7f7de400cc80 comp rx=0 tx=0).stop 2024-03-16T09:12:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:50.202+0000 7f7dfa29b700 1 -- 172.21.15.112:0/3909001066 shutdown_connections 2024-03-16T09:12:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:50.202+0000 7f7dfa29b700 1 --2- 172.21.15.112:0/3909001066 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7ddc0a2890 0x7f7ddc0a4d50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:50.202+0000 7f7dfa29b700 1 --2- 172.21.15.112:0/3909001066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7df41011d0 0x7f7df41999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:50.202+0000 7f7dfa29b700 1 --2- 172.21.15.112:0/3909001066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7df4107210 0x7f7df41a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:50.202+0000 7f7dfa29b700 1 --2- 172.21.15.112:0/3909001066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7df4101b70 0x7f7df4199f30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:50.202+0000 7f7dfa29b700 1 -- 172.21.15.112:0/3909001066 >> 172.21.15.112:0/3909001066 conn(0x7f7df4078100 msgr2=0x7f7df40ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:50.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:50.202+0000 7f7dfa29b700 1 -- 172.21.15.112:0/3909001066 shutdown_connections 2024-03-16T09:12:50.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:50.202+0000 7f7dfa29b700 1 -- 172.21.15.112:0/3909001066 wait complete. 2024-03-16T09:12:50.550 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:50 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3909001066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:12:50.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:50 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3909001066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:12:50.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:50 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3909001066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:12:50.902 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T09:08:57.135312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T09:12:50.903 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-16T09:12:51.149 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:12:51.521 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:51 smithi112 ceph-mon[32307]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:12:51.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:51 smithi194 ceph-mon[36729]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:12:51.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:51 smithi159 ceph-mon[36739]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:12:52.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.966+0000 7f669a908700 1 -- 172.21.15.112:0/2314413935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6694107690 msgr2=0x7f669410f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:52.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.966+0000 7f669a908700 1 --2- 172.21.15.112:0/2314413935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6694107690 0x7f669410f210 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f668c00b0d0 tx=0x7f668c00b3e0 comp rx=0 tx=0).stop 2024-03-16T09:12:52.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.966+0000 7f669a908700 1 -- 172.21.15.112:0/2314413935 shutdown_connections 2024-03-16T09:12:52.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.966+0000 7f669a908700 1 --2- 172.21.15.112:0/2314413935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6694107690 0x7f669410f210 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:52.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.966+0000 7f669a908700 1 --2- 172.21.15.112:0/2314413935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6694100e70 0x7f6694101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:52.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.966+0000 7f669a908700 1 --2- 172.21.15.112:0/2314413935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6694101820 0x7f6694106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:52.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.966+0000 7f669a908700 1 -- 172.21.15.112:0/2314413935 >> 172.21.15.112:0/2314413935 conn(0x7f6694076510 msgr2=0x7f6694076920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:52.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.966+0000 7f669a908700 1 -- 172.21.15.112:0/2314413935 shutdown_connections 2024-03-16T09:12:52.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.966+0000 7f669a908700 1 -- 172.21.15.112:0/2314413935 wait complete. 2024-03-16T09:12:52.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.967+0000 7f669a908700 1 Processor -- start 2024-03-16T09:12:52.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.967+0000 7f669a908700 1 -- start start 2024-03-16T09:12:52.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.967+0000 7f669a908700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6694100e70 0x7f6694101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:52.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.967+0000 7f669a908700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6694101820 0x7f6694102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:52.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.967+0000 7f669a908700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6694107690 0x7f6694102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:52.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.967+0000 7f669a908700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66941127e0 con 0x7f6694101820 2024-03-16T09:12:52.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.967+0000 7f669a908700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6694112660 con 0x7f6694107690 2024-03-16T09:12:52.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.967+0000 7f669a908700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6694112960 con 0x7f6694100e70 2024-03-16T09:12:52.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.968+0000 7f66937fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6694101820 0x7f6694102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:52.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.968+0000 7f66937fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6694101820 0x7f6694102330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36694/0 (socket says 172.21.15.112:36694) 2024-03-16T09:12:52.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.968+0000 7f66937fe700 1 -- 172.21.15.112:0/3681366494 learned_addr learned my addr 172.21.15.112:0/3681366494 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:12:52.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.968+0000 7f6693fff700 1 --2- 172.21.15.112:0/3681366494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6694100e70 0x7f6694101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:52.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.968+0000 7f6698ea5700 1 --2- 172.21.15.112:0/3681366494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6694107690 0x7f6694102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:52.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.968+0000 7f66937fe700 1 -- 172.21.15.112:0/3681366494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6694100e70 msgr2=0x7f6694101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:52.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.968+0000 7f66937fe700 1 --2- 172.21.15.112:0/3681366494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6694100e70 0x7f6694101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:52.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.968+0000 7f66937fe700 1 -- 172.21.15.112:0/3681366494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6694107690 msgr2=0x7f6694102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:52.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.968+0000 7f66937fe700 1 --2- 172.21.15.112:0/3681366494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6694107690 0x7f6694102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:52.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.968+0000 7f66937fe700 1 -- 172.21.15.112:0/3681366494 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66941a7060 con 0x7f6694101820 2024-03-16T09:12:52.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.968+0000 7f66937fe700 1 --2- 172.21.15.112:0/3681366494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6694101820 0x7f6694102330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f668800b4d0 tx=0x7f668800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:52.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.968+0000 7f66917fa700 1 -- 172.21.15.112:0/3681366494 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688015020 con 0x7f6694101820 2024-03-16T09:12:52.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.968+0000 7f669a908700 1 -- 172.21.15.112:0/3681366494 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66941a7350 con 0x7f6694101820 2024-03-16T09:12:52.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.968+0000 7f66917fa700 1 -- 172.21.15.112:0/3681366494 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6688011d30 con 0x7f6694101820 2024-03-16T09:12:52.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.968+0000 7f669a908700 1 -- 172.21.15.112:0/3681366494 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66941a7890 con 0x7f6694101820 2024-03-16T09:12:52.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.968+0000 7f66917fa700 1 -- 172.21.15.112:0/3681366494 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688010e50 con 0x7f6694101820 2024-03-16T09:12:52.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.969+0000 7f669a908700 1 -- 172.21.15.112:0/3681366494 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6694068f90 con 0x7f6694101820 2024-03-16T09:12:52.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.970+0000 7f66917fa700 1 -- 172.21.15.112:0/3681366494 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f6688018020 con 0x7f6694101820 2024-03-16T09:12:52.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.971+0000 7f66917fa700 1 --2- 172.21.15.112:0/3681366494 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f667c0a2a30 0x7f667c0a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:52.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.971+0000 7f6693fff700 1 --2- 172.21.15.112:0/3681366494 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f667c0a2a30 0x7f667c0a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:52.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.971+0000 7f66917fa700 1 -- 172.21.15.112:0/3681366494 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f66880ce180 con 0x7f6694101820 2024-03-16T09:12:52.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.972+0000 7f6693fff700 1 --2- 172.21.15.112:0/3681366494 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f667c0a2a30 0x7f667c0a4ef0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6684004630 tx=0x7f668400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:52.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:52.975+0000 7f66917fa700 1 -- 172.21.15.112:0/3681366494 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f668801f020 con 0x7f6694101820 2024-03-16T09:12:53.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:53.170+0000 7f669a908700 1 -- 172.21.15.112:0/3681366494 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f66941a7ba0 con 0x7f6694101820 2024-03-16T09:12:53.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:53.171+0000 7f66917fa700 1 -- 172.21.15.112:0/3681366494 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f668809b620 con 0x7f6694101820 2024-03-16T09:12:53.174 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-03-16T09:12:53.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:53.173+0000 7f669a908700 1 -- 172.21.15.112:0/3681366494 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f667c0a2a30 msgr2=0x7f667c0a4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:53.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:53.173+0000 7f669a908700 1 --2- 172.21.15.112:0/3681366494 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f667c0a2a30 0x7f667c0a4ef0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6684004630 tx=0x7f668400b3c0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:53.173+0000 7f669a908700 1 -- 172.21.15.112:0/3681366494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6694101820 msgr2=0x7f6694102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:53.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:53.173+0000 7f669a908700 1 --2- 172.21.15.112:0/3681366494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6694101820 0x7f6694102330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f668800b4d0 tx=0x7f668800b7e0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:53.173+0000 7f669a908700 1 -- 172.21.15.112:0/3681366494 shutdown_connections 2024-03-16T09:12:53.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:53.173+0000 7f669a908700 1 --2- 172.21.15.112:0/3681366494 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f667c0a2a30 0x7f667c0a4ef0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:53.173+0000 7f669a908700 1 --2- 172.21.15.112:0/3681366494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6694107690 0x7f6694102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:53.173+0000 7f669a908700 1 --2- 172.21.15.112:0/3681366494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6694100e70 0x7f6694101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:53.173+0000 7f669a908700 1 --2- 172.21.15.112:0/3681366494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6694101820 0x7f6694102330 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:53.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:53.173+0000 7f669a908700 1 -- 172.21.15.112:0/3681366494 >> 172.21.15.112:0/3681366494 conn(0x7f6694076510 msgr2=0x7f66940feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:53.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:53.173+0000 7f669a908700 1 -- 172.21.15.112:0/3681366494 shutdown_connections 2024-03-16T09:12:53.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:53.173+0000 7f669a908700 1 -- 172.21.15.112:0/3681366494 wait complete. 2024-03-16T09:12:53.902 INFO:tasks.cephadm:Adding mds.a on smithi112 2024-03-16T09:12:53.902 INFO:tasks.cephadm:Adding mds.d on smithi112 2024-03-16T09:12:53.902 INFO:tasks.cephadm:Adding mds.g on smithi112 2024-03-16T09:12:53.902 INFO:tasks.cephadm:Adding mds.j on smithi112 2024-03-16T09:12:53.902 INFO:tasks.cephadm:Adding mds.b on smithi159 2024-03-16T09:12:53.902 INFO:tasks.cephadm:Adding mds.e on smithi159 2024-03-16T09:12:53.902 INFO:tasks.cephadm:Adding mds.h on smithi159 2024-03-16T09:12:53.902 INFO:tasks.cephadm:Adding mds.k on smithi159 2024-03-16T09:12:53.903 INFO:tasks.cephadm:Adding mds.c on smithi194 2024-03-16T09:12:53.903 INFO:tasks.cephadm:Adding mds.f on smithi194 2024-03-16T09:12:53.903 INFO:tasks.cephadm:Adding mds.i on smithi194 2024-03-16T09:12:53.903 INFO:tasks.cephadm:Adding mds.l on smithi194 2024-03-16T09:12:53.903 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph orch apply mds all '12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l' 2024-03-16T09:12:54.153 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:12:54.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:53 smithi159 ceph-mon[36739]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:12:54.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:53 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3681366494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:12:54.274 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:53 smithi194 ceph-mon[36729]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:12:54.274 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:53 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3681366494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:12:54.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:53 smithi112 ceph-mon[32307]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:12:54.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:53 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3681366494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:12:55.769 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.765+0000 7fe053b0d700 1 -- 172.21.15.194:0/3944369347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe04c102da0 msgr2=0x7fe04c109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:55.769 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.765+0000 7fe053b0d700 1 --2- 172.21.15.194:0/3944369347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe04c102da0 0x7fe04c109650 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe04400b0a0 tx=0x7fe04400b3b0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.769 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.765+0000 7fe053b0d700 1 -- 172.21.15.194:0/3944369347 shutdown_connections 2024-03-16T09:12:55.769 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.765+0000 7fe053b0d700 1 --2- 172.21.15.194:0/3944369347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe04c101200 0x7fe04c101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.769 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.765+0000 7fe053b0d700 1 --2- 172.21.15.194:0/3944369347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe04c102da0 0x7fe04c109650 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.770 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.765+0000 7fe053b0d700 1 --2- 172.21.15.194:0/3944369347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04c102400 0x7fe04c102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.770 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.765+0000 7fe053b0d700 1 -- 172.21.15.194:0/3944369347 >> 172.21.15.194:0/3944369347 conn(0x7fe04c0fc7a0 msgr2=0x7fe04c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:55.770 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.765+0000 7fe053b0d700 1 -- 172.21.15.194:0/3944369347 shutdown_connections 2024-03-16T09:12:55.770 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.765+0000 7fe053b0d700 1 -- 172.21.15.194:0/3944369347 wait complete. 2024-03-16T09:12:55.770 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.766+0000 7fe053b0d700 1 Processor -- start 2024-03-16T09:12:55.770 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.766+0000 7fe053b0d700 1 -- start start 2024-03-16T09:12:55.770 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.766+0000 7fe053b0d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04c101200 0x7fe04c105190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:55.770 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.766+0000 7fe053b0d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe04c102400 0x7fe04c1037e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:55.770 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.766+0000 7fe053b0d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe04c102da0 0x7fe04c103d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:55.770 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.766+0000 7fe053b0d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c10cfd0 con 0x7fe04c101200 2024-03-16T09:12:55.770 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.766+0000 7fe053b0d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c10ce50 con 0x7fe04c102400 2024-03-16T09:12:55.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.766+0000 7fe053b0d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c10d150 con 0x7fe04c102da0 2024-03-16T09:12:55.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.766+0000 7fe0520aa700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe04c102da0 0x7fe04c103d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:55.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.766+0000 7fe0520aa700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe04c102da0 0x7fe04c103d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:56396/0 (socket says 172.21.15.194:56396) 2024-03-16T09:12:55.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.766+0000 7fe0520aa700 1 -- 172.21.15.194:0/736422221 learned_addr learned my addr 172.21.15.194:0/736422221 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-03-16T09:12:55.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.766+0000 7fe0518a9700 1 --2- 172.21.15.194:0/736422221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04c101200 0x7fe04c105190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:55.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.766+0000 7fe0520aa700 1 -- 172.21.15.194:0/736422221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe04c102400 msgr2=0x7fe04c1037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:55.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.766+0000 7fe0520aa700 1 --2- 172.21.15.194:0/736422221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe04c102400 0x7fe04c1037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.766+0000 7fe0520aa700 1 -- 172.21.15.194:0/736422221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04c101200 msgr2=0x7fe04c105190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:55.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.766+0000 7fe0520aa700 1 --2- 172.21.15.194:0/736422221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04c101200 0x7fe04c105190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:55.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.766+0000 7fe0520aa700 1 -- 172.21.15.194:0/736422221 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04c1045d0 con 0x7fe04c102da0 2024-03-16T09:12:55.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.766+0000 7fe0518a9700 1 --2- 172.21.15.194:0/736422221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04c101200 0x7fe04c105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:12:55.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.767+0000 7fe0520aa700 1 --2- 172.21.15.194:0/736422221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe04c102da0 0x7fe04c103d20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe044004dc0 tx=0x7fe044004ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:55.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.767+0000 7fe042ffd700 1 -- 172.21.15.194:0/736422221 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04401f070 con 0x7fe04c102da0 2024-03-16T09:12:55.771 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.767+0000 7fe042ffd700 1 -- 172.21.15.194:0/736422221 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe044007d00 con 0x7fe04c102da0 2024-03-16T09:12:55.772 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.767+0000 7fe053b0d700 1 -- 172.21.15.194:0/736422221 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04c1a2710 con 0x7fe04c102da0 2024-03-16T09:12:55.772 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.767+0000 7fe042ffd700 1 -- 172.21.15.194:0/736422221 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe044005650 con 0x7fe04c102da0 2024-03-16T09:12:55.772 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.767+0000 7fe053b0d700 1 -- 172.21.15.194:0/736422221 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe04c1a2b70 con 0x7fe04c102da0 2024-03-16T09:12:55.772 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.768+0000 7fe053b0d700 1 -- 172.21.15.194:0/736422221 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe04c068f90 con 0x7fe04c102da0 2024-03-16T09:12:55.773 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.769+0000 7fe042ffd700 1 -- 172.21.15.194:0/736422221 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fe0440057f0 con 0x7fe04c102da0 2024-03-16T09:12:55.773 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.769+0000 7fe042ffd700 1 --2- 172.21.15.194:0/736422221 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe0380a2890 0x7fe0380a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:55.773 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.770+0000 7fe042ffd700 1 -- 172.21.15.194:0/736422221 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe04400f020 con 0x7fe04c102da0 2024-03-16T09:12:55.773 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.770+0000 7fe0518a9700 1 --2- 172.21.15.194:0/736422221 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe0380a2890 0x7fe0380a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:55.773 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.770+0000 7fe0518a9700 1 --2- 172.21.15.194:0/736422221 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe0380a2890 0x7fe0380a4d50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe04c102260 tx=0x7fe048005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:55.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.773+0000 7fe042ffd700 1 -- 172.21.15.194:0/736422221 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe044028020 con 0x7fe04c102da0 2024-03-16T09:12:55.991 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.989+0000 7fe053b0d700 1 -- 172.21.15.194:0/736422221 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}) -- 0x7fe04c0fe960 con 0x7fe0380a2890 2024-03-16T09:12:55.998 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.996+0000 7fe042ffd700 1 -- 172.21.15.194:0/736422221 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fe04c0fe960 con 0x7fe0380a2890 2024-03-16T09:12:55.999 INFO:teuthology.orchestra.run.smithi194.stdout:Scheduled mds.all update... 2024-03-16T09:12:56.002 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.999+0000 7fe053b0d700 1 -- 172.21.15.194:0/736422221 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe0380a2890 msgr2=0x7fe0380a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:56.002 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.999+0000 7fe053b0d700 1 --2- 172.21.15.194:0/736422221 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe0380a2890 0x7fe0380a4d50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe04c102260 tx=0x7fe048005f90 comp rx=0 tx=0).stop 2024-03-16T09:12:56.002 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.999+0000 7fe053b0d700 1 -- 172.21.15.194:0/736422221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe04c102da0 msgr2=0x7fe04c103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:56.002 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:55.999+0000 7fe053b0d700 1 --2- 172.21.15.194:0/736422221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe04c102da0 0x7fe04c103d20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe044004dc0 tx=0x7fe044004ea0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.002 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:56.000+0000 7fe053b0d700 1 -- 172.21.15.194:0/736422221 shutdown_connections 2024-03-16T09:12:56.003 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:56.000+0000 7fe053b0d700 1 --2- 172.21.15.194:0/736422221 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe0380a2890 0x7fe0380a4d50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.003 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:56.000+0000 7fe053b0d700 1 --2- 172.21.15.194:0/736422221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe04c102da0 0x7fe04c103d20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.003 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:56.000+0000 7fe053b0d700 1 --2- 172.21.15.194:0/736422221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04c101200 0x7fe04c105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.003 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:56.000+0000 7fe053b0d700 1 --2- 172.21.15.194:0/736422221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe04c102400 0x7fe04c1037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:56.003 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:56.000+0000 7fe053b0d700 1 -- 172.21.15.194:0/736422221 >> 172.21.15.194:0/736422221 conn(0x7fe04c0fc7a0 msgr2=0x7fe04c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:56.003 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:56.000+0000 7fe053b0d700 1 -- 172.21.15.194:0/736422221 shutdown_connections 2024-03-16T09:12:56.003 INFO:teuthology.orchestra.run.smithi194.stderr:2024-03-16T09:12:56.000+0000 7fe053b0d700 1 -- 172.21.15.194:0/736422221 wait complete. 2024-03-16T09:12:56.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:55 smithi159 ceph-mon[36739]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:12:56.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:55 smithi112 ceph-mon[32307]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:12:56.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:55 smithi194 ceph-mon[36729]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:12:56.737 DEBUG:teuthology.orchestra.run.smithi112:mds.a> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.a.service 2024-03-16T09:12:56.740 DEBUG:teuthology.orchestra.run.smithi112:mds.d> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.d.service 2024-03-16T09:12:56.783 DEBUG:teuthology.orchestra.run.smithi112:mds.g> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.g.service 2024-03-16T09:12:56.826 DEBUG:teuthology.orchestra.run.smithi112:mds.j> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.j.service 2024-03-16T09:12:56.869 DEBUG:teuthology.orchestra.run.smithi159:mds.b> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.b.service 2024-03-16T09:12:56.872 DEBUG:teuthology.orchestra.run.smithi159:mds.e> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.e.service 2024-03-16T09:12:56.915 DEBUG:teuthology.orchestra.run.smithi159:mds.h> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.h.service 2024-03-16T09:12:56.958 DEBUG:teuthology.orchestra.run.smithi159:mds.k> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.k.service 2024-03-16T09:12:57.000 DEBUG:teuthology.orchestra.run.smithi194:mds.c> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.c.service 2024-03-16T09:12:57.003 DEBUG:teuthology.orchestra.run.smithi194:mds.f> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.f.service 2024-03-16T09:12:57.047 DEBUG:teuthology.orchestra.run.smithi194:mds.i> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.i.service 2024-03-16T09:12:57.091 DEBUG:teuthology.orchestra.run.smithi194:mds.l> sudo journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.l.service 2024-03-16T09:12:57.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:56 smithi112 ceph-mon[32307]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:12:57.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:56 smithi112 ceph-mon[32307]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2024-03-16T09:12:57.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:57.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:12:57.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:57.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:12:57.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:57.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:12:57.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:12:57.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:56 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:57.134 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-16T09:12:57.135 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:12:57.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:56 smithi159 ceph-mon[36739]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:12:57.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:56 smithi159 ceph-mon[36739]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2024-03-16T09:12:57.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:57.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:12:57.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:57.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:12:57.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:57.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:12:57.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:56 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:12:57.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:57.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.246+0000 7f53a2627640 1 -- 172.21.15.112:0/1841016629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c100700 msgr2=0x7f539c100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.246+0000 7f53a2627640 1 --2- 172.21.15.112:0/1841016629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c100700 0x7f539c100b20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f538c009900 tx=0x7f538c02bc30 comp rx=0 tx=0).stop 2024-03-16T09:12:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.246+0000 7f53a2627640 1 -- 172.21.15.112:0/1841016629 shutdown_connections 2024-03-16T09:12:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.246+0000 7f53a2627640 1 --2- 172.21.15.112:0/1841016629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f539c1022b0 0x7f539c108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.246+0000 7f53a2627640 1 --2- 172.21.15.112:0/1841016629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f539c1018f0 0x7f539c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.246+0000 7f53a2627640 1 --2- 172.21.15.112:0/1841016629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c100700 0x7f539c100b20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.246+0000 7f53a2627640 1 -- 172.21.15.112:0/1841016629 >> 172.21.15.112:0/1841016629 conn(0x7f539c0fbc70 msgr2=0x7f539c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.246+0000 7f53a2627640 1 -- 172.21.15.112:0/1841016629 shutdown_connections 2024-03-16T09:12:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.246+0000 7f53a2627640 1 -- 172.21.15.112:0/1841016629 wait complete. 2024-03-16T09:12:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.246+0000 7f53a2627640 1 Processor -- start 2024-03-16T09:12:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.246+0000 7f53a2627640 1 -- start start 2024-03-16T09:12:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f53a2627640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f539c100700 0x7f539c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f53a2627640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c1018f0 0x7f539c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f53a2627640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f539c1022b0 0x7f539c1a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:57.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f53a2627640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f539c10c460 con 0x7f539c1018f0 2024-03-16T09:12:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f53a2627640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f539c10c2e0 con 0x7f539c100700 2024-03-16T09:12:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f53a2627640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f539c10c5e0 con 0x7f539c1022b0 2024-03-16T09:12:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f539b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c1018f0 0x7f539c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f539b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c1018f0 0x7f539c19ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36722/0 (socket says 172.21.15.112:36722) 2024-03-16T09:12:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f539b7fe640 1 -- 172.21.15.112:0/107637789 learned_addr learned my addr 172.21.15.112:0/107637789 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:12:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f53a0bc4640 1 --2- 172.21.15.112:0/107637789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f539c1022b0 0x7f539c1a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f539b7fe640 1 -- 172.21.15.112:0/107637789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f539c1022b0 msgr2=0x7f539c1a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f539bfff640 1 --2- 172.21.15.112:0/107637789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f539c100700 0x7f539c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f539b7fe640 1 --2- 172.21.15.112:0/107637789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f539c1022b0 0x7f539c1a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f539b7fe640 1 -- 172.21.15.112:0/107637789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f539c100700 msgr2=0x7f539c0694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f539b7fe640 1 --2- 172.21.15.112:0/107637789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f539c100700 0x7f539c0694b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f539b7fe640 1 -- 172.21.15.112:0/107637789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c1a1a60 con 0x7f539c1018f0 2024-03-16T09:12:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f539b7fe640 1 --2- 172.21.15.112:0/107637789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c1018f0 0x7f539c19ed20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f539000e9f0 tx=0x7f539000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:57.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f53997fa640 1 -- 172.21.15.112:0/107637789 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539000cd80 con 0x7f539c1018f0 2024-03-16T09:12:57.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f53a2627640 1 -- 172.21.15.112:0/107637789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539c1a1cf0 con 0x7f539c1018f0 2024-03-16T09:12:57.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f53997fa640 1 -- 172.21.15.112:0/107637789 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5390005520 con 0x7f539c1018f0 2024-03-16T09:12:57.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f53997fa640 1 -- 172.21.15.112:0/107637789 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390010690 con 0x7f539c1018f0 2024-03-16T09:12:57.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.247+0000 7f53a2627640 1 -- 172.21.15.112:0/107637789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539c1a2190 con 0x7f539c1018f0 2024-03-16T09:12:57.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.248+0000 7f53a2627640 1 -- 172.21.15.112:0/107637789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f539c107a20 con 0x7f539c1018f0 2024-03-16T09:12:57.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.249+0000 7f53997fa640 1 -- 172.21.15.112:0/107637789 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f53900046c0 con 0x7f539c1018f0 2024-03-16T09:12:57.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.250+0000 7f53997fa640 1 --2- 172.21.15.112:0/107637789 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f53840a2830 0x7f53840a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:57.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.250+0000 7f539bfff640 1 --2- 172.21.15.112:0/107637789 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f53840a2830 0x7f53840a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:57.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.250+0000 7f53997fa640 1 -- 172.21.15.112:0/107637789 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f53900cea40 con 0x7f539c1018f0 2024-03-16T09:12:57.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.251+0000 7f539bfff640 1 --2- 172.21.15.112:0/107637789 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f53840a2830 0x7f53840a4cf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f538c02ca30 tx=0x7f538c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:57.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.255+0000 7f53997fa640 1 -- 172.21.15.112:0/107637789 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f539009bf80 con 0x7f539c1018f0 2024-03-16T09:12:57.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:57 smithi194 ceph-mon[36729]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:12:57.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:57 smithi194 ceph-mon[36729]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2024-03-16T09:12:57.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:57.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:12:57.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:57.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:12:57.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:12:57.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:12:57.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:12:57.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:12:57.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.429+0000 7f53a2627640 1 -- 172.21.15.112:0/107637789 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f539c02d400 con 0x7f539c1018f0 2024-03-16T09:12:57.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.430+0000 7f53997fa640 1 -- 172.21.15.112:0/107637789 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18528 (secure 0 0 0) 0x7f539009b880 con 0x7f539c1018f0 2024-03-16T09:12:57.432 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:12:57.432 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":81,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:12:48.106465+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:12:57.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.432+0000 7f53a2627640 1 -- 172.21.15.112:0/107637789 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f53840a2830 msgr2=0x7f53840a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.432+0000 7f53a2627640 1 --2- 172.21.15.112:0/107637789 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f53840a2830 0x7f53840a4cf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f538c02ca30 tx=0x7f538c000bc0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.432+0000 7f53a2627640 1 -- 172.21.15.112:0/107637789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c1018f0 msgr2=0x7f539c19ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.432+0000 7f53a2627640 1 --2- 172.21.15.112:0/107637789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c1018f0 0x7f539c19ed20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f539000e9f0 tx=0x7f539000edf0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.432+0000 7f53a2627640 1 -- 172.21.15.112:0/107637789 shutdown_connections 2024-03-16T09:12:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.432+0000 7f53a2627640 1 --2- 172.21.15.112:0/107637789 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f53840a2830 0x7f53840a4cf0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.432+0000 7f53a2627640 1 --2- 172.21.15.112:0/107637789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f539c1022b0 0x7f539c1a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.432+0000 7f53a2627640 1 --2- 172.21.15.112:0/107637789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f539c1018f0 0x7f539c19ed20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.432+0000 7f53a2627640 1 --2- 172.21.15.112:0/107637789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f539c100700 0x7f539c0694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.432+0000 7f53a2627640 1 -- 172.21.15.112:0/107637789 >> 172.21.15.112:0/107637789 conn(0x7f539c0fbc70 msgr2=0x7f539c104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.432+0000 7f53a2627640 1 -- 172.21.15.112:0/107637789 shutdown_connections 2024-03-16T09:12:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.432+0000 7f53a2627640 1 -- 172.21.15.112:0/107637789 wait complete. 2024-03-16T09:12:57.445 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T09:08:57.135312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T09:12:57.446 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T09:12:57.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.549+0000 7f9415097640 1 -- 172.21.15.112:0/1661423406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94100ff850 msgr2=0x7f94100ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.549+0000 7f9415097640 1 --2- 172.21.15.112:0/1661423406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94100ff850 0x7f94100ffcd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9404009900 tx=0x7f940402bc30 comp rx=0 tx=0).stop 2024-03-16T09:12:57.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.549+0000 7f9415097640 1 -- 172.21.15.112:0/1661423406 shutdown_connections 2024-03-16T09:12:57.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.549+0000 7f9415097640 1 --2- 172.21.15.112:0/1661423406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9410100210 0x7f9410109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.549+0000 7f9415097640 1 --2- 172.21.15.112:0/1661423406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94100ff850 0x7f94100ffcd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.549+0000 7f9415097640 1 --2- 172.21.15.112:0/1661423406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9410102de0 0x7f9410103200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.549+0000 7f9415097640 1 -- 172.21.15.112:0/1661423406 >> 172.21.15.112:0/1661423406 conn(0x7f94100fb3f0 msgr2=0x7f94100fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:57.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.549+0000 7f9415097640 1 -- 172.21.15.112:0/1661423406 shutdown_connections 2024-03-16T09:12:57.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.549+0000 7f9415097640 1 -- 172.21.15.112:0/1661423406 wait complete. 2024-03-16T09:12:57.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.549+0000 7f9415097640 1 Processor -- start 2024-03-16T09:12:57.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.550+0000 7f9415097640 1 -- start start 2024-03-16T09:12:57.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.550+0000 7f9415097640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94100ff850 0x7f9410197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:57.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.550+0000 7f9415097640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9410100210 0x7f9410198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:57.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.550+0000 7f9415097640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9410102de0 0x7f941019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:57.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.550+0000 7f9415097640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f941010d590 con 0x7f9410100210 2024-03-16T09:12:57.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.550+0000 7f9415097640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f941010d410 con 0x7f9410102de0 2024-03-16T09:12:57.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.550+0000 7f9415097640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f941010d710 con 0x7f94100ff850 2024-03-16T09:12:57.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.550+0000 7f940e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9410100210 0x7f9410198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:57.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.550+0000 7f940e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9410100210 0x7f9410198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36752/0 (socket says 172.21.15.112:36752) 2024-03-16T09:12:57.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.550+0000 7f940e59c640 1 -- 172.21.15.112:0/981756846 learned_addr learned my addr 172.21.15.112:0/981756846 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:12:57.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.550+0000 7f940ed9d640 1 --2- 172.21.15.112:0/981756846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94100ff850 0x7f9410197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:57.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.550+0000 7f940f59e640 1 --2- 172.21.15.112:0/981756846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9410102de0 0x7f941019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:57.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.550+0000 7f940e59c640 1 -- 172.21.15.112:0/981756846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94100ff850 msgr2=0x7f9410197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.550+0000 7f940e59c640 1 --2- 172.21.15.112:0/981756846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94100ff850 0x7f9410197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.550+0000 7f940e59c640 1 -- 172.21.15.112:0/981756846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9410102de0 msgr2=0x7f941019f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.550+0000 7f940e59c640 1 --2- 172.21.15.112:0/981756846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9410102de0 0x7f941019f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.550+0000 7f940e59c640 1 -- 172.21.15.112:0/981756846 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f941019f7e0 con 0x7f9410100210 2024-03-16T09:12:57.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.550+0000 7f940e59c640 1 --2- 172.21.15.112:0/981756846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9410100210 0x7f9410198100 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9404009230 tx=0x7f940402c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:57.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.550+0000 7f93f7fff640 1 -- 172.21.15.112:0/981756846 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9404043020 con 0x7f9410100210 2024-03-16T09:12:57.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.551+0000 7f9415097640 1 -- 172.21.15.112:0/981756846 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f941019fa70 con 0x7f9410100210 2024-03-16T09:12:57.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.551+0000 7f93f7fff640 1 -- 172.21.15.112:0/981756846 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f94040043d0 con 0x7f9410100210 2024-03-16T09:12:57.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.551+0000 7f93f7fff640 1 -- 172.21.15.112:0/981756846 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9404003c30 con 0x7f9410100210 2024-03-16T09:12:57.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.551+0000 7f9415097640 1 -- 172.21.15.112:0/981756846 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f941019ffc0 con 0x7f9410100210 2024-03-16T09:12:57.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.552+0000 7f9415097640 1 -- 172.21.15.112:0/981756846 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9410108ca0 con 0x7f9410100210 2024-03-16T09:12:57.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.552+0000 7f93f7fff640 1 -- 172.21.15.112:0/981756846 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f9404046020 con 0x7f9410100210 2024-03-16T09:12:57.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.553+0000 7f93f7fff640 1 --2- 172.21.15.112:0/981756846 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f93f80a2800 0x7f93f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:57.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.553+0000 7f940ed9d640 1 --2- 172.21.15.112:0/981756846 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f93f80a2800 0x7f93f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:57.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.553+0000 7f93f7fff640 1 -- 172.21.15.112:0/981756846 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f94040f0800 con 0x7f9410100210 2024-03-16T09:12:57.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.553+0000 7f940ed9d640 1 --2- 172.21.15.112:0/981756846 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f93f80a2800 0x7f93f80a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9400005fd0 tx=0x7f940000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:57.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.557+0000 7f93f7fff640 1 -- 172.21.15.112:0/981756846 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f940404d020 con 0x7f9410100210 2024-03-16T09:12:57.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.730+0000 7f9415097640 1 -- 172.21.15.112:0/981756846 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f941002d360 con 0x7f9410100210 2024-03-16T09:12:57.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.730+0000 7f93f7fff640 1 -- 172.21.15.112:0/981756846 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f94040b9020 con 0x7f9410100210 2024-03-16T09:12:57.732 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-03-16T09:12:57.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.732+0000 7f9415097640 1 -- 172.21.15.112:0/981756846 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f93f80a2800 msgr2=0x7f93f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.732+0000 7f9415097640 1 --2- 172.21.15.112:0/981756846 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f93f80a2800 0x7f93f80a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9400005fd0 tx=0x7f940000b410 comp rx=0 tx=0).stop 2024-03-16T09:12:57.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.732+0000 7f9415097640 1 -- 172.21.15.112:0/981756846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9410100210 msgr2=0x7f9410198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.732+0000 7f9415097640 1 --2- 172.21.15.112:0/981756846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9410100210 0x7f9410198100 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9404009230 tx=0x7f940402c820 comp rx=0 tx=0).stop 2024-03-16T09:12:57.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.732+0000 7f9415097640 1 -- 172.21.15.112:0/981756846 shutdown_connections 2024-03-16T09:12:57.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.732+0000 7f9415097640 1 --2- 172.21.15.112:0/981756846 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f93f80a2800 0x7f93f80a4cc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.732+0000 7f9415097640 1 --2- 172.21.15.112:0/981756846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9410102de0 0x7f941019f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.732+0000 7f9415097640 1 --2- 172.21.15.112:0/981756846 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9410100210 0x7f9410198100 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.732+0000 7f9415097640 1 --2- 172.21.15.112:0/981756846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94100ff850 0x7f9410197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.732+0000 7f9415097640 1 -- 172.21.15.112:0/981756846 >> 172.21.15.112:0/981756846 conn(0x7f94100fb3f0 msgr2=0x7f9410105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:57.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.733+0000 7f9415097640 1 -- 172.21.15.112:0/981756846 shutdown_connections 2024-03-16T09:12:57.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.733+0000 7f9415097640 1 -- 172.21.15.112:0/981756846 wait complete. 2024-03-16T09:12:57.745 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:12:57.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.844+0000 7ffadfe52640 1 -- 172.21.15.112:0/3509975545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad81033b0 msgr2=0x7ffad8109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.844+0000 7ffadfe52640 1 --2- 172.21.15.112:0/3509975545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad81033b0 0x7ffad8109c40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009ca0 tx=0x7ffad002c0f0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.844+0000 7ffadfe52640 1 -- 172.21.15.112:0/3509975545 shutdown_connections 2024-03-16T09:12:57.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.844+0000 7ffadfe52640 1 --2- 172.21.15.112:0/3509975545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad81033b0 0x7ffad8109c40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.844+0000 7ffadfe52640 1 --2- 172.21.15.112:0/3509975545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffad81029f0 0x7ffad8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.844+0000 7ffadfe52640 1 --2- 172.21.15.112:0/3509975545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffad8101800 0x7ffad8101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.844+0000 7ffadfe52640 1 -- 172.21.15.112:0/3509975545 >> 172.21.15.112:0/3509975545 conn(0x7ffad80fcd90 msgr2=0x7ffad80ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:57.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.844+0000 7ffadfe52640 1 -- 172.21.15.112:0/3509975545 shutdown_connections 2024-03-16T09:12:57.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.844+0000 7ffadfe52640 1 -- 172.21.15.112:0/3509975545 wait complete. 2024-03-16T09:12:57.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.845+0000 7ffadfe52640 1 Processor -- start 2024-03-16T09:12:57.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.845+0000 7ffadfe52640 1 -- start start 2024-03-16T09:12:57.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.845+0000 7ffadfe52640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffad8101800 0x7ffad8197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:57.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.845+0000 7ffadfe52640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffad81029f0 0x7ffad81980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:57.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.845+0000 7ffadfe52640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad81033b0 0x7ffad819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:57.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.845+0000 7ffadfe52640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffad810d5b0 con 0x7ffad81033b0 2024-03-16T09:12:57.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.845+0000 7ffadfe52640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffad810d430 con 0x7ffad8101800 2024-03-16T09:12:57.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.845+0000 7ffadfe52640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffad810d730 con 0x7ffad81029f0 2024-03-16T09:12:57.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.845+0000 7ffade3ef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad81033b0 0x7ffad819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:57.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.845+0000 7ffade3ef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad81033b0 0x7ffad819f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36778/0 (socket says 172.21.15.112:36778) 2024-03-16T09:12:57.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.845+0000 7ffade3ef640 1 -- 172.21.15.112:0/1141537399 learned_addr learned my addr 172.21.15.112:0/1141537399 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:12:57.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.845+0000 7ffadd3ed640 1 --2- 172.21.15.112:0/1141537399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffad81029f0 0x7ffad81980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:57.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.845+0000 7ffaddbee640 1 --2- 172.21.15.112:0/1141537399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffad8101800 0x7ffad8197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:57.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.845+0000 7ffade3ef640 1 -- 172.21.15.112:0/1141537399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffad81029f0 msgr2=0x7ffad81980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.845+0000 7ffade3ef640 1 --2- 172.21.15.112:0/1141537399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffad81029f0 0x7ffad81980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.845+0000 7ffade3ef640 1 -- 172.21.15.112:0/1141537399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffad8101800 msgr2=0x7ffad8197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:57.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.845+0000 7ffade3ef640 1 --2- 172.21.15.112:0/1141537399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffad8101800 0x7ffad8197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:57.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.846+0000 7ffade3ef640 1 -- 172.21.15.112:0/1141537399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad819f790 con 0x7ffad81033b0 2024-03-16T09:12:57.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.846+0000 7ffade3ef640 1 --2- 172.21.15.112:0/1141537399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad81033b0 0x7ffad819f120 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffad002c700 tx=0x7ffad00346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:57.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.846+0000 7ffaceffd640 1 -- 172.21.15.112:0/1141537399 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad0039040 con 0x7ffad81033b0 2024-03-16T09:12:57.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.846+0000 7ffadfe52640 1 -- 172.21.15.112:0/1141537399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad819fa20 con 0x7ffad81033b0 2024-03-16T09:12:57.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.846+0000 7ffaceffd640 1 -- 172.21.15.112:0/1141537399 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ffad0044070 con 0x7ffad81033b0 2024-03-16T09:12:57.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.846+0000 7ffaceffd640 1 -- 172.21.15.112:0/1141537399 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad00043c0 con 0x7ffad81033b0 2024-03-16T09:12:57.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.846+0000 7ffadfe52640 1 -- 172.21.15.112:0/1141537399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad819ff60 con 0x7ffad81033b0 2024-03-16T09:12:57.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.847+0000 7ffadfe52640 1 -- 172.21.15.112:0/1141537399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad8108a50 con 0x7ffad81033b0 2024-03-16T09:12:57.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.848+0000 7ffaceffd640 1 -- 172.21.15.112:0/1141537399 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7ffad0004580 con 0x7ffad81033b0 2024-03-16T09:12:57.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.848+0000 7ffaceffd640 1 --2- 172.21.15.112:0/1141537399 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ffac40a2800 0x7ffac40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:12:57.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.848+0000 7ffaddbee640 1 --2- 172.21.15.112:0/1141537399 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ffac40a2800 0x7ffac40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:12:57.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.848+0000 7ffaceffd640 1 -- 172.21.15.112:0/1141537399 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ffad00f0ba0 con 0x7ffad81033b0 2024-03-16T09:12:57.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.849+0000 7ffaddbee640 1 --2- 172.21.15.112:0/1141537399 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ffac40a2800 0x7ffac40a4cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffad8102850 tx=0x7ffad4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:12:57.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:57.853+0000 7ffaceffd640 1 -- 172.21.15.112:0/1141537399 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ffad00be0e0 con 0x7ffad81033b0 2024-03-16T09:12:58.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:58.065+0000 7ffadfe52640 1 -- 172.21.15.112:0/1141537399 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffad802d590 con 0x7ffad81033b0 2024-03-16T09:12:58.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:58.065+0000 7ffaceffd640 1 -- 172.21.15.112:0/1141537399 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7ffad802d590 con 0x7ffad81033b0 2024-03-16T09:12:58.067 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:12:58.067 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-16T09:12:58.068 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 1 2024-03-16T09:12:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:58.067+0000 7ffadfe52640 1 -- 172.21.15.112:0/1141537399 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ffac40a2800 msgr2=0x7ffac40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:58.067+0000 7ffadfe52640 1 --2- 172.21.15.112:0/1141537399 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ffac40a2800 0x7ffac40a4cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffad8102850 tx=0x7ffad4005f90 comp rx=0 tx=0).stop 2024-03-16T09:12:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:58.067+0000 7ffadfe52640 1 -- 172.21.15.112:0/1141537399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad81033b0 msgr2=0x7ffad819f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:12:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:58.067+0000 7ffadfe52640 1 --2- 172.21.15.112:0/1141537399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad81033b0 0x7ffad819f120 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffad002c700 tx=0x7ffad00346d0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:58.067+0000 7ffadfe52640 1 -- 172.21.15.112:0/1141537399 shutdown_connections 2024-03-16T09:12:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:58.067+0000 7ffadfe52640 1 --2- 172.21.15.112:0/1141537399 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ffac40a2800 0x7ffac40a4cc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:58.067+0000 7ffadfe52640 1 --2- 172.21.15.112:0/1141537399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad81033b0 0x7ffad819f120 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:58.067+0000 7ffadfe52640 1 --2- 172.21.15.112:0/1141537399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffad81029f0 0x7ffad81980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:58.067+0000 7ffadfe52640 1 --2- 172.21.15.112:0/1141537399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffad8101800 0x7ffad8197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:12:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:58.067+0000 7ffadfe52640 1 -- 172.21.15.112:0/1141537399 >> 172.21.15.112:0/1141537399 conn(0x7ffad80fcd90 msgr2=0x7ffad8105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:12:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:58.068+0000 7ffadfe52640 1 -- 172.21.15.112:0/1141537399 shutdown_connections 2024-03-16T09:12:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:12:58.068+0000 7ffadfe52640 1 -- 172.21.15.112:0/1141537399 wait complete. 2024-03-16T09:12:58.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:58 smithi194 ceph-mon[36729]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:12:58.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:58 smithi194 ceph-mon[36729]: Deploying daemon mds.l on smithi194 2024-03-16T09:12:58.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:58 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/107637789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:12:58.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:58 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/981756846' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:12:58.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:58 smithi112 ceph-mon[32307]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:12:58.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:58 smithi112 ceph-mon[32307]: Deploying daemon mds.l on smithi194 2024-03-16T09:12:58.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:58 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/107637789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:12:58.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:58 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/981756846' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:12:58.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:58 smithi159 ceph-mon[36739]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:12:58.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:58 smithi159 ceph-mon[36739]: Deploying daemon mds.l on smithi194 2024-03-16T09:12:58.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:58 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/107637789' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:12:58.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:58 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/981756846' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:12:59.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:12:59 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1141537399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:12:59.153 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 09:12:59 smithi194 systemd[1]: Starting Ceph mds.l for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:12:59.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:12:59 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1141537399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:12:59.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:12:59 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1141537399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:12:59.557 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 09:12:59 smithi194 podman[67243]: 2024-03-16 09:12:59.178007475 +0000 UTC m=+0.018430868 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:12:59.557 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 09:12:59 smithi194 podman[67243]: 2024-03-16 09:12:59.294469212 +0000 UTC m=+0.134892589 container create acce808a8f08258f56196c61fdcd92e0ff7a159e36a626b39445dfe8631cf5b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-l, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-16T09:12:59.557 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 09:12:59 smithi194 podman[67243]: 2024-03-16 09:12:59.555454696 +0000 UTC m=+0.395878064 container init acce808a8f08258f56196c61fdcd92e0ff7a159e36a626b39445dfe8631cf5b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-l, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-16T09:12:59.902 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 09:12:59 smithi194 podman[67243]: 2024-03-16 09:12:59.558260352 +0000 UTC m=+0.398683724 container start acce808a8f08258f56196c61fdcd92e0ff7a159e36a626b39445dfe8631cf5b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-l, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-03-16T09:12:59.902 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 09:12:59 smithi194 ceph-mds[67263]: -- 172.21.15.194:0/3129096931 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56535b183a40 con 0x56535b156800 2024-03-16T09:12:59.902 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 09:12:59 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-l[67259]: starting mds.l at 2024-03-16T09:12:59.902 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 09:12:59 smithi194 bash[67243]: acce808a8f08258f56196c61fdcd92e0ff7a159e36a626b39445dfe8631cf5b2 2024-03-16T09:12:59.902 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 09:12:59 smithi194 systemd[1]: Started Ceph mds.l for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:13:00.081 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:13:00.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.199+0000 7f7f58162640 1 -- 172.21.15.112:0/590398380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f501022a0 msgr2=0x7f7f50108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:00.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.199+0000 7f7f58162640 1 --2- 172.21.15.112:0/590398380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f501022a0 0x7f7f50108b30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009ca0 tx=0x7f7f4802c0f0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.199+0000 7f7f58162640 1 -- 172.21.15.112:0/590398380 shutdown_connections 2024-03-16T09:13:00.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.199+0000 7f7f58162640 1 --2- 172.21.15.112:0/590398380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f501022a0 0x7f7f50108b30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.199+0000 7f7f58162640 1 --2- 172.21.15.112:0/590398380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f501018e0 0x7f7f50101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.199+0000 7f7f58162640 1 --2- 172.21.15.112:0/590398380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f501006f0 0x7f7f50100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.199+0000 7f7f58162640 1 -- 172.21.15.112:0/590398380 >> 172.21.15.112:0/590398380 conn(0x7f7f500fbc60 msgr2=0x7f7f500fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:00.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.199+0000 7f7f58162640 1 -- 172.21.15.112:0/590398380 shutdown_connections 2024-03-16T09:13:00.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.199+0000 7f7f58162640 1 -- 172.21.15.112:0/590398380 wait complete. 2024-03-16T09:13:00.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f58162640 1 Processor -- start 2024-03-16T09:13:00.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f58162640 1 -- start start 2024-03-16T09:13:00.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f58162640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f501006f0 0x7f7f50197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:00.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f58162640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f501018e0 0x7f7f50198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:00.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f58162640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f501022a0 0x7f7f5019f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:00.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f58162640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5010d5b0 con 0x7f7f501006f0 2024-03-16T09:13:00.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f58162640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5010d430 con 0x7f7f501018e0 2024-03-16T09:13:00.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f58162640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5010d730 con 0x7f7f501022a0 2024-03-16T09:13:00.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f55efe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f501006f0 0x7f7f50197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f55efe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f501006f0 0x7f7f50197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36808/0 (socket says 172.21.15.112:36808) 2024-03-16T09:13:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f55efe640 1 -- 172.21.15.112:0/1261157966 learned_addr learned my addr 172.21.15.112:0/1261157966 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f556fd640 1 --2- 172.21.15.112:0/1261157966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f501018e0 0x7f7f50198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f566ff640 1 --2- 172.21.15.112:0/1261157966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f501022a0 0x7f7f5019f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f55efe640 1 -- 172.21.15.112:0/1261157966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f501022a0 msgr2=0x7f7f5019f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f55efe640 1 --2- 172.21.15.112:0/1261157966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f501022a0 0x7f7f5019f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f55efe640 1 -- 172.21.15.112:0/1261157966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f501018e0 msgr2=0x7f7f50198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f55efe640 1 --2- 172.21.15.112:0/1261157966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f501018e0 0x7f7f50198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f55efe640 1 -- 172.21.15.112:0/1261157966 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5019f850 con 0x7f7f501006f0 2024-03-16T09:13:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f55efe640 1 --2- 172.21.15.112:0/1261157966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f501006f0 0x7f7f50197c40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00d950 tx=0x7f7f4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.200+0000 7f7f46ffd640 1 -- 172.21.15.112:0/1261157966 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c017020 con 0x7f7f501006f0 2024-03-16T09:13:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.201+0000 7f7f58162640 1 -- 172.21.15.112:0/1261157966 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f5019fb40 con 0x7f7f501006f0 2024-03-16T09:13:00.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.201+0000 7f7f46ffd640 1 -- 172.21.15.112:0/1261157966 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7f4c009c30 con 0x7f7f501006f0 2024-03-16T09:13:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.201+0000 7f7f46ffd640 1 -- 172.21.15.112:0/1261157966 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c00f7e0 con 0x7f7f501006f0 2024-03-16T09:13:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.201+0000 7f7f58162640 1 -- 172.21.15.112:0/1261157966 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f501a0090 con 0x7f7f501006f0 2024-03-16T09:13:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.202+0000 7f7f58162640 1 -- 172.21.15.112:0/1261157966 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f50107940 con 0x7f7f501006f0 2024-03-16T09:13:00.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.202+0000 7f7f46ffd640 1 -- 172.21.15.112:0/1261157966 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f7f4c01a020 con 0x7f7f501006f0 2024-03-16T09:13:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.203+0000 7f7f46ffd640 1 --2- 172.21.15.112:0/1261157966 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7f3c0a2800 0x7f7f3c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.203+0000 7f7f556fd640 1 --2- 172.21.15.112:0/1261157966 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7f3c0a2800 0x7f7f3c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.203+0000 7f7f46ffd640 1 -- 172.21.15.112:0/1261157966 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f7f4c0cdf60 con 0x7f7f501006f0 2024-03-16T09:13:00.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.203+0000 7f7f556fd640 1 --2- 172.21.15.112:0/1261157966 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7f3c0a2800 0x7f7f3c0a4cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7f40005fd0 tx=0x7f7f4000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:00.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.207+0000 7f7f46ffd640 1 -- 172.21.15.112:0/1261157966 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7f4c021070 con 0x7f7f501006f0 2024-03-16T09:13:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:00 smithi112 ceph-mon[32307]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:00 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.194:6832/1514670407,v1:172.21.15.194:6833/1514670407] up:boot 2024-03-16T09:13:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:00 smithi112 ceph-mon[32307]: fsmap 1 up:standby 2024-03-16T09:13:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:00 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-16T09:13:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:00 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:00 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:00 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:00 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:00 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:00 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:00.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:00 smithi194 ceph-mon[36729]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:00.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:00 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.194:6832/1514670407,v1:172.21.15.194:6833/1514670407] up:boot 2024-03-16T09:13:00.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:00 smithi194 ceph-mon[36729]: fsmap 1 up:standby 2024-03-16T09:13:00.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:00 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-16T09:13:00.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:00 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:00.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:00 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:00.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:00 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:00.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:00 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:00.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:00 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:00.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:00 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:00.415 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:00 smithi159 ceph-mon[36739]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:00.416 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:00 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.194:6832/1514670407,v1:172.21.15.194:6833/1514670407] up:boot 2024-03-16T09:13:00.416 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:00 smithi159 ceph-mon[36739]: fsmap 1 up:standby 2024-03-16T09:13:00.416 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:00 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-16T09:13:00.416 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:00 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:00.416 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:00 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:00.416 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:00 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:00.416 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:00 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:00.416 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:00 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:00.416 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:00 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:00.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.422+0000 7f7f58162640 1 -- 172.21.15.112:0/1261157966 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f5002d1b0 con 0x7f7f501006f0 2024-03-16T09:13:00.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.422+0000 7f7f46ffd640 1 -- 172.21.15.112:0/1261157966 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f7f4c009de0 con 0x7f7f501006f0 2024-03-16T09:13:00.424 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:00.424 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-16T09:13:00.425 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 2 2024-03-16T09:13:00.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.424+0000 7f7f58162640 1 -- 172.21.15.112:0/1261157966 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7f3c0a2800 msgr2=0x7f7f3c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.424+0000 7f7f58162640 1 --2- 172.21.15.112:0/1261157966 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7f3c0a2800 0x7f7f3c0a4cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7f40005fd0 tx=0x7f7f4000b390 comp rx=0 tx=0).stop 2024-03-16T09:13:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.424+0000 7f7f58162640 1 -- 172.21.15.112:0/1261157966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f501006f0 msgr2=0x7f7f50197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.424+0000 7f7f58162640 1 --2- 172.21.15.112:0/1261157966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f501006f0 0x7f7f50197c40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00d950 tx=0x7f7f4c00de10 comp rx=0 tx=0).stop 2024-03-16T09:13:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.425+0000 7f7f58162640 1 -- 172.21.15.112:0/1261157966 shutdown_connections 2024-03-16T09:13:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.425+0000 7f7f58162640 1 --2- 172.21.15.112:0/1261157966 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7f3c0a2800 0x7f7f3c0a4cc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.425+0000 7f7f58162640 1 --2- 172.21.15.112:0/1261157966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f501022a0 0x7f7f5019f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.425+0000 7f7f58162640 1 --2- 172.21.15.112:0/1261157966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f501018e0 0x7f7f50198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.425+0000 7f7f58162640 1 --2- 172.21.15.112:0/1261157966 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f501006f0 0x7f7f50197c40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.425+0000 7f7f58162640 1 -- 172.21.15.112:0/1261157966 >> 172.21.15.112:0/1261157966 conn(0x7f7f500fbc60 msgr2=0x7f7f50104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.425+0000 7f7f58162640 1 -- 172.21.15.112:0/1261157966 shutdown_connections 2024-03-16T09:13:00.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:00.425+0000 7f7f58162640 1 -- 172.21.15.112:0/1261157966 wait complete. 2024-03-16T09:13:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:01 smithi112 ceph-mon[32307]: Deploying daemon mds.k on smithi159 2024-03-16T09:13:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:01 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1261157966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:01.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:01 smithi194 ceph-mon[36729]: Deploying daemon mds.k on smithi159 2024-03-16T09:13:01.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:01 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1261157966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:01.456 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:01 smithi159 ceph-mon[36739]: Deploying daemon mds.k on smithi159 2024-03-16T09:13:01.456 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:01 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1261157966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:02.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:02 smithi159 ceph-mon[36739]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:02.237 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 16 09:13:02 smithi159 systemd[1]: Starting Ceph mds.k for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:13:02.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:02 smithi112 ceph-mon[32307]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:02.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:02 smithi194 ceph-mon[36729]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:02.439 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:13:02.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.550+0000 7fc98b236640 1 -- 172.21.15.112:0/4134959888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102a60 msgr2=0x7fc984104ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:02.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.550+0000 7fc98b236640 1 --2- 172.21.15.112:0/4134959888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102a60 0x7fc984104ed0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc970009900 tx=0x7fc97002bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:02.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.550+0000 7fc98b236640 1 -- 172.21.15.112:0/4134959888 shutdown_connections 2024-03-16T09:13:02.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.550+0000 7fc98b236640 1 --2- 172.21.15.112:0/4134959888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc984105410 0x7fc984105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.550+0000 7fc98b236640 1 --2- 172.21.15.112:0/4134959888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc984102a60 0x7fc984104ed0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.550+0000 7fc98b236640 1 --2- 172.21.15.112:0/4134959888 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9841000f0 0x7fc984102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.550+0000 7fc98b236640 1 -- 172.21.15.112:0/4134959888 >> 172.21.15.112:0/4134959888 conn(0x7fc9840fbd10 msgr2=0x7fc9840fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:02.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.550+0000 7fc98b236640 1 -- 172.21.15.112:0/4134959888 shutdown_connections 2024-03-16T09:13:02.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.550+0000 7fc98b236640 1 -- 172.21.15.112:0/4134959888 wait complete. 2024-03-16T09:13:02.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.551+0000 7fc98b236640 1 Processor -- start 2024-03-16T09:13:02.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.551+0000 7fc98b236640 1 -- start start 2024-03-16T09:13:02.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.551+0000 7fc98b236640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9841000f0 0x7fc98419e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:02.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.551+0000 7fc98b236640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc984102a60 0x7fc98419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:02.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.551+0000 7fc98b236640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc984105410 0x7fc9841a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:02.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.551+0000 7fc98b236640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc984113b70 con 0x7fc9841000f0 2024-03-16T09:13:02.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.551+0000 7fc98b236640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841139f0 con 0x7fc984102a60 2024-03-16T09:13:02.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.551+0000 7fc98b236640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc984113cf0 con 0x7fc984105410 2024-03-16T09:13:02.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.551+0000 7fc988fd2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9841000f0 0x7fc98419e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:02.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.551+0000 7fc988fd2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9841000f0 0x7fc98419e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52132/0 (socket says 172.21.15.112:52132) 2024-03-16T09:13:02.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.551+0000 7fc988fd2640 1 -- 172.21.15.112:0/743229824 learned_addr learned my addr 172.21.15.112:0/743229824 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:02.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.551+0000 7fc983fff640 1 --2- 172.21.15.112:0/743229824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc984102a60 0x7fc98419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:02.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.551+0000 7fc9897d3640 1 --2- 172.21.15.112:0/743229824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc984105410 0x7fc9841a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:02.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.551+0000 7fc988fd2640 1 -- 172.21.15.112:0/743229824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc984105410 msgr2=0x7fc9841a5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:02.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.551+0000 7fc988fd2640 1 --2- 172.21.15.112:0/743229824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc984105410 0x7fc9841a5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.551+0000 7fc988fd2640 1 -- 172.21.15.112:0/743229824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc984102a60 msgr2=0x7fc98419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:02.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.551+0000 7fc988fd2640 1 --2- 172.21.15.112:0/743229824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc984102a60 0x7fc98419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.551+0000 7fc988fd2640 1 -- 172.21.15.112:0/743229824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9841a5e00 con 0x7fc9841000f0 2024-03-16T09:13:02.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.551+0000 7fc988fd2640 1 --2- 172.21.15.112:0/743229824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9841000f0 0x7fc98419e1e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc97800c970 tx=0x7fc97800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:02.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.552+0000 7fc981ffb640 1 -- 172.21.15.112:0/743229824 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978007bb0 con 0x7fc9841000f0 2024-03-16T09:13:02.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.552+0000 7fc981ffb640 1 -- 172.21.15.112:0/743229824 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc978007d50 con 0x7fc9841000f0 2024-03-16T09:13:02.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.552+0000 7fc981ffb640 1 -- 172.21.15.112:0/743229824 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97800f4b0 con 0x7fc9841000f0 2024-03-16T09:13:02.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.552+0000 7fc98b236640 1 -- 172.21.15.112:0/743229824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9841a60f0 con 0x7fc9841000f0 2024-03-16T09:13:02.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.552+0000 7fc98b236640 1 -- 172.21.15.112:0/743229824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9841a65c0 con 0x7fc9841000f0 2024-03-16T09:13:02.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.553+0000 7fc98b236640 1 -- 172.21.15.112:0/743229824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9840fd800 con 0x7fc9841000f0 2024-03-16T09:13:02.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.554+0000 7fc981ffb640 1 -- 172.21.15.112:0/743229824 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fc9780046c0 con 0x7fc9841000f0 2024-03-16T09:13:02.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.554+0000 7fc981ffb640 1 --2- 172.21.15.112:0/743229824 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc9740a2800 0x7fc9740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:02.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.554+0000 7fc983fff640 1 --2- 172.21.15.112:0/743229824 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc9740a2800 0x7fc9740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:02.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.554+0000 7fc981ffb640 1 -- 172.21.15.112:0/743229824 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc9780ce390 con 0x7fc9841000f0 2024-03-16T09:13:02.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.555+0000 7fc983fff640 1 --2- 172.21.15.112:0/743229824 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc9740a2800 0x7fc9740a4cc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc97002c900 tx=0x7fc970005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:02.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.559+0000 7fc981ffb640 1 -- 172.21.15.112:0/743229824 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc978014030 con 0x7fc9841000f0 2024-03-16T09:13:02.567 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 16 09:13:02 smithi159 podman[65547]: 2024-03-16 09:13:02.217237281 +0000 UTC m=+0.017411506 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:13:02.567 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 16 09:13:02 smithi159 podman[65547]: 2024-03-16 09:13:02.320646589 +0000 UTC m=+0.120820803 container create c4afbd131895d0a28145873a7f4837ab559b7400de202e222f80c28912fdae68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-k, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-16T09:13:02.567 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 16 09:13:02 smithi159 podman[65547]: 2024-03-16 09:13:02.540196213 +0000 UTC m=+0.340370437 container init c4afbd131895d0a28145873a7f4837ab559b7400de202e222f80c28912fdae68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-k, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:13:02.567 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 16 09:13:02 smithi159 podman[65547]: 2024-03-16 09:13:02.542890745 +0000 UTC m=+0.343064959 container start c4afbd131895d0a28145873a7f4837ab559b7400de202e222f80c28912fdae68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-k, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-16T09:13:02.568 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 16 09:13:02 smithi159 ceph-mds[65566]: -- 172.21.15.159:0/2664989078 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562b85909a40 con 0x562b858dcc00 2024-03-16T09:13:02.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.774+0000 7fc98b236640 1 -- 172.21.15.112:0/743229824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc98402d1b0 con 0x7fc9841000f0 2024-03-16T09:13:02.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.774+0000 7fc981ffb640 1 -- 172.21.15.112:0/743229824 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fc97809b8d0 con 0x7fc9841000f0 2024-03-16T09:13:02.776 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:02.776 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-16T09:13:02.777 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 2 2024-03-16T09:13:02.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.776+0000 7fc98b236640 1 -- 172.21.15.112:0/743229824 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc9740a2800 msgr2=0x7fc9740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:02.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.776+0000 7fc98b236640 1 --2- 172.21.15.112:0/743229824 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc9740a2800 0x7fc9740a4cc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc97002c900 tx=0x7fc970005fd0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.776+0000 7fc98b236640 1 -- 172.21.15.112:0/743229824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9841000f0 msgr2=0x7fc98419e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:02.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.776+0000 7fc98b236640 1 --2- 172.21.15.112:0/743229824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9841000f0 0x7fc98419e1e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc97800c970 tx=0x7fc97800ce30 comp rx=0 tx=0).stop 2024-03-16T09:13:02.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.776+0000 7fc98b236640 1 -- 172.21.15.112:0/743229824 shutdown_connections 2024-03-16T09:13:02.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.776+0000 7fc98b236640 1 --2- 172.21.15.112:0/743229824 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc9740a2800 0x7fc9740a4cc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.776+0000 7fc98b236640 1 --2- 172.21.15.112:0/743229824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc984105410 0x7fc9841a5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.776+0000 7fc98b236640 1 --2- 172.21.15.112:0/743229824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc984102a60 0x7fc98419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.776+0000 7fc98b236640 1 --2- 172.21.15.112:0/743229824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9841000f0 0x7fc98419e1e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:02.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.776+0000 7fc98b236640 1 -- 172.21.15.112:0/743229824 >> 172.21.15.112:0/743229824 conn(0x7fc9840fbd10 msgr2=0x7fc984100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:02.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.776+0000 7fc98b236640 1 -- 172.21.15.112:0/743229824 shutdown_connections 2024-03-16T09:13:02.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:02.776+0000 7fc98b236640 1 -- 172.21.15.112:0/743229824 wait complete. 2024-03-16T09:13:02.986 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 16 09:13:02 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-k[65562]: starting mds.k at 2024-03-16T09:13:02.986 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 16 09:13:02 smithi159 bash[65547]: c4afbd131895d0a28145873a7f4837ab559b7400de202e222f80c28912fdae68 2024-03-16T09:13:02.986 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 16 09:13:02 smithi159 systemd[1]: Started Ceph mds.k for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:13:03.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:03 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:03.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:03 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:03.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:03 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:03.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:03 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:03.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:03 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/743229824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:03.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:03 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:03.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:03 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:03.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:03 smithi159 ceph-mon[36739]: Deploying daemon mds.j on smithi112 2024-03-16T09:13:03.988 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:03 smithi159 ceph-mon[36739]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:03.988 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:03 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.159:6832/2947166433,v1:172.21.15.159:6833/2947166433] up:boot 2024-03-16T09:13:03.988 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:03 smithi159 ceph-mon[36739]: fsmap 2 up:standby 2024-03-16T09:13:03.988 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:03 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-16T09:13:04.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:03 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:04.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:03 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:04.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:03 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:04.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:03 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:04.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:03 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/743229824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:04.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:03 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:04.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:03 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:04.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:03 smithi112 ceph-mon[32307]: Deploying daemon mds.j on smithi112 2024-03-16T09:13:04.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:03 smithi112 ceph-mon[32307]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:04.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:03 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.159:6832/2947166433,v1:172.21.15.159:6833/2947166433] up:boot 2024-03-16T09:13:04.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:03 smithi112 ceph-mon[32307]: fsmap 2 up:standby 2024-03-16T09:13:04.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:03 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-16T09:13:04.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:03 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:04.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:03 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:04.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:03 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:04.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:03 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:04.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:03 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/743229824' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:04.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:03 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:04.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:03 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:04.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:03 smithi194 ceph-mon[36729]: Deploying daemon mds.j on smithi112 2024-03-16T09:13:04.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:03 smithi194 ceph-mon[36729]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:04.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:03 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.159:6832/2947166433,v1:172.21.15.159:6833/2947166433] up:boot 2024-03-16T09:13:04.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:03 smithi194 ceph-mon[36729]: fsmap 2 up:standby 2024-03-16T09:13:04.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:03 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-16T09:13:04.790 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:13:04.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.963+0000 7f8364636640 1 -- 172.21.15.112:0/449740141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f835c1022c0 msgr2=0x7f835c1046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:04.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.963+0000 7f8364636640 1 --2- 172.21.15.112:0/449740141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f835c1022c0 0x7f835c1046b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8358009900 tx=0x7f835802bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:04.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.963+0000 7f8364636640 1 -- 172.21.15.112:0/449740141 shutdown_connections 2024-03-16T09:13:04.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.963+0000 7f8364636640 1 --2- 172.21.15.112:0/449740141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f835c107520 0x7f835c109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.963+0000 7f8364636640 1 --2- 172.21.15.112:0/449740141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f835c104bf0 0x7f835c106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.963+0000 7f8364636640 1 --2- 172.21.15.112:0/449740141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f835c1022c0 0x7f835c1046b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.963+0000 7f8364636640 1 -- 172.21.15.112:0/449740141 >> 172.21.15.112:0/449740141 conn(0x7f835c0fbc70 msgr2=0x7f835c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:04.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.963+0000 7f8364636640 1 -- 172.21.15.112:0/449740141 shutdown_connections 2024-03-16T09:13:04.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.963+0000 7f8364636640 1 -- 172.21.15.112:0/449740141 wait complete. 2024-03-16T09:13:04.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.964+0000 7f8364636640 1 Processor -- start 2024-03-16T09:13:04.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.964+0000 7f8364636640 1 -- start start 2024-03-16T09:13:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.964+0000 7f8364636640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f835c1022c0 0x7f835c195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.964+0000 7f8364636640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f835c104bf0 0x7f835c196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.964+0000 7f8364636640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f835c107520 0x7f835c19d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.964+0000 7f8364636640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f835c10d5e0 con 0x7f835c107520 2024-03-16T09:13:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.964+0000 7f8364636640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f835c10d460 con 0x7f835c1022c0 2024-03-16T09:13:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.964+0000 7f8364636640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f835c10d760 con 0x7f835c104bf0 2024-03-16T09:13:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.964+0000 7f8362bd3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f835c107520 0x7f835c19d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.964+0000 7f8362bd3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f835c107520 0x7f835c19d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52144/0 (socket says 172.21.15.112:52144) 2024-03-16T09:13:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.964+0000 7f8362bd3640 1 -- 172.21.15.112:0/3983812851 learned_addr learned my addr 172.21.15.112:0/3983812851 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.964+0000 7f8361bd1640 1 --2- 172.21.15.112:0/3983812851 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f835c104bf0 0x7f835c196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.964+0000 7f83623d2640 1 --2- 172.21.15.112:0/3983812851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f835c1022c0 0x7f835c195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.964+0000 7f8362bd3640 1 -- 172.21.15.112:0/3983812851 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f835c104bf0 msgr2=0x7f835c196010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.964+0000 7f8362bd3640 1 --2- 172.21.15.112:0/3983812851 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f835c104bf0 0x7f835c196010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.964+0000 7f8362bd3640 1 -- 172.21.15.112:0/3983812851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f835c1022c0 msgr2=0x7f835c195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:04.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.964+0000 7f8362bd3640 1 --2- 172.21.15.112:0/3983812851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f835c1022c0 0x7f835c195ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.964+0000 7f8362bd3640 1 -- 172.21.15.112:0/3983812851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c19d6f0 con 0x7f835c107520 2024-03-16T09:13:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.965+0000 7f8362bd3640 1 --2- 172.21.15.112:0/3983812851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f835c107520 0x7f835c19d080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8354009fd0 tx=0x7f835400def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.965+0000 7f83537fe640 1 -- 172.21.15.112:0/3983812851 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354017070 con 0x7f835c107520 2024-03-16T09:13:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.965+0000 7f83537fe640 1 -- 172.21.15.112:0/3983812851 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f835400bdd0 con 0x7f835c107520 2024-03-16T09:13:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.965+0000 7f8364636640 1 -- 172.21.15.112:0/3983812851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835c19d9e0 con 0x7f835c107520 2024-03-16T09:13:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.965+0000 7f83537fe640 1 -- 172.21.15.112:0/3983812851 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83540106c0 con 0x7f835c107520 2024-03-16T09:13:04.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.965+0000 7f8364636640 1 -- 172.21.15.112:0/3983812851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f835c19df30 con 0x7f835c107520 2024-03-16T09:13:04.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.966+0000 7f8364636640 1 -- 172.21.15.112:0/3983812851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f835c02d1b0 con 0x7f835c107520 2024-03-16T09:13:04.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.967+0000 7f83537fe640 1 -- 172.21.15.112:0/3983812851 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f8354003970 con 0x7f835c107520 2024-03-16T09:13:04.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.968+0000 7f83537fe640 1 --2- 172.21.15.112:0/3983812851 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f83480a2880 0x7f83480a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:04.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.968+0000 7f83623d2640 1 --2- 172.21.15.112:0/3983812851 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f83480a2880 0x7f83480a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:04.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.968+0000 7f83537fe640 1 -- 172.21.15.112:0/3983812851 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f83540ce220 con 0x7f835c107520 2024-03-16T09:13:04.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.968+0000 7f83623d2640 1 --2- 172.21.15.112:0/3983812851 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f83480a2880 0x7f83480a4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f835802c900 tx=0x7f8358005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:04.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:04.973+0000 7f83537fe640 1 -- 172.21.15.112:0/3983812851 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f835409b760 con 0x7f835c107520 2024-03-16T09:13:05.092 INFO:journalctl@ceph.mds.j.smithi112.stdout:Mar 16 09:13:04 smithi112 systemd[1]: Starting Ceph mds.j for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:13:05.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:05.195+0000 7f8364636640 1 -- 172.21.15.112:0/3983812851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f835c108bc0 con 0x7f835c107520 2024-03-16T09:13:05.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:05.196+0000 7f83537fe640 1 -- 172.21.15.112:0/3983812851 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f835409b060 con 0x7f835c107520 2024-03-16T09:13:05.198 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:05.198 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-16T09:13:05.198 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 3 2024-03-16T09:13:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:05.198+0000 7f8364636640 1 -- 172.21.15.112:0/3983812851 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f83480a2880 msgr2=0x7f83480a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:05.198+0000 7f8364636640 1 --2- 172.21.15.112:0/3983812851 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f83480a2880 0x7f83480a4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f835802c900 tx=0x7f8358005fd0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:05.198+0000 7f8364636640 1 -- 172.21.15.112:0/3983812851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f835c107520 msgr2=0x7f835c19d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:05.198+0000 7f8364636640 1 --2- 172.21.15.112:0/3983812851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f835c107520 0x7f835c19d080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8354009fd0 tx=0x7f835400def0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:05.198+0000 7f8364636640 1 -- 172.21.15.112:0/3983812851 shutdown_connections 2024-03-16T09:13:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:05.198+0000 7f8364636640 1 --2- 172.21.15.112:0/3983812851 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f83480a2880 0x7f83480a4d40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:05.198+0000 7f8364636640 1 --2- 172.21.15.112:0/3983812851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f835c107520 0x7f835c19d080 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:05.198+0000 7f8364636640 1 --2- 172.21.15.112:0/3983812851 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f835c104bf0 0x7f835c196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:05.198+0000 7f8364636640 1 --2- 172.21.15.112:0/3983812851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f835c1022c0 0x7f835c195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:05.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:05.198+0000 7f8364636640 1 -- 172.21.15.112:0/3983812851 >> 172.21.15.112:0/3983812851 conn(0x7f835c0fbc70 msgr2=0x7f835c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:05.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:05.198+0000 7f8364636640 1 -- 172.21.15.112:0/3983812851 shutdown_connections 2024-03-16T09:13:05.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:05.198+0000 7f8364636640 1 -- 172.21.15.112:0/3983812851 wait complete. 2024-03-16T09:13:05.369 INFO:journalctl@ceph.mds.j.smithi112.stdout:Mar 16 09:13:05 smithi112 podman[72805]: 2024-03-16 09:13:05.018553509 +0000 UTC m=+0.018340621 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:13:05.369 INFO:journalctl@ceph.mds.j.smithi112.stdout:Mar 16 09:13:05 smithi112 podman[72805]: 2024-03-16 09:13:05.131615556 +0000 UTC m=+0.131402650 container create ed78459aba8d1370934756262d67207e8c3d631996620a3b38bddb406060e0d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-j, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:13:05.369 INFO:journalctl@ceph.mds.j.smithi112.stdout:Mar 16 09:13:05 smithi112 podman[72805]: 2024-03-16 09:13:05.367893172 +0000 UTC m=+0.367680266 container init ed78459aba8d1370934756262d67207e8c3d631996620a3b38bddb406060e0d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-j, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, ceph=True) 2024-03-16T09:13:05.842 INFO:journalctl@ceph.mds.j.smithi112.stdout:Mar 16 09:13:05 smithi112 podman[72805]: 2024-03-16 09:13:05.370710884 +0000 UTC m=+0.370497987 container start ed78459aba8d1370934756262d67207e8c3d631996620a3b38bddb406060e0d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-j, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-16T09:13:05.842 INFO:journalctl@ceph.mds.j.smithi112.stdout:Mar 16 09:13:05 smithi112 ceph-mds[72826]: -- 172.21.15.112:0/3112454331 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eab5c3ba40 con 0x55eab5c0e800 2024-03-16T09:13:05.842 INFO:journalctl@ceph.mds.j.smithi112.stdout:Mar 16 09:13:05 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-j[72822]: starting mds.j at 2024-03-16T09:13:05.842 INFO:journalctl@ceph.mds.j.smithi112.stdout:Mar 16 09:13:05 smithi112 bash[72805]: ed78459aba8d1370934756262d67207e8c3d631996620a3b38bddb406060e0d0 2024-03-16T09:13:05.842 INFO:journalctl@ceph.mds.j.smithi112.stdout:Mar 16 09:13:05 smithi112 systemd[1]: Started Ceph mds.j for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:13:06.187 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:05 smithi112 ceph-mon[32307]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:06.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:05 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3983812851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:06.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:06.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:06.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:06.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:06.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:06.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:05 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:06.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:05 smithi159 ceph-mon[36739]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:06.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:05 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3983812851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:06.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:06.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:06.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:06.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:06.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:06.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:05 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:06.333 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:05 smithi194 ceph-mon[36729]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:06.333 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:05 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3983812851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:06.334 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:06.334 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:06.334 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:06.334 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:06.334 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:06.334 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:05 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:07.211 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:13:07.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:06 smithi159 ceph-mon[36739]: Deploying daemon mds.d on smithi112 2024-03-16T09:13:07.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:06 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.112:6834/1370033819,v1:172.21.15.112:6835/1370033819] up:boot 2024-03-16T09:13:07.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:06 smithi159 ceph-mon[36739]: fsmap 3 up:standby 2024-03-16T09:13:07.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:06 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-16T09:13:07.242 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:06 smithi112 ceph-mon[32307]: Deploying daemon mds.d on smithi112 2024-03-16T09:13:07.242 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:06 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.112:6834/1370033819,v1:172.21.15.112:6835/1370033819] up:boot 2024-03-16T09:13:07.242 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:06 smithi112 ceph-mon[32307]: fsmap 3 up:standby 2024-03-16T09:13:07.242 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:06 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-16T09:13:07.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.329+0000 7f20ec663640 1 -- 172.21.15.112:0/1259563939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20e4107540 msgr2=0x7f20e4109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.329+0000 7f20ec663640 1 --2- 172.21.15.112:0/1259563939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20e4107540 0x7f20e4109930 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009ca0 tx=0x7f20dc02c0f0 comp rx=0 tx=0).stop 2024-03-16T09:13:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.329+0000 7f20ec663640 1 -- 172.21.15.112:0/1259563939 shutdown_connections 2024-03-16T09:13:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.329+0000 7f20ec663640 1 --2- 172.21.15.112:0/1259563939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20e4107540 0x7f20e4109930 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.329+0000 7f20ec663640 1 --2- 172.21.15.112:0/1259563939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20e4104c10 0x7f20e4107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.329+0000 7f20ec663640 1 --2- 172.21.15.112:0/1259563939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20e41022e0 0x7f20e41046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.329+0000 7f20ec663640 1 -- 172.21.15.112:0/1259563939 >> 172.21.15.112:0/1259563939 conn(0x7f20e40fbc70 msgr2=0x7f20e40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.329+0000 7f20ec663640 1 -- 172.21.15.112:0/1259563939 shutdown_connections 2024-03-16T09:13:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.329+0000 7f20ec663640 1 -- 172.21.15.112:0/1259563939 wait complete. 2024-03-16T09:13:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.330+0000 7f20ec663640 1 Processor -- start 2024-03-16T09:13:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.330+0000 7f20ec663640 1 -- start start 2024-03-16T09:13:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.330+0000 7f20ec663640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20e41022e0 0x7f20e4197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:07.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.330+0000 7f20ec663640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20e4104c10 0x7f20e41980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.330+0000 7f20ec663640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20e4107540 0x7f20e419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.330+0000 7f20ec663640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20e410d5b0 con 0x7f20e4104c10 2024-03-16T09:13:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.330+0000 7f20ec663640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f20e410d430 con 0x7f20e4107540 2024-03-16T09:13:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.330+0000 7f20ec663640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f20e410d730 con 0x7f20e41022e0 2024-03-16T09:13:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.330+0000 7f20e9bfe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20e4104c10 0x7f20e41980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.330+0000 7f20e9bfe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20e4104c10 0x7f20e41980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52186/0 (socket says 172.21.15.112:52186) 2024-03-16T09:13:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.330+0000 7f20e9bfe640 1 -- 172.21.15.112:0/3048800250 learned_addr learned my addr 172.21.15.112:0/3048800250 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.330+0000 7f20ea3ff640 1 --2- 172.21.15.112:0/3048800250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20e41022e0 0x7f20e4197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.331+0000 7f20eac00640 1 --2- 172.21.15.112:0/3048800250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20e4107540 0x7f20e419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.331+0000 7f20e9bfe640 1 -- 172.21.15.112:0/3048800250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20e41022e0 msgr2=0x7f20e4197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.331+0000 7f20e9bfe640 1 --2- 172.21.15.112:0/3048800250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20e41022e0 0x7f20e4197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.331+0000 7f20e9bfe640 1 -- 172.21.15.112:0/3048800250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20e4107540 msgr2=0x7f20e419f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.331+0000 7f20e9bfe640 1 --2- 172.21.15.112:0/3048800250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20e4107540 0x7f20e419f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:07.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.331+0000 7f20e9bfe640 1 -- 172.21.15.112:0/3048800250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e419f7b0 con 0x7f20e4104c10 2024-03-16T09:13:07.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.331+0000 7f20ea3ff640 1 --2- 172.21.15.112:0/3048800250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20e41022e0 0x7f20e4197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:13:07.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.331+0000 7f20e9bfe640 1 --2- 172.21.15.112:0/3048800250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20e4104c10 0x7f20e41980d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f20e000d770 tx=0x7f20e000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:07.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.331+0000 7f20d77fe640 1 -- 172.21.15.112:0/3048800250 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0017070 con 0x7f20e4104c10 2024-03-16T09:13:07.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.331+0000 7f20d77fe640 1 -- 172.21.15.112:0/3048800250 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f20e0009c30 con 0x7f20e4104c10 2024-03-16T09:13:07.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.331+0000 7f20ec663640 1 -- 172.21.15.112:0/3048800250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e419faa0 con 0x7f20e4104c10 2024-03-16T09:13:07.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.331+0000 7f20d77fe640 1 -- 172.21.15.112:0/3048800250 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e000f6b0 con 0x7f20e4104c10 2024-03-16T09:13:07.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.331+0000 7f20ec663640 1 -- 172.21.15.112:0/3048800250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e419fff0 con 0x7f20e4104c10 2024-03-16T09:13:07.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.332+0000 7f20ec663640 1 -- 172.21.15.112:0/3048800250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e40fd760 con 0x7f20e4104c10 2024-03-16T09:13:07.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.333+0000 7f20d77fe640 1 -- 172.21.15.112:0/3048800250 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f20e001a070 con 0x7f20e4104c10 2024-03-16T09:13:07.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.333+0000 7f20d77fe640 1 --2- 172.21.15.112:0/3048800250 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f20d00a2800 0x7f20d00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:07.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.334+0000 7f20ea3ff640 1 --2- 172.21.15.112:0/3048800250 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f20d00a2800 0x7f20d00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:07.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.334+0000 7f20d77fe640 1 -- 172.21.15.112:0/3048800250 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f20e00ce170 con 0x7f20e4104c10 2024-03-16T09:13:07.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.334+0000 7f20ea3ff640 1 --2- 172.21.15.112:0/3048800250 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f20d00a2800 0x7f20d00a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f20d80058c0 tx=0x7f20d8005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:07.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.338+0000 7f20d77fe640 1 -- 172.21.15.112:0/3048800250 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f20e0009de0 con 0x7f20e4104c10 2024-03-16T09:13:07.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:06 smithi194 ceph-mon[36729]: Deploying daemon mds.d on smithi112 2024-03-16T09:13:07.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:06 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.112:6834/1370033819,v1:172.21.15.112:6835/1370033819] up:boot 2024-03-16T09:13:07.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:06 smithi194 ceph-mon[36729]: fsmap 3 up:standby 2024-03-16T09:13:07.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:06 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-16T09:13:07.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.588+0000 7f20ec663640 1 -- 172.21.15.112:0/3048800250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20e402d1b0 con 0x7f20e4104c10 2024-03-16T09:13:07.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.588+0000 7f20d77fe640 1 -- 172.21.15.112:0/3048800250 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3052 (secure 0 0 0) 0x7f20e009b6b0 con 0x7f20e4104c10 2024-03-16T09:13:07.590 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:07.590 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-16T09:13:07.591 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 4 2024-03-16T09:13:07.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.590+0000 7f20ec663640 1 -- 172.21.15.112:0/3048800250 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f20d00a2800 msgr2=0x7f20d00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.590+0000 7f20ec663640 1 --2- 172.21.15.112:0/3048800250 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f20d00a2800 0x7f20d00a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f20d80058c0 tx=0x7f20d8005ec0 comp rx=0 tx=0).stop 2024-03-16T09:13:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.590+0000 7f20ec663640 1 -- 172.21.15.112:0/3048800250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20e4104c10 msgr2=0x7f20e41980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.590+0000 7f20ec663640 1 --2- 172.21.15.112:0/3048800250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20e4104c10 0x7f20e41980d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f20e000d770 tx=0x7f20e000dc30 comp rx=0 tx=0).stop 2024-03-16T09:13:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.591+0000 7f20ec663640 1 -- 172.21.15.112:0/3048800250 shutdown_connections 2024-03-16T09:13:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.591+0000 7f20ec663640 1 --2- 172.21.15.112:0/3048800250 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f20d00a2800 0x7f20d00a4cc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.591+0000 7f20ec663640 1 --2- 172.21.15.112:0/3048800250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20e4107540 0x7f20e419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.591+0000 7f20ec663640 1 --2- 172.21.15.112:0/3048800250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20e4104c10 0x7f20e41980d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.591+0000 7f20ec663640 1 --2- 172.21.15.112:0/3048800250 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20e41022e0 0x7f20e4197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.591+0000 7f20ec663640 1 -- 172.21.15.112:0/3048800250 >> 172.21.15.112:0/3048800250 conn(0x7f20e40fbc70 msgr2=0x7f20e4102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.591+0000 7f20ec663640 1 -- 172.21.15.112:0/3048800250 shutdown_connections 2024-03-16T09:13:07.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:07.591+0000 7f20ec663640 1 -- 172.21.15.112:0/3048800250 wait complete. 2024-03-16T09:13:07.842 INFO:journalctl@ceph.mds.d.smithi112.stdout:Mar 16 09:13:07 smithi112 systemd[1]: Starting Ceph mds.d for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:13:08.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:07 smithi159 ceph-mon[36739]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:08.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:07 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3048800250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:08.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:07 smithi112 ceph-mon[32307]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:08.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:07 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3048800250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:08.253 INFO:journalctl@ceph.mds.d.smithi112.stdout:Mar 16 09:13:07 smithi112 podman[73154]: 2024-03-16 09:13:07.866491456 +0000 UTC m=+0.017824889 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:13:08.253 INFO:journalctl@ceph.mds.d.smithi112.stdout:Mar 16 09:13:07 smithi112 podman[73154]: 2024-03-16 09:13:07.998833986 +0000 UTC m=+0.150167408 container create efe476354faaaa9f913e9952b493f7925ef6970ff08f4f2998e6d441ac0dce88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-d, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:13:08.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:07 smithi194 ceph-mon[36729]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:08.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:07 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3048800250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:08.592 INFO:journalctl@ceph.mds.d.smithi112.stdout:Mar 16 09:13:08 smithi112 podman[73154]: 2024-03-16 09:13:08.25191262 +0000 UTC m=+0.403246058 container init efe476354faaaa9f913e9952b493f7925ef6970ff08f4f2998e6d441ac0dce88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-d, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-16T09:13:08.592 INFO:journalctl@ceph.mds.d.smithi112.stdout:Mar 16 09:13:08 smithi112 podman[73154]: 2024-03-16 09:13:08.254814046 +0000 UTC m=+0.406147477 container start efe476354faaaa9f913e9952b493f7925ef6970ff08f4f2998e6d441ac0dce88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-d, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-16T09:13:08.592 INFO:journalctl@ceph.mds.d.smithi112.stdout:Mar 16 09:13:08 smithi112 ceph-mds[73173]: -- 172.21.15.112:0/1716300729 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564184dd7a40 con 0x564184dab000 2024-03-16T09:13:08.592 INFO:journalctl@ceph.mds.d.smithi112.stdout:Mar 16 09:13:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-d[73169]: starting mds.d at 2024-03-16T09:13:08.592 INFO:journalctl@ceph.mds.d.smithi112.stdout:Mar 16 09:13:08 smithi112 bash[73154]: efe476354faaaa9f913e9952b493f7925ef6970ff08f4f2998e6d441ac0dce88 2024-03-16T09:13:08.592 INFO:journalctl@ceph.mds.d.smithi112.stdout:Mar 16 09:13:08 smithi112 systemd[1]: Started Ceph mds.d for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:13:09.605 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:13:09.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.721+0000 7f8a78e0b640 1 -- 172.21.15.112:0/3347650668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a740ffaf0 msgr2=0x7f8a740fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:09.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.721+0000 7f8a78e0b640 1 --2- 172.21.15.112:0/3347650668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a740ffaf0 0x7f8a740fff10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009930 tx=0x7f8a6c02bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:09.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.721+0000 7f8a78e0b640 1 -- 172.21.15.112:0/3347650668 shutdown_connections 2024-03-16T09:13:09.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.721+0000 7f8a78e0b640 1 --2- 172.21.15.112:0/3347650668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a74103480 0x7f8a74109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:09.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.721+0000 7f8a78e0b640 1 --2- 172.21.15.112:0/3347650668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a74102ac0 0x7f8a74102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:09.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.721+0000 7f8a78e0b640 1 --2- 172.21.15.112:0/3347650668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a740ffaf0 0x7f8a740fff10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:09.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.721+0000 7f8a78e0b640 1 -- 172.21.15.112:0/3347650668 >> 172.21.15.112:0/3347650668 conn(0x7f8a740fb420 msgr2=0x7f8a740fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:09.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.721+0000 7f8a78e0b640 1 -- 172.21.15.112:0/3347650668 shutdown_connections 2024-03-16T09:13:09.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.721+0000 7f8a78e0b640 1 -- 172.21.15.112:0/3347650668 wait complete. 2024-03-16T09:13:09.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.722+0000 7f8a78e0b640 1 Processor -- start 2024-03-16T09:13:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.722+0000 7f8a78e0b640 1 -- start start 2024-03-16T09:13:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.722+0000 7f8a78e0b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a740ffaf0 0x7f8a74197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.722+0000 7f8a78e0b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a74102ac0 0x7f8a741981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.722+0000 7f8a78e0b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a74103480 0x7f8a7419f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.722+0000 7f8a78e0b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7410d5b0 con 0x7f8a74102ac0 2024-03-16T09:13:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.722+0000 7f8a78e0b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7410d430 con 0x7f8a740ffaf0 2024-03-16T09:13:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.722+0000 7f8a78e0b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7410d730 con 0x7f8a74103480 2024-03-16T09:13:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.722+0000 7f8a71d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a74102ac0 0x7f8a741981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.722+0000 7f8a71d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a74102ac0 0x7f8a741981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52238/0 (socket says 172.21.15.112:52238) 2024-03-16T09:13:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.722+0000 7f8a71d9b640 1 -- 172.21.15.112:0/2460647771 learned_addr learned my addr 172.21.15.112:0/2460647771 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.722+0000 7f8a7259c640 1 --2- 172.21.15.112:0/2460647771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a740ffaf0 0x7f8a74197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.722+0000 7f8a72d9d640 1 --2- 172.21.15.112:0/2460647771 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a74103480 0x7f8a7419f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:09.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.722+0000 7f8a71d9b640 1 -- 172.21.15.112:0/2460647771 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a74103480 msgr2=0x7f8a7419f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.722+0000 7f8a71d9b640 1 --2- 172.21.15.112:0/2460647771 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a74103480 0x7f8a7419f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.722+0000 7f8a71d9b640 1 -- 172.21.15.112:0/2460647771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a740ffaf0 msgr2=0x7f8a74197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.722+0000 7f8a71d9b640 1 --2- 172.21.15.112:0/2460647771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a740ffaf0 0x7f8a74197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.722+0000 7f8a71d9b640 1 -- 172.21.15.112:0/2460647771 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7419f880 con 0x7f8a74102ac0 2024-03-16T09:13:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.722+0000 7f8a71d9b640 1 --2- 172.21.15.112:0/2460647771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a74102ac0 0x7f8a741981b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00d950 tx=0x7f8a5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.723+0000 7f8a6b7fe640 1 -- 172.21.15.112:0/2460647771 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c017070 con 0x7f8a74102ac0 2024-03-16T09:13:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.723+0000 7f8a6b7fe640 1 -- 172.21.15.112:0/2460647771 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8a5c009c30 con 0x7f8a74102ac0 2024-03-16T09:13:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.723+0000 7f8a78e0b640 1 -- 172.21.15.112:0/2460647771 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a7419fb70 con 0x7f8a74102ac0 2024-03-16T09:13:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.723+0000 7f8a6b7fe640 1 -- 172.21.15.112:0/2460647771 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c00f740 con 0x7f8a74102ac0 2024-03-16T09:13:09.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.723+0000 7f8a78e0b640 1 -- 172.21.15.112:0/2460647771 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a741a0090 con 0x7f8a74102ac0 2024-03-16T09:13:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.724+0000 7f8a78e0b640 1 -- 172.21.15.112:0/2460647771 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a74108bf0 con 0x7f8a74102ac0 2024-03-16T09:13:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.725+0000 7f8a6b7fe640 1 -- 172.21.15.112:0/2460647771 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f8a5c010a80 con 0x7f8a74102ac0 2024-03-16T09:13:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.725+0000 7f8a6b7fe640 1 --2- 172.21.15.112:0/2460647771 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8a600a2800 0x7f8a600a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.726+0000 7f8a7259c640 1 --2- 172.21.15.112:0/2460647771 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8a600a2800 0x7f8a600a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.726+0000 7f8a6b7fe640 1 -- 172.21.15.112:0/2460647771 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8a5c0ce1e0 con 0x7f8a74102ac0 2024-03-16T09:13:09.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.726+0000 7f8a7259c640 1 --2- 172.21.15.112:0/2460647771 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8a600a2800 0x7f8a600a4cc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c02c900 tx=0x7f8a6c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:09.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.730+0000 7f8a6b7fe640 1 -- 172.21.15.112:0/2460647771 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8a5c009de0 con 0x7f8a74102ac0 2024-03-16T09:13:09.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:09 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:09.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:09 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:09.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:09 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:09.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:09 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:09.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:09 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:09.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:09 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:09.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:09 smithi159 ceph-mon[36739]: Deploying daemon mds.e on smithi159 2024-03-16T09:13:09.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:09 smithi159 ceph-mon[36739]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:09.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:09 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.112:6836/1353236253,v1:172.21.15.112:6837/1353236253] up:boot 2024-03-16T09:13:09.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:09 smithi159 ceph-mon[36739]: fsmap 4 up:standby 2024-03-16T09:13:09.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:09 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-16T09:13:09.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:09 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:09.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:09 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:09.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:09 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:09.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:09 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:09.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:09 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:09.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:09 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:09.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:09 smithi112 ceph-mon[32307]: Deploying daemon mds.e on smithi159 2024-03-16T09:13:09.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:09 smithi112 ceph-mon[32307]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:09.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:09 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.112:6836/1353236253,v1:172.21.15.112:6837/1353236253] up:boot 2024-03-16T09:13:09.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:09 smithi112 ceph-mon[32307]: fsmap 4 up:standby 2024-03-16T09:13:09.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:09 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-16T09:13:09.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:09 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:09.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:09 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:09.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:09 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:09.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:09 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:09.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:09 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:09.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:09 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:09.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:09 smithi194 ceph-mon[36729]: Deploying daemon mds.e on smithi159 2024-03-16T09:13:09.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:09 smithi194 ceph-mon[36729]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:09.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:09 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.112:6836/1353236253,v1:172.21.15.112:6837/1353236253] up:boot 2024-03-16T09:13:09.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:09 smithi194 ceph-mon[36729]: fsmap 4 up:standby 2024-03-16T09:13:09.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:09 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-16T09:13:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.941+0000 7f8a78e0b640 1 -- 172.21.15.112:0/2460647771 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a7402d400 con 0x7f8a74102ac0 2024-03-16T09:13:09.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.942+0000 7f8a6b7fe640 1 -- 172.21.15.112:0/2460647771 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3880 (secure 0 0 0) 0x7f8a5c09b0d0 con 0x7f8a74102ac0 2024-03-16T09:13:09.944 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:09.944 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-16T09:13:09.944 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 5 2024-03-16T09:13:09.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.944+0000 7f8a78e0b640 1 -- 172.21.15.112:0/2460647771 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8a600a2800 msgr2=0x7f8a600a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:09.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.944+0000 7f8a78e0b640 1 --2- 172.21.15.112:0/2460647771 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8a600a2800 0x7f8a600a4cc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c02c900 tx=0x7f8a6c005fd0 comp rx=0 tx=0).stop 2024-03-16T09:13:09.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.944+0000 7f8a78e0b640 1 -- 172.21.15.112:0/2460647771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a74102ac0 msgr2=0x7f8a741981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:09.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.944+0000 7f8a78e0b640 1 --2- 172.21.15.112:0/2460647771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a74102ac0 0x7f8a741981b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00d950 tx=0x7f8a5c00de10 comp rx=0 tx=0).stop 2024-03-16T09:13:09.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.944+0000 7f8a78e0b640 1 -- 172.21.15.112:0/2460647771 shutdown_connections 2024-03-16T09:13:09.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.944+0000 7f8a78e0b640 1 --2- 172.21.15.112:0/2460647771 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8a600a2800 0x7f8a600a4cc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:09.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.944+0000 7f8a78e0b640 1 --2- 172.21.15.112:0/2460647771 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a74103480 0x7f8a7419f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:09.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.944+0000 7f8a78e0b640 1 --2- 172.21.15.112:0/2460647771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a74102ac0 0x7f8a741981b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:09.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.944+0000 7f8a78e0b640 1 --2- 172.21.15.112:0/2460647771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a740ffaf0 0x7f8a74197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:09.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.944+0000 7f8a78e0b640 1 -- 172.21.15.112:0/2460647771 >> 172.21.15.112:0/2460647771 conn(0x7f8a740fb420 msgr2=0x7f8a74105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:09.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.944+0000 7f8a78e0b640 1 -- 172.21.15.112:0/2460647771 shutdown_connections 2024-03-16T09:13:09.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:09.944+0000 7f8a78e0b640 1 -- 172.21.15.112:0/2460647771 wait complete. 2024-03-16T09:13:10.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:10 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2460647771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:10.737 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:13:10 smithi159 systemd[1]: Starting Ceph mds.e for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:13:10.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:10 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2460647771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:10.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:10 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2460647771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:11.105 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:13:10 smithi159 podman[65869]: 2024-03-16 09:13:10.732558797 +0000 UTC m=+0.018383946 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:13:11.106 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:13:10 smithi159 podman[65869]: 2024-03-16 09:13:10.851670268 +0000 UTC m=+0.137495450 container create f5bbe49efcbb5e5650369bd3bd3ed6036a3184dec6c9deb8d1cc1bfcc19f4fcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-e, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-16T09:13:11.459 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:13:11 smithi159 podman[65869]: 2024-03-16 09:13:11.104382508 +0000 UTC m=+0.390207664 container init f5bbe49efcbb5e5650369bd3bd3ed6036a3184dec6c9deb8d1cc1bfcc19f4fcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-e, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:13:11.459 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:13:11 smithi159 podman[65869]: 2024-03-16 09:13:11.107019777 +0000 UTC m=+0.392844910 container start f5bbe49efcbb5e5650369bd3bd3ed6036a3184dec6c9deb8d1cc1bfcc19f4fcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-e, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-16T09:13:11.459 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:13:11 smithi159 ceph-mds[65888]: -- 172.21.15.159:0/1620000377 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5610d27f9a40 con 0x5610d27cc800 2024-03-16T09:13:11.459 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:13:11 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-e[65884]: starting mds.e at 2024-03-16T09:13:11.459 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:13:11 smithi159 bash[65869]: f5bbe49efcbb5e5650369bd3bd3ed6036a3184dec6c9deb8d1cc1bfcc19f4fcd 2024-03-16T09:13:11.459 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:13:11 smithi159 systemd[1]: Started Ceph mds.e for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:13:11.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:11 smithi159 ceph-mon[36739]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:11.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:11 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:11.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:11 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:11.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:11 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:11.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:11 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:11.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:11 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:11.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:11 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:11.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:11 smithi112 ceph-mon[32307]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:11.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:11 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:11.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:11 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:11.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:11 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:11.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:11 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:11.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:11 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:11.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:11 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:11.899 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:11 smithi194 ceph-mon[36729]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:11.899 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:11 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:11.899 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:11 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:11.900 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:11 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:11.900 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:11 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:11.900 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:11 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:11.900 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:11 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:11.958 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:13:12.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.070+0000 7fe0b767e640 1 -- 172.21.15.112:0/1646871449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b0100720 msgr2=0x7fe0b0100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:12.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.070+0000 7fe0b767e640 1 --2- 172.21.15.112:0/1646871449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b0100720 0x7fe0b0100b40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009900 tx=0x7fe0a002bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:12.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.070+0000 7fe0b767e640 1 -- 172.21.15.112:0/1646871449 shutdown_connections 2024-03-16T09:13:12.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.070+0000 7fe0b767e640 1 --2- 172.21.15.112:0/1646871449 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0b01022d0 0x7fe0b0108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:12.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.070+0000 7fe0b767e640 1 --2- 172.21.15.112:0/1646871449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0b0101910 0x7fe0b0101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:12.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.070+0000 7fe0b767e640 1 --2- 172.21.15.112:0/1646871449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b0100720 0x7fe0b0100b40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:12.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.070+0000 7fe0b767e640 1 -- 172.21.15.112:0/1646871449 >> 172.21.15.112:0/1646871449 conn(0x7fe0b00fbc90 msgr2=0x7fe0b00fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:12.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.070+0000 7fe0b767e640 1 -- 172.21.15.112:0/1646871449 shutdown_connections 2024-03-16T09:13:12.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.070+0000 7fe0b767e640 1 -- 172.21.15.112:0/1646871449 wait complete. 2024-03-16T09:13:12.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.071+0000 7fe0b767e640 1 Processor -- start 2024-03-16T09:13:12.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.071+0000 7fe0b767e640 1 -- start start 2024-03-16T09:13:12.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.071+0000 7fe0b767e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b0100720 0x7fe0b006a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:12.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.071+0000 7fe0b767e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0b0101910 0x7fe0b019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:12.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.071+0000 7fe0b767e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0b01022d0 0x7fe0b019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:12.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.071+0000 7fe0b767e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b010c4e0 con 0x7fe0b0100720 2024-03-16T09:13:12.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.071+0000 7fe0b767e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b010c360 con 0x7fe0b01022d0 2024-03-16T09:13:12.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.071+0000 7fe0b767e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b010c660 con 0x7fe0b0101910 2024-03-16T09:13:12.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.071+0000 7fe0b541a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b0100720 0x7fe0b006a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:12.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.071+0000 7fe0b541a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b0100720 0x7fe0b006a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51896/0 (socket says 172.21.15.112:51896) 2024-03-16T09:13:12.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.071+0000 7fe0b541a640 1 -- 172.21.15.112:0/3368607845 learned_addr learned my addr 172.21.15.112:0/3368607845 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:12.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.071+0000 7fe0b5c1b640 1 --2- 172.21.15.112:0/3368607845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0b01022d0 0x7fe0b019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:12.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.071+0000 7fe0b4c19640 1 --2- 172.21.15.112:0/3368607845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0b0101910 0x7fe0b019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:12.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.071+0000 7fe0b541a640 1 -- 172.21.15.112:0/3368607845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0b0101910 msgr2=0x7fe0b019c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:12.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.071+0000 7fe0b541a640 1 --2- 172.21.15.112:0/3368607845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0b0101910 0x7fe0b019c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:12.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.071+0000 7fe0b541a640 1 -- 172.21.15.112:0/3368607845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0b01022d0 msgr2=0x7fe0b019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:12.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.071+0000 7fe0b541a640 1 --2- 172.21.15.112:0/3368607845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0b01022d0 0x7fe0b019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:12.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.071+0000 7fe0b541a640 1 -- 172.21.15.112:0/3368607845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b019f8a0 con 0x7fe0b0100720 2024-03-16T09:13:12.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.071+0000 7fe0b4c19640 1 --2- 172.21.15.112:0/3368607845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0b0101910 0x7fe0b019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:13:12.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.072+0000 7fe0b541a640 1 --2- 172.21.15.112:0/3368607845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b0100720 0x7fe0b006a9e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0005ab0 tx=0x7fe0a002c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:12.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.072+0000 7fe0a67fc640 1 -- 172.21.15.112:0/3368607845 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0043020 con 0x7fe0b0100720 2024-03-16T09:13:12.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.072+0000 7fe0a67fc640 1 -- 172.21.15.112:0/3368607845 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe0a0004350 con 0x7fe0b0100720 2024-03-16T09:13:12.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.072+0000 7fe0b767e640 1 -- 172.21.15.112:0/3368607845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b019fb30 con 0x7fe0b0100720 2024-03-16T09:13:12.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.072+0000 7fe0a67fc640 1 -- 172.21.15.112:0/3368607845 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0003d70 con 0x7fe0b0100720 2024-03-16T09:13:12.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.072+0000 7fe0b767e640 1 -- 172.21.15.112:0/3368607845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b01a00d0 con 0x7fe0b0100720 2024-03-16T09:13:12.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.073+0000 7fe0b767e640 1 -- 172.21.15.112:0/3368607845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0b0107ac0 con 0x7fe0b0100720 2024-03-16T09:13:12.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.074+0000 7fe0a67fc640 1 -- 172.21.15.112:0/3368607845 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fe0a0046020 con 0x7fe0b0100720 2024-03-16T09:13:12.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.074+0000 7fe0a67fc640 1 --2- 172.21.15.112:0/3368607845 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe09c0a2880 0x7fe09c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:12.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.074+0000 7fe0b4c19640 1 --2- 172.21.15.112:0/3368607845 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe09c0a2880 0x7fe09c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:12.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.074+0000 7fe0a67fc640 1 -- 172.21.15.112:0/3368607845 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe0a00f07e0 con 0x7fe0b0100720 2024-03-16T09:13:12.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.075+0000 7fe0b4c19640 1 --2- 172.21.15.112:0/3368607845 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe09c0a2880 0x7fe09c0a4d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe0b006a750 tx=0x7fe0ac005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:12.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.079+0000 7fe0a67fc640 1 -- 172.21.15.112:0/3368607845 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe0a004d020 con 0x7fe0b0100720 2024-03-16T09:13:12.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.289+0000 7fe0b767e640 1 -- 172.21.15.112:0/3368607845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0b002d380 con 0x7fe0b0100720 2024-03-16T09:13:12.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.290+0000 7fe0a67fc640 1 -- 172.21.15.112:0/3368607845 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4705 (secure 0 0 0) 0x7fe0a00bdd20 con 0x7fe0b0100720 2024-03-16T09:13:12.292 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:12.292 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-16T09:13:12.292 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 6 2024-03-16T09:13:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.292+0000 7fe0b767e640 1 -- 172.21.15.112:0/3368607845 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe09c0a2880 msgr2=0x7fe09c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.292+0000 7fe0b767e640 1 --2- 172.21.15.112:0/3368607845 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe09c0a2880 0x7fe09c0a4d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe0b006a750 tx=0x7fe0ac005e50 comp rx=0 tx=0).stop 2024-03-16T09:13:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.292+0000 7fe0b767e640 1 -- 172.21.15.112:0/3368607845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b0100720 msgr2=0x7fe0b006a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.292+0000 7fe0b767e640 1 --2- 172.21.15.112:0/3368607845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b0100720 0x7fe0b006a9e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0005ab0 tx=0x7fe0a002c7a0 comp rx=0 tx=0).stop 2024-03-16T09:13:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.292+0000 7fe0b767e640 1 -- 172.21.15.112:0/3368607845 shutdown_connections 2024-03-16T09:13:12.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.292+0000 7fe0b767e640 1 --2- 172.21.15.112:0/3368607845 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe09c0a2880 0x7fe09c0a4d40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:12.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.292+0000 7fe0b767e640 1 --2- 172.21.15.112:0/3368607845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0b01022d0 0x7fe0b019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:12.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.292+0000 7fe0b767e640 1 --2- 172.21.15.112:0/3368607845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0b0101910 0x7fe0b019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:12.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.292+0000 7fe0b767e640 1 --2- 172.21.15.112:0/3368607845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0b0100720 0x7fe0b006a9e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:12.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.292+0000 7fe0b767e640 1 -- 172.21.15.112:0/3368607845 >> 172.21.15.112:0/3368607845 conn(0x7fe0b00fbc90 msgr2=0x7fe0b0104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:12.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.292+0000 7fe0b767e640 1 -- 172.21.15.112:0/3368607845 shutdown_connections 2024-03-16T09:13:12.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:12.292+0000 7fe0b767e640 1 -- 172.21.15.112:0/3368607845 wait complete. 2024-03-16T09:13:12.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:12 smithi112 ceph-mon[32307]: Deploying daemon mds.f on smithi194 2024-03-16T09:13:12.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:12 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.159:6834/567707106,v1:172.21.15.159:6835/567707106] up:boot 2024-03-16T09:13:12.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:12 smithi112 ceph-mon[32307]: fsmap 5 up:standby 2024-03-16T09:13:12.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:12 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T09:13:12.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:12 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3368607845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:12.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:12 smithi159 ceph-mon[36739]: Deploying daemon mds.f on smithi194 2024-03-16T09:13:12.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:12 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.159:6834/567707106,v1:172.21.15.159:6835/567707106] up:boot 2024-03-16T09:13:12.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:12 smithi159 ceph-mon[36739]: fsmap 5 up:standby 2024-03-16T09:13:12.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:12 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T09:13:12.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:12 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3368607845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:12.872 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:12 smithi194 ceph-mon[36729]: Deploying daemon mds.f on smithi194 2024-03-16T09:13:12.872 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:12 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.159:6834/567707106,v1:172.21.15.159:6835/567707106] up:boot 2024-03-16T09:13:12.872 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:12 smithi194 ceph-mon[36729]: fsmap 5 up:standby 2024-03-16T09:13:12.872 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:12 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T09:13:12.872 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:12 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3368607845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:13.468 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 09:13:13 smithi194 systemd[1]: Starting Ceph mds.f for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:13:13.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:13 smithi159 ceph-mon[36739]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:13.737 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:13 smithi194 ceph-mon[36729]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:13.737 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 09:13:13 smithi194 podman[67558]: 2024-03-16 09:13:13.480440181 +0000 UTC m=+0.019850451 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:13:13.737 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 09:13:13 smithi194 podman[67558]: 2024-03-16 09:13:13.607069121 +0000 UTC m=+0.146479371 container create 0b0b811fcf98628c3a07f487dc958ba11cf8e22da8313668a498b72ef9ec24a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-f, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:13:13.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:13 smithi112 ceph-mon[32307]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:14.152 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 09:13:13 smithi194 podman[67558]: 2024-03-16 09:13:13.859337261 +0000 UTC m=+0.398747520 container init 0b0b811fcf98628c3a07f487dc958ba11cf8e22da8313668a498b72ef9ec24a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:13:14.152 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 09:13:13 smithi194 podman[67558]: 2024-03-16 09:13:13.861979096 +0000 UTC m=+0.401389349 container start 0b0b811fcf98628c3a07f487dc958ba11cf8e22da8313668a498b72ef9ec24a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-f, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-16T09:13:14.152 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 09:13:13 smithi194 ceph-mds[67577]: -- 172.21.15.194:0/3571945351 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff183e7a40 con 0x55ff183ba800 2024-03-16T09:13:14.152 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 09:13:13 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-f[67573]: starting mds.f at 2024-03-16T09:13:14.152 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 09:13:13 smithi194 bash[67558]: 0b0b811fcf98628c3a07f487dc958ba11cf8e22da8313668a498b72ef9ec24a5 2024-03-16T09:13:14.152 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 09:13:13 smithi194 systemd[1]: Started Ceph mds.f for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:13:14.306 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:13:14.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.418+0000 7fc0748ae640 1 -- 172.21.15.112:0/1247950331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0700ffaf0 msgr2=0x7fc0700fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:14.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.418+0000 7fc0748ae640 1 --2- 172.21.15.112:0/1247950331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0700ffaf0 0x7fc0700fff10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc060009930 tx=0x7fc06002bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:14.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.418+0000 7fc0748ae640 1 -- 172.21.15.112:0/1247950331 shutdown_connections 2024-03-16T09:13:14.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.418+0000 7fc0748ae640 1 --2- 172.21.15.112:0/1247950331 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc070103480 0x7fc070109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:14.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.418+0000 7fc0748ae640 1 --2- 172.21.15.112:0/1247950331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc070102ac0 0x7fc070102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:14.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.418+0000 7fc0748ae640 1 --2- 172.21.15.112:0/1247950331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0700ffaf0 0x7fc0700fff10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:14.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.418+0000 7fc0748ae640 1 -- 172.21.15.112:0/1247950331 >> 172.21.15.112:0/1247950331 conn(0x7fc0700fb420 msgr2=0x7fc0700fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:14.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.418+0000 7fc0748ae640 1 -- 172.21.15.112:0/1247950331 shutdown_connections 2024-03-16T09:13:14.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.418+0000 7fc0748ae640 1 -- 172.21.15.112:0/1247950331 wait complete. 2024-03-16T09:13:14.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.419+0000 7fc0748ae640 1 Processor -- start 2024-03-16T09:13:14.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.419+0000 7fc0748ae640 1 -- start start 2024-03-16T09:13:14.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.419+0000 7fc0748ae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0700ffaf0 0x7fc070197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:14.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.419+0000 7fc0748ae640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc070102ac0 0x7fc0701981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:14.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.419+0000 7fc0748ae640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc070103480 0x7fc07019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:14.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.419+0000 7fc0748ae640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc07010d5b0 con 0x7fc0700ffaf0 2024-03-16T09:13:14.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.419+0000 7fc0748ae640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc07010d430 con 0x7fc070103480 2024-03-16T09:13:14.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.419+0000 7fc0748ae640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc07010d730 con 0x7fc070102ac0 2024-03-16T09:13:14.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.419+0000 7fc06e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0700ffaf0 0x7fc070197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:14.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.419+0000 7fc06e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0700ffaf0 0x7fc070197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51906/0 (socket says 172.21.15.112:51906) 2024-03-16T09:13:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.419+0000 7fc06e59c640 1 -- 172.21.15.112:0/821627558 learned_addr learned my addr 172.21.15.112:0/821627558 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.419+0000 7fc06dd9b640 1 --2- 172.21.15.112:0/821627558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc070102ac0 0x7fc0701981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.420+0000 7fc06ed9d640 1 --2- 172.21.15.112:0/821627558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc070103480 0x7fc07019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.420+0000 7fc06e59c640 1 -- 172.21.15.112:0/821627558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc070102ac0 msgr2=0x7fc0701981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.420+0000 7fc06e59c640 1 --2- 172.21.15.112:0/821627558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc070102ac0 0x7fc0701981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.420+0000 7fc06e59c640 1 -- 172.21.15.112:0/821627558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc070103480 msgr2=0x7fc07019f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.420+0000 7fc06e59c640 1 --2- 172.21.15.112:0/821627558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc070103480 0x7fc07019f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.420+0000 7fc06e59c640 1 -- 172.21.15.112:0/821627558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07019f8b0 con 0x7fc0700ffaf0 2024-03-16T09:13:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.420+0000 7fc06ed9d640 1 --2- 172.21.15.112:0/821627558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc070103480 0x7fc07019f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:13:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.420+0000 7fc06e59c640 1 --2- 172.21.15.112:0/821627558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0700ffaf0 0x7fc070197c70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc06002c900 tx=0x7fc060004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.420+0000 7fc05f7fe640 1 -- 172.21.15.112:0/821627558 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc060043020 con 0x7fc0700ffaf0 2024-03-16T09:13:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.420+0000 7fc05f7fe640 1 -- 172.21.15.112:0/821627558 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc060004250 con 0x7fc0700ffaf0 2024-03-16T09:13:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.420+0000 7fc0748ae640 1 -- 172.21.15.112:0/821627558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07019fb40 con 0x7fc0700ffaf0 2024-03-16T09:13:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.420+0000 7fc05f7fe640 1 -- 172.21.15.112:0/821627558 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc060003ac0 con 0x7fc0700ffaf0 2024-03-16T09:13:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.420+0000 7fc0748ae640 1 -- 172.21.15.112:0/821627558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0701a0090 con 0x7fc0700ffaf0 2024-03-16T09:13:14.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.421+0000 7fc0748ae640 1 -- 172.21.15.112:0/821627558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc070108cc0 con 0x7fc0700ffaf0 2024-03-16T09:13:14.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.422+0000 7fc05f7fe640 1 -- 172.21.15.112:0/821627558 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fc060046020 con 0x7fc0700ffaf0 2024-03-16T09:13:14.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.423+0000 7fc05f7fe640 1 --2- 172.21.15.112:0/821627558 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc0580a2800 0x7fc0580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:14.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.423+0000 7fc06dd9b640 1 --2- 172.21.15.112:0/821627558 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc0580a2800 0x7fc0580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:14.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.423+0000 7fc05f7fe640 1 -- 172.21.15.112:0/821627558 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc0600f0740 con 0x7fc0700ffaf0 2024-03-16T09:13:14.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.423+0000 7fc06dd9b640 1 --2- 172.21.15.112:0/821627558 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc0580a2800 0x7fc0580a4cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc070199170 tx=0x7fc064005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:14.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.427+0000 7fc05f7fe640 1 -- 172.21.15.112:0/821627558 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc06004d020 con 0x7fc0700ffaf0 2024-03-16T09:13:14.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.642+0000 7fc0748ae640 1 -- 172.21.15.112:0/821627558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc07002d1b0 con 0x7fc0700ffaf0 2024-03-16T09:13:14.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.642+0000 7fc05f7fe640 1 -- 172.21.15.112:0/821627558 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5533 (secure 0 0 0) 0x7fc0600bdc80 con 0x7fc0700ffaf0 2024-03-16T09:13:14.644 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:14.644 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-03-16T09:13:14.644 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 7 2024-03-16T09:13:14.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.644+0000 7fc0748ae640 1 -- 172.21.15.112:0/821627558 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc0580a2800 msgr2=0x7fc0580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:14.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.644+0000 7fc0748ae640 1 --2- 172.21.15.112:0/821627558 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc0580a2800 0x7fc0580a4cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc070199170 tx=0x7fc064005f90 comp rx=0 tx=0).stop 2024-03-16T09:13:14.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.644+0000 7fc0748ae640 1 -- 172.21.15.112:0/821627558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0700ffaf0 msgr2=0x7fc070197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:14.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.644+0000 7fc0748ae640 1 --2- 172.21.15.112:0/821627558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0700ffaf0 0x7fc070197c70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc06002c900 tx=0x7fc060004a30 comp rx=0 tx=0).stop 2024-03-16T09:13:14.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.644+0000 7fc0748ae640 1 -- 172.21.15.112:0/821627558 shutdown_connections 2024-03-16T09:13:14.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.644+0000 7fc0748ae640 1 --2- 172.21.15.112:0/821627558 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc0580a2800 0x7fc0580a4cc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:14.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.644+0000 7fc0748ae640 1 --2- 172.21.15.112:0/821627558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc070103480 0x7fc07019f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:14.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.644+0000 7fc0748ae640 1 --2- 172.21.15.112:0/821627558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc070102ac0 0x7fc0701981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:14.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.644+0000 7fc0748ae640 1 --2- 172.21.15.112:0/821627558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0700ffaf0 0x7fc070197c70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:14.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.644+0000 7fc0748ae640 1 -- 172.21.15.112:0/821627558 >> 172.21.15.112:0/821627558 conn(0x7fc0700fb420 msgr2=0x7fc070105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:14.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.645+0000 7fc0748ae640 1 -- 172.21.15.112:0/821627558 shutdown_connections 2024-03-16T09:13:14.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:14.645+0000 7fc0748ae640 1 -- 172.21.15.112:0/821627558 wait complete. 2024-03-16T09:13:15.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:15 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:15.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:15 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:15.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:15 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:15.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:15 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:15.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:15 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:15.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:15 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:15.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:15 smithi112 ceph-mon[32307]: Deploying daemon mds.c on smithi194 2024-03-16T09:13:15.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:15 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.194:6834/2682964784,v1:172.21.15.194:6835/2682964784] up:boot 2024-03-16T09:13:15.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:15 smithi112 ceph-mon[32307]: fsmap 6 up:standby 2024-03-16T09:13:15.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:15 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T09:13:15.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:15 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/821627558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:15.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:15 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:15.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:15 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:15.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:15 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:15.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:15 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:15.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:15 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:15.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:15 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:15.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:15 smithi194 ceph-mon[36729]: Deploying daemon mds.c on smithi194 2024-03-16T09:13:15.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:15 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.194:6834/2682964784,v1:172.21.15.194:6835/2682964784] up:boot 2024-03-16T09:13:15.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:15 smithi194 ceph-mon[36729]: fsmap 6 up:standby 2024-03-16T09:13:15.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:15 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T09:13:15.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:15 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/821627558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:15.413 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:15 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:15.413 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:15 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:15.413 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:15 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:15.413 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:15 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:15.413 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:15 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:15.413 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:15 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:15.413 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:15 smithi159 ceph-mon[36739]: Deploying daemon mds.c on smithi194 2024-03-16T09:13:15.413 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:15 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.194:6834/2682964784,v1:172.21.15.194:6835/2682964784] up:boot 2024-03-16T09:13:15.413 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:15 smithi159 ceph-mon[36739]: fsmap 6 up:standby 2024-03-16T09:13:15.413 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:15 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T09:13:15.413 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:15 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/821627558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:16.290 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:16 smithi194 ceph-mon[36729]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:16.290 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:13:16 smithi194 systemd[1]: Starting Ceph mds.c for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:13:16.341 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:16 smithi112 ceph-mon[32307]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:16.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:16 smithi159 ceph-mon[36739]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:16.597 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:13:16 smithi194 podman[67869]: 2024-03-16 09:13:16.357546671 +0000 UTC m=+0.017937022 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:13:16.597 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:13:16 smithi194 podman[67869]: 2024-03-16 09:13:16.478261437 +0000 UTC m=+0.138651787 container create 42b46e581e529129b8f5c72293e3dc1f54012327be9e5cdc97d89793a9f71aa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS) 2024-03-16T09:13:16.658 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:13:16.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.769+0000 7fd631bd4640 1 -- 172.21.15.112:0/724400348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd62c101800 msgr2=0x7fd62c101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:16.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.769+0000 7fd631bd4640 1 --2- 172.21.15.112:0/724400348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd62c101800 0x7fd62c101c20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009900 tx=0x7fd61c02bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:16.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.769+0000 7fd631bd4640 1 -- 172.21.15.112:0/724400348 shutdown_connections 2024-03-16T09:13:16.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.769+0000 7fd631bd4640 1 --2- 172.21.15.112:0/724400348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd62c1033b0 0x7fd62c109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:16.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.769+0000 7fd631bd4640 1 --2- 172.21.15.112:0/724400348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd62c1029f0 0x7fd62c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:16.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.769+0000 7fd631bd4640 1 --2- 172.21.15.112:0/724400348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd62c101800 0x7fd62c101c20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:16.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.769+0000 7fd631bd4640 1 -- 172.21.15.112:0/724400348 >> 172.21.15.112:0/724400348 conn(0x7fd62c0fcd90 msgr2=0x7fd62c0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:16.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.769+0000 7fd631bd4640 1 -- 172.21.15.112:0/724400348 shutdown_connections 2024-03-16T09:13:16.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.769+0000 7fd631bd4640 1 -- 172.21.15.112:0/724400348 wait complete. 2024-03-16T09:13:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.769+0000 7fd631bd4640 1 Processor -- start 2024-03-16T09:13:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.770+0000 7fd631bd4640 1 -- start start 2024-03-16T09:13:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.770+0000 7fd631bd4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd62c101800 0x7fd62c075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:16.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.770+0000 7fd631bd4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd62c1029f0 0x7fd62c073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.770+0000 7fd631bd4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd62c1033b0 0x7fd62c074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.770+0000 7fd631bd4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd62c10d560 con 0x7fd62c101800 2024-03-16T09:13:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.770+0000 7fd631bd4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd62c10d3e0 con 0x7fd62c1033b0 2024-03-16T09:13:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.770+0000 7fd631bd4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd62c10d6e0 con 0x7fd62c1029f0 2024-03-16T09:13:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.770+0000 7fd62b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd62c101800 0x7fd62c075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.770+0000 7fd62b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd62c101800 0x7fd62c075590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51918/0 (socket says 172.21.15.112:51918) 2024-03-16T09:13:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.770+0000 7fd62b7fe640 1 -- 172.21.15.112:0/1750418786 learned_addr learned my addr 172.21.15.112:0/1750418786 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.770+0000 7fd62affd640 1 --2- 172.21.15.112:0/1750418786 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd62c1029f0 0x7fd62c073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.770+0000 7fd62bfff640 1 --2- 172.21.15.112:0/1750418786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd62c1033b0 0x7fd62c074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.770+0000 7fd62b7fe640 1 -- 172.21.15.112:0/1750418786 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd62c1029f0 msgr2=0x7fd62c073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.770+0000 7fd62b7fe640 1 --2- 172.21.15.112:0/1750418786 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd62c1029f0 0x7fd62c073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.770+0000 7fd62b7fe640 1 -- 172.21.15.112:0/1750418786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd62c1033b0 msgr2=0x7fd62c074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:16.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.770+0000 7fd62b7fe640 1 --2- 172.21.15.112:0/1750418786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd62c1033b0 0x7fd62c074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.770+0000 7fd62b7fe640 1 -- 172.21.15.112:0/1750418786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62c0749a0 con 0x7fd62c101800 2024-03-16T09:13:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.770+0000 7fd62b7fe640 1 --2- 172.21.15.112:0/1750418786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd62c101800 0x7fd62c075590 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd61c000c00 tx=0x7fd61c004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.771+0000 7fd628ff9640 1 -- 172.21.15.112:0/1750418786 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c043020 con 0x7fd62c101800 2024-03-16T09:13:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.771+0000 7fd631bd4640 1 -- 172.21.15.112:0/1750418786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd62c1a1cf0 con 0x7fd62c101800 2024-03-16T09:13:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.771+0000 7fd628ff9640 1 -- 172.21.15.112:0/1750418786 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd61c004250 con 0x7fd62c101800 2024-03-16T09:13:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.771+0000 7fd628ff9640 1 -- 172.21.15.112:0/1750418786 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c003b90 con 0x7fd62c101800 2024-03-16T09:13:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.771+0000 7fd631bd4640 1 -- 172.21.15.112:0/1750418786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd62c1a2180 con 0x7fd62c101800 2024-03-16T09:13:16.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.772+0000 7fd631bd4640 1 -- 172.21.15.112:0/1750418786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd62c108b20 con 0x7fd62c101800 2024-03-16T09:13:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.773+0000 7fd628ff9640 1 -- 172.21.15.112:0/1750418786 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fd61c046020 con 0x7fd62c101800 2024-03-16T09:13:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.773+0000 7fd628ff9640 1 --2- 172.21.15.112:0/1750418786 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6140a2800 0x7fd6140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.774+0000 7fd62affd640 1 --2- 172.21.15.112:0/1750418786 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6140a2800 0x7fd6140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.774+0000 7fd628ff9640 1 -- 172.21.15.112:0/1750418786 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd61c0f0c80 con 0x7fd62c101800 2024-03-16T09:13:16.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.774+0000 7fd62affd640 1 --2- 172.21.15.112:0/1750418786 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6140a2800 0x7fd6140a4cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd62c075300 tx=0x7fd620005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:16.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.777+0000 7fd628ff9640 1 -- 172.21.15.112:0/1750418786 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd61c04d020 con 0x7fd62c101800 2024-03-16T09:13:16.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.993+0000 7fd631bd4640 1 -- 172.21.15.112:0/1750418786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd62c02d1b0 con 0x7fd62c101800 2024-03-16T09:13:16.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.994+0000 7fd628ff9640 1 -- 172.21.15.112:0/1750418786 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5533 (secure 0 0 0) 0x7fd61c0be1c0 con 0x7fd62c101800 2024-03-16T09:13:16.996 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:16.996 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-03-16T09:13:16.996 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 7 2024-03-16T09:13:16.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.996+0000 7fd631bd4640 1 -- 172.21.15.112:0/1750418786 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6140a2800 msgr2=0x7fd6140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:16.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.996+0000 7fd631bd4640 1 --2- 172.21.15.112:0/1750418786 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6140a2800 0x7fd6140a4cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd62c075300 tx=0x7fd620005f90 comp rx=0 tx=0).stop 2024-03-16T09:13:16.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.996+0000 7fd631bd4640 1 -- 172.21.15.112:0/1750418786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd62c101800 msgr2=0x7fd62c075590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:16.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.996+0000 7fd631bd4640 1 --2- 172.21.15.112:0/1750418786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd62c101800 0x7fd62c075590 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd61c000c00 tx=0x7fd61c004a30 comp rx=0 tx=0).stop 2024-03-16T09:13:16.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.996+0000 7fd631bd4640 1 -- 172.21.15.112:0/1750418786 shutdown_connections 2024-03-16T09:13:16.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.996+0000 7fd631bd4640 1 --2- 172.21.15.112:0/1750418786 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6140a2800 0x7fd6140a4cc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:16.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.996+0000 7fd631bd4640 1 --2- 172.21.15.112:0/1750418786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd62c1033b0 0x7fd62c074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:16.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.996+0000 7fd631bd4640 1 --2- 172.21.15.112:0/1750418786 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd62c1029f0 0x7fd62c073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:16.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.996+0000 7fd631bd4640 1 --2- 172.21.15.112:0/1750418786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd62c101800 0x7fd62c075590 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:16.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.996+0000 7fd631bd4640 1 -- 172.21.15.112:0/1750418786 >> 172.21.15.112:0/1750418786 conn(0x7fd62c0fcd90 msgr2=0x7fd62c105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:16.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.996+0000 7fd631bd4640 1 -- 172.21.15.112:0/1750418786 shutdown_connections 2024-03-16T09:13:16.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:16.996+0000 7fd631bd4640 1 -- 172.21.15.112:0/1750418786 wait complete. 2024-03-16T09:13:17.152 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:13:16 smithi194 podman[67869]: 2024-03-16 09:13:16.723264633 +0000 UTC m=+0.383654987 container init 42b46e581e529129b8f5c72293e3dc1f54012327be9e5cdc97d89793a9f71aa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:13:17.152 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:13:16 smithi194 podman[67869]: 2024-03-16 09:13:16.726064797 +0000 UTC m=+0.386455145 container start 42b46e581e529129b8f5c72293e3dc1f54012327be9e5cdc97d89793a9f71aa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-c, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-16T09:13:17.152 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:13:16 smithi194 ceph-mds[67898]: -- 172.21.15.194:0/3015915936 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561c60ea1a40 con 0x561c60e74800 2024-03-16T09:13:17.152 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:13:16 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-c[67892]: starting mds.c at 2024-03-16T09:13:17.152 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:13:16 smithi194 bash[67869]: 42b46e581e529129b8f5c72293e3dc1f54012327be9e5cdc97d89793a9f71aa2 2024-03-16T09:13:17.152 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:13:16 smithi194 systemd[1]: Started Ceph mds.c for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:13:18.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:17 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:18.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:17 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:18.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:17 smithi194 ceph-mon[36729]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:18.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:17 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:18.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:17 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:18.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:17 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:18.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:17 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:18.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:17 smithi194 ceph-mon[36729]: Deploying daemon mds.h on smithi159 2024-03-16T09:13:18.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:17 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1750418786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:18.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:17 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.194:6836/1682076,v1:172.21.15.194:6837/1682076] up:boot 2024-03-16T09:13:18.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:17 smithi194 ceph-mon[36729]: fsmap 7 up:standby 2024-03-16T09:13:18.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:17 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T09:13:18.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:17 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:18.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:17 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:18.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:17 smithi159 ceph-mon[36739]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:18.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:17 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:18.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:17 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:18.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:17 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:18.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:17 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:18.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:17 smithi159 ceph-mon[36739]: Deploying daemon mds.h on smithi159 2024-03-16T09:13:18.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:17 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1750418786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:18.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:17 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.194:6836/1682076,v1:172.21.15.194:6837/1682076] up:boot 2024-03-16T09:13:18.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:17 smithi159 ceph-mon[36739]: fsmap 7 up:standby 2024-03-16T09:13:18.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:17 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T09:13:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:17 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:17 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:17 smithi112 ceph-mon[32307]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:17 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:17 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:17 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:18.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:17 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:18.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:17 smithi112 ceph-mon[32307]: Deploying daemon mds.h on smithi159 2024-03-16T09:13:18.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:17 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1750418786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:18.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:17 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.194:6836/1682076,v1:172.21.15.194:6837/1682076] up:boot 2024-03-16T09:13:18.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:17 smithi112 ceph-mon[32307]: fsmap 7 up:standby 2024-03-16T09:13:18.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:17 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T09:13:19.010 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:13:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.122+0000 7fb55490c640 1 -- 172.21.15.112:0/810768240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb550102e30 msgr2=0x7fb550103250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.122+0000 7fb55490c640 1 --2- 172.21.15.112:0/810768240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb550102e30 0x7fb550103250 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb548009ca0 tx=0x7fb54802c0f0 comp rx=0 tx=0).stop 2024-03-16T09:13:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.122+0000 7fb55490c640 1 -- 172.21.15.112:0/810768240 shutdown_connections 2024-03-16T09:13:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.122+0000 7fb55490c640 1 --2- 172.21.15.112:0/810768240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb550100260 0x7fb550109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.122+0000 7fb55490c640 1 --2- 172.21.15.112:0/810768240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5500ff8a0 0x7fb5500ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.122+0000 7fb55490c640 1 --2- 172.21.15.112:0/810768240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb550102e30 0x7fb550103250 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.122+0000 7fb55490c640 1 -- 172.21.15.112:0/810768240 >> 172.21.15.112:0/810768240 conn(0x7fb5500fb420 msgr2=0x7fb5500fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.122+0000 7fb55490c640 1 -- 172.21.15.112:0/810768240 shutdown_connections 2024-03-16T09:13:19.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.122+0000 7fb55490c640 1 -- 172.21.15.112:0/810768240 wait complete. 2024-03-16T09:13:19.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb55490c640 1 Processor -- start 2024-03-16T09:13:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb55490c640 1 -- start start 2024-03-16T09:13:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb55490c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5500ff8a0 0x7fb550102530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb55490c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb550100260 0x7fb550100b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb55490c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb550102e30 0x7fb5501010b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb55490c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb55010d650 con 0x7fb5500ff8a0 2024-03-16T09:13:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb55490c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb55010d4d0 con 0x7fb550100260 2024-03-16T09:13:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb55490c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb55010d7d0 con 0x7fb550102e30 2024-03-16T09:13:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb54e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5500ff8a0 0x7fb550102530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb54e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5500ff8a0 0x7fb550102530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51940/0 (socket says 172.21.15.112:51940) 2024-03-16T09:13:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb54e59c640 1 -- 172.21.15.112:0/1453992673 learned_addr learned my addr 172.21.15.112:0/1453992673 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb54dd9b640 1 --2- 172.21.15.112:0/1453992673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb550100260 0x7fb550100b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb54ed9d640 1 --2- 172.21.15.112:0/1453992673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb550102e30 0x7fb5501010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb54e59c640 1 -- 172.21.15.112:0/1453992673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb550102e30 msgr2=0x7fb5501010b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb54e59c640 1 --2- 172.21.15.112:0/1453992673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb550102e30 0x7fb5501010b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb54e59c640 1 -- 172.21.15.112:0/1453992673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb550100260 msgr2=0x7fb550100b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb54e59c640 1 --2- 172.21.15.112:0/1453992673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb550100260 0x7fb550100b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb54e59c640 1 -- 172.21.15.112:0/1453992673 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb550101940 con 0x7fb5500ff8a0 2024-03-16T09:13:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb54dd9b640 1 --2- 172.21.15.112:0/1453992673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb550100260 0x7fb550100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:13:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb54ed9d640 1 --2- 172.21.15.112:0/1453992673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb550102e30 0x7fb5501010b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:13:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.123+0000 7fb54e59c640 1 --2- 172.21.15.112:0/1453992673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5500ff8a0 0x7fb550102530 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb548037040 tx=0x7fb5480346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.124+0000 7fb5477fe640 1 -- 172.21.15.112:0/1453992673 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548039040 con 0x7fb5500ff8a0 2024-03-16T09:13:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.124+0000 7fb55490c640 1 -- 172.21.15.112:0/1453992673 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5501a1d20 con 0x7fb5500ff8a0 2024-03-16T09:13:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.124+0000 7fb5477fe640 1 -- 172.21.15.112:0/1453992673 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb548044070 con 0x7fb5500ff8a0 2024-03-16T09:13:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.124+0000 7fb5477fe640 1 -- 172.21.15.112:0/1453992673 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548004500 con 0x7fb5500ff8a0 2024-03-16T09:13:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.124+0000 7fb55490c640 1 -- 172.21.15.112:0/1453992673 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5501a21c0 con 0x7fb5500ff8a0 2024-03-16T09:13:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.125+0000 7fb55490c640 1 -- 172.21.15.112:0/1453992673 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb550108c90 con 0x7fb5500ff8a0 2024-03-16T09:13:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.126+0000 7fb5477fe640 1 -- 172.21.15.112:0/1453992673 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fb5480046a0 con 0x7fb5500ff8a0 2024-03-16T09:13:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.126+0000 7fb5477fe640 1 --2- 172.21.15.112:0/1453992673 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb53c0a2800 0x7fb53c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.126+0000 7fb54dd9b640 1 --2- 172.21.15.112:0/1453992673 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb53c0a2800 0x7fb53c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.126+0000 7fb5477fe640 1 -- 172.21.15.112:0/1453992673 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb5480f0e10 con 0x7fb5500ff8a0 2024-03-16T09:13:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.127+0000 7fb54dd9b640 1 --2- 172.21.15.112:0/1453992673 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb53c0a2800 0x7fb53c0a4cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb5501022a0 tx=0x7fb538008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:19.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.131+0000 7fb5477fe640 1 -- 172.21.15.112:0/1453992673 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb5480be350 con 0x7fb5500ff8a0 2024-03-16T09:13:19.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.344+0000 7fb55490c640 1 -- 172.21.15.112:0/1453992673 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb55002d400 con 0x7fb5500ff8a0 2024-03-16T09:13:19.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.344+0000 7fb5477fe640 1 -- 172.21.15.112:0/1453992673 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7fb5480bdc50 con 0x7fb5500ff8a0 2024-03-16T09:13:19.346 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:19.346 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1682076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":1682076},{"type":"v1","addr":"172.21.15.194:6837","nonce":1682076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-16T09:13:19.346 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 8 2024-03-16T09:13:19.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.346+0000 7fb55490c640 1 -- 172.21.15.112:0/1453992673 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb53c0a2800 msgr2=0x7fb53c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:19.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.346+0000 7fb55490c640 1 --2- 172.21.15.112:0/1453992673 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb53c0a2800 0x7fb53c0a4cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb5501022a0 tx=0x7fb538008040 comp rx=0 tx=0).stop 2024-03-16T09:13:19.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.346+0000 7fb55490c640 1 -- 172.21.15.112:0/1453992673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5500ff8a0 msgr2=0x7fb550102530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:19.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.346+0000 7fb55490c640 1 --2- 172.21.15.112:0/1453992673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5500ff8a0 0x7fb550102530 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb548037040 tx=0x7fb5480346d0 comp rx=0 tx=0).stop 2024-03-16T09:13:19.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.346+0000 7fb55490c640 1 -- 172.21.15.112:0/1453992673 shutdown_connections 2024-03-16T09:13:19.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.346+0000 7fb55490c640 1 --2- 172.21.15.112:0/1453992673 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb53c0a2800 0x7fb53c0a4cc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:19.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.346+0000 7fb55490c640 1 --2- 172.21.15.112:0/1453992673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb550102e30 0x7fb5501010b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:19.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.346+0000 7fb55490c640 1 --2- 172.21.15.112:0/1453992673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb550100260 0x7fb550100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:19.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.346+0000 7fb55490c640 1 --2- 172.21.15.112:0/1453992673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5500ff8a0 0x7fb550102530 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:19.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.346+0000 7fb55490c640 1 -- 172.21.15.112:0/1453992673 >> 172.21.15.112:0/1453992673 conn(0x7fb5500fb420 msgr2=0x7fb550105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:19.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.346+0000 7fb55490c640 1 -- 172.21.15.112:0/1453992673 shutdown_connections 2024-03-16T09:13:19.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:19.346+0000 7fb55490c640 1 -- 172.21.15.112:0/1453992673 wait complete. 2024-03-16T09:13:19.517 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:13:19 smithi159 podman[66182]: 2024-03-16 09:13:19.377473164 +0000 UTC m=+0.018188657 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:13:19.517 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:13:19 smithi159 podman[66182]: 2024-03-16 09:13:19.516117587 +0000 UTC m=+0.156833058 container create d9c2c9c807002a6d4930484051a7e339810f326fa25291b9c62b048bf3820d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-16T09:13:20.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:19 smithi159 ceph-mon[36739]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:20.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:19 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1453992673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:20.237 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:13:19 smithi159 podman[66182]: 2024-03-16 09:13:19.777181777 +0000 UTC m=+0.417897257 container init d9c2c9c807002a6d4930484051a7e339810f326fa25291b9c62b048bf3820d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-03-16T09:13:20.237 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:13:19 smithi159 podman[66182]: 2024-03-16 09:13:19.779754738 +0000 UTC m=+0.420470228 container start d9c2c9c807002a6d4930484051a7e339810f326fa25291b9c62b048bf3820d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:13:20.237 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:13:19 smithi159 ceph-mds[66201]: -- 172.21.15.159:0/898113168 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55db55539a40 con 0x55db5550d000 2024-03-16T09:13:20.237 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:13:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: starting mds.h at 2024-03-16T09:13:20.237 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:13:19 smithi159 bash[66182]: d9c2c9c807002a6d4930484051a7e339810f326fa25291b9c62b048bf3820d03 2024-03-16T09:13:20.237 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:13:19 smithi159 systemd[1]: Started Ceph mds.h for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:13:20.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:19 smithi112 ceph-mon[32307]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:20.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:19 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1453992673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:20.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:19 smithi194 ceph-mon[36729]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:20.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:19 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1453992673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:21.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:20 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] up:boot 2024-03-16T09:13:21.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:20 smithi194 ceph-mon[36729]: fsmap 8 up:standby 2024-03-16T09:13:21.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T09:13:21.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:21.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:21.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:21.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:21.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:21.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:20 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:21.153 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:20 smithi194 ceph-mon[36729]: Deploying daemon mds.i on smithi194 2024-03-16T09:13:21.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:20 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] up:boot 2024-03-16T09:13:21.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:20 smithi159 ceph-mon[36739]: fsmap 8 up:standby 2024-03-16T09:13:21.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:20 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T09:13:21.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:20 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:21.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:20 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:21.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:20 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:21.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:20 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:21.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:20 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:21.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:20 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:21.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:20 smithi159 ceph-mon[36739]: Deploying daemon mds.i on smithi194 2024-03-16T09:13:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:20 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] up:boot 2024-03-16T09:13:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:20 smithi112 ceph-mon[32307]: fsmap 8 up:standby 2024-03-16T09:13:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T09:13:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:20 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:20 smithi112 ceph-mon[32307]: Deploying daemon mds.i on smithi194 2024-03-16T09:13:21.360 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:13:21.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.461+0000 7ff345fd3640 1 -- 172.21.15.112:0/1208257278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3401022b0 msgr2=0x7ff340108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:21.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.461+0000 7ff345fd3640 1 --2- 172.21.15.112:0/1208257278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3401022b0 0x7ff340108b40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff338009ca0 tx=0x7ff33802c0f0 comp rx=0 tx=0).stop 2024-03-16T09:13:21.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.461+0000 7ff345fd3640 1 -- 172.21.15.112:0/1208257278 shutdown_connections 2024-03-16T09:13:21.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.461+0000 7ff345fd3640 1 --2- 172.21.15.112:0/1208257278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3401022b0 0x7ff340108b40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:21.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.461+0000 7ff345fd3640 1 --2- 172.21.15.112:0/1208257278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3401018f0 0x7ff340101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:21.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.461+0000 7ff345fd3640 1 --2- 172.21.15.112:0/1208257278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff340100700 0x7ff340100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:21.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.461+0000 7ff345fd3640 1 -- 172.21.15.112:0/1208257278 >> 172.21.15.112:0/1208257278 conn(0x7ff3400fbc70 msgr2=0x7ff3400fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:21.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.461+0000 7ff345fd3640 1 -- 172.21.15.112:0/1208257278 shutdown_connections 2024-03-16T09:13:21.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.461+0000 7ff345fd3640 1 -- 172.21.15.112:0/1208257278 wait complete. 2024-03-16T09:13:21.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.462+0000 7ff345fd3640 1 Processor -- start 2024-03-16T09:13:21.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.462+0000 7ff345fd3640 1 -- start start 2024-03-16T09:13:21.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.462+0000 7ff345fd3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff340100700 0x7ff340104690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:21.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.462+0000 7ff345fd3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3401018f0 0x7ff340102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:21.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.462+0000 7ff345fd3640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3401022b0 0x7ff340103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:21.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.462+0000 7ff345fd3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff34010c530 con 0x7ff3401018f0 2024-03-16T09:13:21.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.462+0000 7ff345fd3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff34010c3b0 con 0x7ff340100700 2024-03-16T09:13:21.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.462+0000 7ff345fd3640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff34010c6b0 con 0x7ff3401022b0 2024-03-16T09:13:21.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.462+0000 7ff33effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3401018f0 0x7ff340102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:21.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.462+0000 7ff33effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3401018f0 0x7ff340102cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41808/0 (socket says 172.21.15.112:41808) 2024-03-16T09:13:21.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.462+0000 7ff33effd640 1 -- 172.21.15.112:0/1081481367 learned_addr learned my addr 172.21.15.112:0/1081481367 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:21.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.462+0000 7ff33f7fe640 1 --2- 172.21.15.112:0/1081481367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff340100700 0x7ff340104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:21.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.462+0000 7ff33effd640 1 -- 172.21.15.112:0/1081481367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3401022b0 msgr2=0x7ff340103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:21.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.462+0000 7ff33ffff640 1 --2- 172.21.15.112:0/1081481367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3401022b0 0x7ff340103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:21.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.462+0000 7ff33effd640 1 --2- 172.21.15.112:0/1081481367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3401022b0 0x7ff340103210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:21.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.462+0000 7ff33effd640 1 -- 172.21.15.112:0/1081481367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff340100700 msgr2=0x7ff340104690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:21.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.462+0000 7ff33effd640 1 --2- 172.21.15.112:0/1081481367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff340100700 0x7ff340104690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:21.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.462+0000 7ff33effd640 1 -- 172.21.15.112:0/1081481367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff340103aa0 con 0x7ff3401018f0 2024-03-16T09:13:21.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.463+0000 7ff33f7fe640 1 --2- 172.21.15.112:0/1081481367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff340100700 0x7ff340104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:13:21.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.463+0000 7ff33effd640 1 --2- 172.21.15.112:0/1081481367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3401018f0 0x7ff340102cd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff33400d950 tx=0x7ff33400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:21.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.463+0000 7ff33cff9640 1 -- 172.21.15.112:0/1081481367 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334017070 con 0x7ff3401018f0 2024-03-16T09:13:21.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.463+0000 7ff33cff9640 1 -- 172.21.15.112:0/1081481367 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff334009c30 con 0x7ff3401018f0 2024-03-16T09:13:21.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.463+0000 7ff345fd3640 1 -- 172.21.15.112:0/1081481367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3401a1d50 con 0x7ff3401018f0 2024-03-16T09:13:21.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.463+0000 7ff33cff9640 1 -- 172.21.15.112:0/1081481367 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33400f740 con 0x7ff3401018f0 2024-03-16T09:13:21.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.463+0000 7ff345fd3640 1 -- 172.21.15.112:0/1081481367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3401a2260 con 0x7ff3401018f0 2024-03-16T09:13:21.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.464+0000 7ff345fd3640 1 -- 172.21.15.112:0/1081481367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3401a24f0 con 0x7ff3401018f0 2024-03-16T09:13:21.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.465+0000 7ff33cff9640 1 -- 172.21.15.112:0/1081481367 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7ff334010a80 con 0x7ff3401018f0 2024-03-16T09:13:21.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.466+0000 7ff33cff9640 1 --2- 172.21.15.112:0/1081481367 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff3280a2800 0x7ff3280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:21.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.466+0000 7ff33f7fe640 1 --2- 172.21.15.112:0/1081481367 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff3280a2800 0x7ff3280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:21.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.466+0000 7ff33cff9640 1 -- 172.21.15.112:0/1081481367 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff3340ce2b0 con 0x7ff3401018f0 2024-03-16T09:13:21.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.466+0000 7ff33f7fe640 1 --2- 172.21.15.112:0/1081481367 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff3280a2800 0x7ff3280a4cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff340101750 tx=0x7ff330008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:21.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.470+0000 7ff33cff9640 1 -- 172.21.15.112:0/1081481367 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff334009de0 con 0x7ff3401018f0 2024-03-16T09:13:21.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.717+0000 7ff345fd3640 1 -- 172.21.15.112:0/1081481367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3401a2760 con 0x7ff3401018f0 2024-03-16T09:13:21.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.718+0000 7ff33cff9640 1 -- 172.21.15.112:0/1081481367 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7180 (secure 0 0 0) 0x7ff33409b7f0 con 0x7ff3401018f0 2024-03-16T09:13:21.721 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:21.721 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1682076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":1682076},{"type":"v1","addr":"172.21.15.194:6837","nonce":1682076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-03-16T09:13:21.721 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 9 2024-03-16T09:13:21.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.720+0000 7ff345fd3640 1 -- 172.21.15.112:0/1081481367 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff3280a2800 msgr2=0x7ff3280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:21.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.720+0000 7ff345fd3640 1 --2- 172.21.15.112:0/1081481367 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff3280a2800 0x7ff3280a4cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff340101750 tx=0x7ff330008040 comp rx=0 tx=0).stop 2024-03-16T09:13:21.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.720+0000 7ff345fd3640 1 -- 172.21.15.112:0/1081481367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3401018f0 msgr2=0x7ff340102cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:21.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.720+0000 7ff345fd3640 1 --2- 172.21.15.112:0/1081481367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3401018f0 0x7ff340102cd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff33400d950 tx=0x7ff33400de10 comp rx=0 tx=0).stop 2024-03-16T09:13:21.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.720+0000 7ff345fd3640 1 -- 172.21.15.112:0/1081481367 shutdown_connections 2024-03-16T09:13:21.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.720+0000 7ff345fd3640 1 --2- 172.21.15.112:0/1081481367 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff3280a2800 0x7ff3280a4cc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:21.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.720+0000 7ff345fd3640 1 --2- 172.21.15.112:0/1081481367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff3401022b0 0x7ff340103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:21.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.720+0000 7ff345fd3640 1 --2- 172.21.15.112:0/1081481367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3401018f0 0x7ff340102cd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:21.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.720+0000 7ff345fd3640 1 --2- 172.21.15.112:0/1081481367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff340100700 0x7ff340104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:21.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.720+0000 7ff345fd3640 1 -- 172.21.15.112:0/1081481367 >> 172.21.15.112:0/1081481367 conn(0x7ff3400fbc70 msgr2=0x7ff3400fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:21.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.720+0000 7ff345fd3640 1 -- 172.21.15.112:0/1081481367 shutdown_connections 2024-03-16T09:13:21.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:21.720+0000 7ff345fd3640 1 -- 172.21.15.112:0/1081481367 wait complete. 2024-03-16T09:13:22.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:21 smithi112 ceph-mon[32307]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:22.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:21 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1081481367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:22.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:21 smithi159 ceph-mon[36739]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:22.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:21 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1081481367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:22.331 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:21 smithi194 ceph-mon[36729]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:22.331 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:21 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1081481367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:22.331 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 09:13:22 smithi194 systemd[1]: Starting Ceph mds.i for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:13:22.629 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 09:13:22 smithi194 podman[68192]: 2024-03-16 09:13:22.231878338 +0000 UTC m=+0.017779533 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:13:22.629 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 09:13:22 smithi194 podman[68192]: 2024-03-16 09:13:22.378006962 +0000 UTC m=+0.163908143 container create bfe012df85d8197e7e0b8ac16aa75f86ee3c57073d9c6453b2af22d3963f1806 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-i, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-16T09:13:22.629 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 09:13:22 smithi194 podman[68192]: 2024-03-16 09:13:22.62772475 +0000 UTC m=+0.413625932 container init bfe012df85d8197e7e0b8ac16aa75f86ee3c57073d9c6453b2af22d3963f1806 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-i, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-16T09:13:22.902 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 09:13:22 smithi194 podman[68192]: 2024-03-16 09:13:22.630351998 +0000 UTC m=+0.416253180 container start bfe012df85d8197e7e0b8ac16aa75f86ee3c57073d9c6453b2af22d3963f1806 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-i, GIT_CLEAN=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-16T09:13:22.902 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 09:13:22 smithi194 ceph-mds[68211]: -- 172.21.15.194:0/250476138 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555d37f27a40 con 0x555d37efb000 2024-03-16T09:13:22.902 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 09:13:22 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-i[68207]: starting mds.i at 2024-03-16T09:13:22.902 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 09:13:22 smithi194 bash[68192]: bfe012df85d8197e7e0b8ac16aa75f86ee3c57073d9c6453b2af22d3963f1806 2024-03-16T09:13:22.902 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 09:13:22 smithi194 systemd[1]: Started Ceph mds.i for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:13:23.734 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:13:23.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.836+0000 7f196a24b640 1 -- 172.21.15.112:0/445274147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 msgr2=0x7f1964108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:23.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.836+0000 7f196a24b640 1 --2- 172.21.15.112:0/445274147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 0x7f1964108b40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f195c009ca0 tx=0x7f195c02c0f0 comp rx=0 tx=0).stop 2024-03-16T09:13:23.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.837+0000 7f196a24b640 1 -- 172.21.15.112:0/445274147 shutdown_connections 2024-03-16T09:13:23.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.837+0000 7f196a24b640 1 --2- 172.21.15.112:0/445274147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 0x7f1964108b40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:23.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.837+0000 7f196a24b640 1 --2- 172.21.15.112:0/445274147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19641018f0 0x7f1964101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:23.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.837+0000 7f196a24b640 1 --2- 172.21.15.112:0/445274147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1964100700 0x7f1964100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:23.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.837+0000 7f196a24b640 1 -- 172.21.15.112:0/445274147 >> 172.21.15.112:0/445274147 conn(0x7f19640fbc70 msgr2=0x7f19640fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:23.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.837+0000 7f196a24b640 1 -- 172.21.15.112:0/445274147 shutdown_connections 2024-03-16T09:13:23.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.837+0000 7f196a24b640 1 -- 172.21.15.112:0/445274147 wait complete. 2024-03-16T09:13:23.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.837+0000 7f196a24b640 1 Processor -- start 2024-03-16T09:13:23.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.837+0000 7f196a24b640 1 -- start start 2024-03-16T09:13:23.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.837+0000 7f196a24b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1964100700 0x7f1964197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:23.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.837+0000 7f196a24b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19641018f0 0x7f1964198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:23.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.837+0000 7f196a24b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 0x7f196419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:23.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.837+0000 7f196a24b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f196410d5c0 con 0x7f19641022b0 2024-03-16T09:13:23.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.837+0000 7f196a24b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f196410d440 con 0x7f19641018f0 2024-03-16T09:13:23.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.837+0000 7f196a24b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f196410d740 con 0x7f1964100700 2024-03-16T09:13:23.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.838+0000 7f1963fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 0x7f196419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:23.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.838+0000 7f1963fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 0x7f196419f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41820/0 (socket says 172.21.15.112:41820) 2024-03-16T09:13:23.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.838+0000 7f1963fff640 1 -- 172.21.15.112:0/1092222405 learned_addr learned my addr 172.21.15.112:0/1092222405 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:23.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.838+0000 7f19637fe640 1 --2- 172.21.15.112:0/1092222405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1964100700 0x7f1964197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:23.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.838+0000 7f195affd640 1 --2- 172.21.15.112:0/1092222405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19641018f0 0x7f1964198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:23.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.838+0000 7f1963fff640 1 -- 172.21.15.112:0/1092222405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1964100700 msgr2=0x7f1964197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:23.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.838+0000 7f1963fff640 1 --2- 172.21.15.112:0/1092222405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1964100700 0x7f1964197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:23.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.838+0000 7f1963fff640 1 -- 172.21.15.112:0/1092222405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19641018f0 msgr2=0x7f1964198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:23.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.838+0000 7f1963fff640 1 --2- 172.21.15.112:0/1092222405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19641018f0 0x7f1964198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:23.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.838+0000 7f1963fff640 1 -- 172.21.15.112:0/1092222405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196419f770 con 0x7f19641022b0 2024-03-16T09:13:23.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.838+0000 7f1963fff640 1 --2- 172.21.15.112:0/1092222405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 0x7f196419f100 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f195c037040 tx=0x7f195c0346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:23.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.838+0000 7f19617fa640 1 -- 172.21.15.112:0/1092222405 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c039040 con 0x7f19641022b0 2024-03-16T09:13:23.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.838+0000 7f196a24b640 1 -- 172.21.15.112:0/1092222405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196419fa00 con 0x7f19641022b0 2024-03-16T09:13:23.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.838+0000 7f19617fa640 1 -- 172.21.15.112:0/1092222405 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f195c044070 con 0x7f19641022b0 2024-03-16T09:13:23.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.838+0000 7f19617fa640 1 -- 172.21.15.112:0/1092222405 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c0045a0 con 0x7f19641022b0 2024-03-16T09:13:23.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.838+0000 7f196a24b640 1 -- 172.21.15.112:0/1092222405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f196419ff40 con 0x7f19641022b0 2024-03-16T09:13:23.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.839+0000 7f196a24b640 1 -- 172.21.15.112:0/1092222405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1964107950 con 0x7f19641022b0 2024-03-16T09:13:23.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.840+0000 7f19617fa640 1 -- 172.21.15.112:0/1092222405 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f195c003970 con 0x7f19641022b0 2024-03-16T09:13:23.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.841+0000 7f19617fa640 1 --2- 172.21.15.112:0/1092222405 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f19500a2800 0x7f19500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:23.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.841+0000 7f19637fe640 1 --2- 172.21.15.112:0/1092222405 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f19500a2800 0x7f19500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:23.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.841+0000 7f19617fa640 1 -- 172.21.15.112:0/1092222405 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f195c0f13e0 con 0x7f19641022b0 2024-03-16T09:13:23.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.841+0000 7f19637fe640 1 --2- 172.21.15.112:0/1092222405 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f19500a2800 0x7f19500a4cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1964101750 tx=0x7f195400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:23.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:23.844+0000 7f19617fa640 1 -- 172.21.15.112:0/1092222405 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f195c0bea40 con 0x7f19641022b0 2024-03-16T09:13:24.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:24.057+0000 7f196a24b640 1 -- 172.21.15.112:0/1092222405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f196402d250 con 0x7f19641022b0 2024-03-16T09:13:24.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:24.057+0000 7f19617fa640 1 -- 172.21.15.112:0/1092222405 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7f195c0be340 con 0x7f19641022b0 2024-03-16T09:13:24.060 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:24.061 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1682076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":1682076},{"type":"v1","addr":"172.21.15.194:6837","nonce":1682076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/2528943309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2528943309},{"type":"v1","addr":"172.21.15.194:6839","nonce":2528943309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-16T09:13:24.061 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 10 2024-03-16T09:13:24.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:24.059+0000 7f196a24b640 1 -- 172.21.15.112:0/1092222405 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f19500a2800 msgr2=0x7f19500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:24.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:24.059+0000 7f196a24b640 1 --2- 172.21.15.112:0/1092222405 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f19500a2800 0x7f19500a4cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1964101750 tx=0x7f195400b390 comp rx=0 tx=0).stop 2024-03-16T09:13:24.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:24.059+0000 7f196a24b640 1 -- 172.21.15.112:0/1092222405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 msgr2=0x7f196419f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:24.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:24.059+0000 7f196a24b640 1 --2- 172.21.15.112:0/1092222405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 0x7f196419f100 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f195c037040 tx=0x7f195c0346d0 comp rx=0 tx=0).stop 2024-03-16T09:13:24.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:24.059+0000 7f196a24b640 1 -- 172.21.15.112:0/1092222405 shutdown_connections 2024-03-16T09:13:24.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:24.059+0000 7f196a24b640 1 --2- 172.21.15.112:0/1092222405 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f19500a2800 0x7f19500a4cc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:24.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:24.059+0000 7f196a24b640 1 --2- 172.21.15.112:0/1092222405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 0x7f196419f100 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:24.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:24.059+0000 7f196a24b640 1 --2- 172.21.15.112:0/1092222405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19641018f0 0x7f1964198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:24.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:24.059+0000 7f196a24b640 1 --2- 172.21.15.112:0/1092222405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1964100700 0x7f1964197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:24.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:24.059+0000 7f196a24b640 1 -- 172.21.15.112:0/1092222405 >> 172.21.15.112:0/1092222405 conn(0x7f19640fbc70 msgr2=0x7f1964104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:24.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:24.059+0000 7f196a24b640 1 -- 172.21.15.112:0/1092222405 shutdown_connections 2024-03-16T09:13:24.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:24.060+0000 7f196a24b640 1 -- 172.21.15.112:0/1092222405 wait complete. 2024-03-16T09:13:24.072 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:23 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:24.072 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:23 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:24.072 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:23 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:24.072 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:23 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:24.072 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:23 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:24.072 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:23 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:24.072 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:23 smithi112 ceph-mon[32307]: Deploying daemon mds.g on smithi112 2024-03-16T09:13:24.073 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:23 smithi112 ceph-mon[32307]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:24.073 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:23 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] up:boot 2024-03-16T09:13:24.073 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:23 smithi112 ceph-mon[32307]: fsmap 9 up:standby 2024-03-16T09:13:24.073 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:23 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T09:13:24.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:23 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:24.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:23 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:24.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:23 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:24.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:23 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:24.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:23 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:24.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:23 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:24.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:23 smithi194 ceph-mon[36729]: Deploying daemon mds.g on smithi112 2024-03-16T09:13:24.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:23 smithi194 ceph-mon[36729]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:24.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:23 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] up:boot 2024-03-16T09:13:24.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:23 smithi194 ceph-mon[36729]: fsmap 9 up:standby 2024-03-16T09:13:24.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:23 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T09:13:24.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:23 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:24.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:23 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:24.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:23 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:24.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:23 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:24.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:23 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:24.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:23 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:24.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:23 smithi159 ceph-mon[36739]: Deploying daemon mds.g on smithi112 2024-03-16T09:13:24.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:23 smithi159 ceph-mon[36739]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:24.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:23 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] up:boot 2024-03-16T09:13:24.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:23 smithi159 ceph-mon[36739]: fsmap 9 up:standby 2024-03-16T09:13:24.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:23 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T09:13:25.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:24 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1092222405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:25.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:24 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1092222405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:25.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:24 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1092222405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:25.809 INFO:journalctl@ceph.mds.g.smithi112.stdout:Mar 16 09:13:25 smithi112 systemd[1]: Started Ceph mds.g for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:13:25.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:25 smithi159 ceph-mon[36739]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:25.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:25 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:25.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:25 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:25.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:25 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:25.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:25 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:25.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:25 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:25.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:25 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:26.075 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:13:26.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:25 smithi112 ceph-mon[32307]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:26.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:25 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:26.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:25 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:26.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:25 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:26.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:25 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:26.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:25 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:26.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:25 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:26.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:25 smithi194 ceph-mon[36729]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:26.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:25 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:26.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:25 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:26.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:25 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:26.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:25 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:26.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:25 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:26.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:25 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:26.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.178+0000 7f405a1d3640 1 -- 172.21.15.112:0/2281066901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40541022b0 msgr2=0x7f4054108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:26.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.178+0000 7f405a1d3640 1 --2- 172.21.15.112:0/2281066901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40541022b0 0x7f4054108b40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f404c009ca0 tx=0x7f404c02c0f0 comp rx=0 tx=0).stop 2024-03-16T09:13:26.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.178+0000 7f405a1d3640 1 -- 172.21.15.112:0/2281066901 shutdown_connections 2024-03-16T09:13:26.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.178+0000 7f405a1d3640 1 --2- 172.21.15.112:0/2281066901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40541022b0 0x7f4054108b40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:26.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.178+0000 7f405a1d3640 1 --2- 172.21.15.112:0/2281066901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40541018f0 0x7f4054101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:26.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.178+0000 7f405a1d3640 1 --2- 172.21.15.112:0/2281066901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4054100700 0x7f4054100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:26.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.178+0000 7f405a1d3640 1 -- 172.21.15.112:0/2281066901 >> 172.21.15.112:0/2281066901 conn(0x7f40540fbc70 msgr2=0x7f40540fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:26.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.178+0000 7f405a1d3640 1 -- 172.21.15.112:0/2281066901 shutdown_connections 2024-03-16T09:13:26.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.178+0000 7f405a1d3640 1 -- 172.21.15.112:0/2281066901 wait complete. 2024-03-16T09:13:26.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.179+0000 7f405a1d3640 1 Processor -- start 2024-03-16T09:13:26.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.179+0000 7f405a1d3640 1 -- start start 2024-03-16T09:13:26.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.179+0000 7f405a1d3640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4054100700 0x7f4054196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:26.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.179+0000 7f405a1d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40541018f0 0x7f4054196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:26.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.179+0000 7f405a1d3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40541022b0 0x7f405419dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:26.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.179+0000 7f405a1d3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f405410c4a0 con 0x7f40541018f0 2024-03-16T09:13:26.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.179+0000 7f405a1d3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f405410c320 con 0x7f40541022b0 2024-03-16T09:13:26.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.179+0000 7f405a1d3640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f405410c620 con 0x7f4054100700 2024-03-16T09:13:26.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.179+0000 7f4052ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40541018f0 0x7f4054196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:26.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.179+0000 7f4052ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40541018f0 0x7f4054196f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41872/0 (socket says 172.21.15.112:41872) 2024-03-16T09:13:26.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.179+0000 7f4052ffd640 1 -- 172.21.15.112:0/2887186819 learned_addr learned my addr 172.21.15.112:0/2887186819 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:26.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.179+0000 7f40537fe640 1 --2- 172.21.15.112:0/2887186819 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4054100700 0x7f4054196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:26.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.179+0000 7f4053fff640 1 --2- 172.21.15.112:0/2887186819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40541022b0 0x7f405419dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:26.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.179+0000 7f4052ffd640 1 -- 172.21.15.112:0/2887186819 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4054100700 msgr2=0x7f4054196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:26.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.179+0000 7f4052ffd640 1 --2- 172.21.15.112:0/2887186819 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4054100700 0x7f4054196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:26.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.179+0000 7f4052ffd640 1 -- 172.21.15.112:0/2887186819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40541022b0 msgr2=0x7f405419dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:26.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.179+0000 7f4052ffd640 1 --2- 172.21.15.112:0/2887186819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40541022b0 0x7f405419dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:26.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.179+0000 7f4052ffd640 1 -- 172.21.15.112:0/2887186819 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405419e620 con 0x7f40541018f0 2024-03-16T09:13:26.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.180+0000 7f4052ffd640 1 --2- 172.21.15.112:0/2887186819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40541018f0 0x7f4054196f40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f404400d950 tx=0x7f404400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:26.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.180+0000 7f4050ff9640 1 -- 172.21.15.112:0/2887186819 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044017070 con 0x7f40541018f0 2024-03-16T09:13:26.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.180+0000 7f4050ff9640 1 -- 172.21.15.112:0/2887186819 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4044009c30 con 0x7f40541018f0 2024-03-16T09:13:26.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.180+0000 7f405a1d3640 1 -- 172.21.15.112:0/2887186819 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f405419e910 con 0x7f40541018f0 2024-03-16T09:13:26.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.180+0000 7f4050ff9640 1 -- 172.21.15.112:0/2887186819 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404400f740 con 0x7f40541018f0 2024-03-16T09:13:26.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.180+0000 7f405a1d3640 1 -- 172.21.15.112:0/2887186819 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f405419ee60 con 0x7f40541018f0 2024-03-16T09:13:26.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.181+0000 7f405a1d3640 1 -- 172.21.15.112:0/2887186819 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4054107950 con 0x7f40541018f0 2024-03-16T09:13:26.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.182+0000 7f4050ff9640 1 -- 172.21.15.112:0/2887186819 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f4044010a80 con 0x7f40541018f0 2024-03-16T09:13:26.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.183+0000 7f4050ff9640 1 --2- 172.21.15.112:0/2887186819 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f403c0a2800 0x7f403c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:26.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.183+0000 7f40537fe640 1 --2- 172.21.15.112:0/2887186819 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f403c0a2800 0x7f403c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:26.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.183+0000 7f4050ff9640 1 -- 172.21.15.112:0/2887186819 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f40440ce2b0 con 0x7f40541018f0 2024-03-16T09:13:26.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.183+0000 7f40537fe640 1 --2- 172.21.15.112:0/2887186819 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f403c0a2800 0x7f403c0a4cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4054101750 tx=0x7f4048005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:26.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.187+0000 7f4050ff9640 1 -- 172.21.15.112:0/2887186819 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4044009de0 con 0x7f40541018f0 2024-03-16T09:13:26.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.427+0000 7f405a1d3640 1 -- 172.21.15.112:0/2887186819 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f405402d410 con 0x7f40541018f0 2024-03-16T09:13:26.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.428+0000 7f4050ff9640 1 -- 172.21.15.112:0/2887186819 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8839 (secure 0 0 0) 0x7f404409b7f0 con 0x7f40541018f0 2024-03-16T09:13:26.431 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:26.431 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3939748854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3939748854},{"type":"v1","addr":"172.21.15.112:6839","nonce":3939748854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1682076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":1682076},{"type":"v1","addr":"172.21.15.194:6837","nonce":1682076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/2528943309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2528943309},{"type":"v1","addr":"172.21.15.194:6839","nonce":2528943309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-16T09:13:26.431 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 11 2024-03-16T09:13:26.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.430+0000 7f405a1d3640 1 -- 172.21.15.112:0/2887186819 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f403c0a2800 msgr2=0x7f403c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:26.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.430+0000 7f405a1d3640 1 --2- 172.21.15.112:0/2887186819 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f403c0a2800 0x7f403c0a4cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4054101750 tx=0x7f4048005f90 comp rx=0 tx=0).stop 2024-03-16T09:13:26.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.430+0000 7f405a1d3640 1 -- 172.21.15.112:0/2887186819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40541018f0 msgr2=0x7f4054196f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:26.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.430+0000 7f405a1d3640 1 --2- 172.21.15.112:0/2887186819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40541018f0 0x7f4054196f40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f404400d950 tx=0x7f404400de10 comp rx=0 tx=0).stop 2024-03-16T09:13:26.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.430+0000 7f405a1d3640 1 -- 172.21.15.112:0/2887186819 shutdown_connections 2024-03-16T09:13:26.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.430+0000 7f405a1d3640 1 --2- 172.21.15.112:0/2887186819 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f403c0a2800 0x7f403c0a4cc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:26.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.430+0000 7f405a1d3640 1 --2- 172.21.15.112:0/2887186819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40541022b0 0x7f405419dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:26.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.430+0000 7f405a1d3640 1 --2- 172.21.15.112:0/2887186819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40541018f0 0x7f4054196f40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:26.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.430+0000 7f405a1d3640 1 --2- 172.21.15.112:0/2887186819 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4054100700 0x7f4054196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:26.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.430+0000 7f405a1d3640 1 -- 172.21.15.112:0/2887186819 >> 172.21.15.112:0/2887186819 conn(0x7f40540fbc70 msgr2=0x7f4054104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:26.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.430+0000 7f405a1d3640 1 -- 172.21.15.112:0/2887186819 shutdown_connections 2024-03-16T09:13:26.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:26.430+0000 7f405a1d3640 1 -- 172.21.15.112:0/2887186819 wait complete. 2024-03-16T09:13:27.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:26 smithi112 ceph-mon[32307]: Deploying daemon mds.a on smithi112 2024-03-16T09:13:27.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:26 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.112:6838/3939748854,v1:172.21.15.112:6839/3939748854] up:boot 2024-03-16T09:13:27.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:26 smithi112 ceph-mon[32307]: fsmap 10 up:standby 2024-03-16T09:13:27.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:26 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-16T09:13:27.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:26 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2887186819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:27.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:26 smithi194 ceph-mon[36729]: Deploying daemon mds.a on smithi112 2024-03-16T09:13:27.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:26 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.112:6838/3939748854,v1:172.21.15.112:6839/3939748854] up:boot 2024-03-16T09:13:27.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:26 smithi194 ceph-mon[36729]: fsmap 10 up:standby 2024-03-16T09:13:27.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:26 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-16T09:13:27.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:26 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2887186819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:27.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:26 smithi159 ceph-mon[36739]: Deploying daemon mds.a on smithi112 2024-03-16T09:13:27.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:26 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.112:6838/3939748854,v1:172.21.15.112:6839/3939748854] up:boot 2024-03-16T09:13:27.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:26 smithi159 ceph-mon[36739]: fsmap 10 up:standby 2024-03-16T09:13:27.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:26 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-16T09:13:27.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:26 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2887186819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:28.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:27 smithi112 ceph-mon[32307]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:28.093 INFO:journalctl@ceph.mds.a.smithi112.stdout:Mar 16 09:13:27 smithi112 systemd[1]: Starting Ceph mds.a for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:13:28.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:27 smithi194 ceph-mon[36729]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:28.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:27 smithi159 ceph-mon[36739]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:28.391 INFO:journalctl@ceph.mds.a.smithi112.stdout:Mar 16 09:13:28 smithi112 podman[74116]: 2024-03-16 09:13:28.02202417 +0000 UTC m=+0.017688769 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:13:28.391 INFO:journalctl@ceph.mds.a.smithi112.stdout:Mar 16 09:13:28 smithi112 podman[74116]: 2024-03-16 09:13:28.128083125 +0000 UTC m=+0.123747725 container create 5b2e1e3e3a94d17fdb14bc25c39a24d53fdbbb282daecd0aff0db78b00a3465a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-16T09:13:28.391 INFO:journalctl@ceph.mds.a.smithi112.stdout:Mar 16 09:13:28 smithi112 podman[74116]: 2024-03-16 09:13:28.364097029 +0000 UTC m=+0.359761630 container init 5b2e1e3e3a94d17fdb14bc25c39a24d53fdbbb282daecd0aff0db78b00a3465a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:13:28.392 INFO:journalctl@ceph.mds.a.smithi112.stdout:Mar 16 09:13:28 smithi112 podman[74116]: 2024-03-16 09:13:28.36665796 +0000 UTC m=+0.362322562 container start 5b2e1e3e3a94d17fdb14bc25c39a24d53fdbbb282daecd0aff0db78b00a3465a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-a, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-16T09:13:28.444 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:13:28.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.550+0000 7f5f8220b640 1 -- 172.21.15.112:0/3407811713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7c107560 msgr2=0x7f5f7c109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:28.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.550+0000 7f5f8220b640 1 --2- 172.21.15.112:0/3407811713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7c107560 0x7f5f7c109950 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5f74009ca0 tx=0x7f5f7402c0f0 comp rx=0 tx=0).stop 2024-03-16T09:13:28.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.550+0000 7f5f8220b640 1 -- 172.21.15.112:0/3407811713 shutdown_connections 2024-03-16T09:13:28.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.550+0000 7f5f8220b640 1 --2- 172.21.15.112:0/3407811713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7c107560 0x7f5f7c109950 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:28.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.550+0000 7f5f8220b640 1 --2- 172.21.15.112:0/3407811713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f7c104c30 0x7f5f7c107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:28.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.550+0000 7f5f8220b640 1 --2- 172.21.15.112:0/3407811713 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f7c102300 0x7f5f7c1046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:28.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.550+0000 7f5f8220b640 1 -- 172.21.15.112:0/3407811713 >> 172.21.15.112:0/3407811713 conn(0x7f5f7c0fbc70 msgr2=0x7f5f7c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:28.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.550+0000 7f5f8220b640 1 -- 172.21.15.112:0/3407811713 shutdown_connections 2024-03-16T09:13:28.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.550+0000 7f5f8220b640 1 -- 172.21.15.112:0/3407811713 wait complete. 2024-03-16T09:13:28.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.551+0000 7f5f8220b640 1 Processor -- start 2024-03-16T09:13:28.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.551+0000 7f5f8220b640 1 -- start start 2024-03-16T09:13:28.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.551+0000 7f5f8220b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7c102300 0x7f5f7c101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:28.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.551+0000 7f5f8220b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f7c104c30 0x7f5f7c1002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:28.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.551+0000 7f5f8220b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f7c107560 0x7f5f7c1007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:28.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.551+0000 7f5f8220b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c10d5b0 con 0x7f5f7c102300 2024-03-16T09:13:28.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.551+0000 7f5f8220b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c10d430 con 0x7f5f7c104c30 2024-03-16T09:13:28.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.551+0000 7f5f8220b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7c10d730 con 0x7f5f7c107560 2024-03-16T09:13:28.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.551+0000 7f5f7b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7c102300 0x7f5f7c101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:28.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.551+0000 7f5f7b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7c102300 0x7f5f7c101c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41902/0 (socket says 172.21.15.112:41902) 2024-03-16T09:13:28.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.551+0000 7f5f7b7fe640 1 -- 172.21.15.112:0/1772360953 learned_addr learned my addr 172.21.15.112:0/1772360953 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:28.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.551+0000 7f5f7affd640 1 --2- 172.21.15.112:0/1772360953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f7c104c30 0x7f5f7c1002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:28.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.551+0000 7f5f7bfff640 1 --2- 172.21.15.112:0/1772360953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f7c107560 0x7f5f7c1007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:28.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.552+0000 7f5f7b7fe640 1 -- 172.21.15.112:0/1772360953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f7c107560 msgr2=0x7f5f7c1007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:28.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.552+0000 7f5f7b7fe640 1 --2- 172.21.15.112:0/1772360953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f7c107560 0x7f5f7c1007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:28.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.552+0000 7f5f7b7fe640 1 -- 172.21.15.112:0/1772360953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f7c104c30 msgr2=0x7f5f7c1002b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:28.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.552+0000 7f5f7b7fe640 1 --2- 172.21.15.112:0/1772360953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f7c104c30 0x7f5f7c1002b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:28.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.552+0000 7f5f7b7fe640 1 -- 172.21.15.112:0/1772360953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7c101080 con 0x7f5f7c102300 2024-03-16T09:13:28.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.552+0000 7f5f7bfff640 1 --2- 172.21.15.112:0/1772360953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f7c107560 0x7f5f7c1007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:13:28.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.552+0000 7f5f7b7fe640 1 --2- 172.21.15.112:0/1772360953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7c102300 0x7f5f7c101c70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00d950 tx=0x7f5f6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:28.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.553+0000 7f5f78ff9640 1 -- 172.21.15.112:0/1772360953 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c017030 con 0x7f5f7c102300 2024-03-16T09:13:28.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.553+0000 7f5f78ff9640 1 -- 172.21.15.112:0/1772360953 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5f6c004020 con 0x7f5f7c102300 2024-03-16T09:13:28.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.553+0000 7f5f8220b640 1 -- 172.21.15.112:0/1772360953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f7c1a1cd0 con 0x7f5f7c102300 2024-03-16T09:13:28.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.553+0000 7f5f78ff9640 1 -- 172.21.15.112:0/1772360953 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c004a00 con 0x7f5f7c102300 2024-03-16T09:13:28.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.553+0000 7f5f8220b640 1 -- 172.21.15.112:0/1772360953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f7c1a21d0 con 0x7f5f7c102300 2024-03-16T09:13:28.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.554+0000 7f5f8220b640 1 -- 172.21.15.112:0/1772360953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f7c0fd760 con 0x7f5f7c102300 2024-03-16T09:13:28.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.555+0000 7f5f78ff9640 1 -- 172.21.15.112:0/1772360953 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f5f6c0041d0 con 0x7f5f7c102300 2024-03-16T09:13:28.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.555+0000 7f5f78ff9640 1 --2- 172.21.15.112:0/1772360953 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5f640a2800 0x7f5f640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:28.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.556+0000 7f5f7affd640 1 --2- 172.21.15.112:0/1772360953 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5f640a2800 0x7f5f640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:28.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.556+0000 7f5f78ff9640 1 -- 172.21.15.112:0/1772360953 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5f6c0ce170 con 0x7f5f7c102300 2024-03-16T09:13:28.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.556+0000 7f5f7affd640 1 --2- 172.21.15.112:0/1772360953 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5f640a2800 0x7f5f640a4cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5f70005e00 tx=0x7f5f70005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:28.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.560+0000 7f5f78ff9640 1 -- 172.21.15.112:0/1772360953 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5f6c09b6b0 con 0x7f5f7c102300 2024-03-16T09:13:28.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.775+0000 7f5f8220b640 1 -- 172.21.15.112:0/1772360953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f7c02d1b0 con 0x7f5f7c102300 2024-03-16T09:13:28.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.776+0000 7f5f78ff9640 1 -- 172.21.15.112:0/1772360953 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8839 (secure 0 0 0) 0x7f5f6c09afb0 con 0x7f5f7c102300 2024-03-16T09:13:28.779 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:28.779 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3939748854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3939748854},{"type":"v1","addr":"172.21.15.112:6839","nonce":3939748854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1682076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":1682076},{"type":"v1","addr":"172.21.15.194:6837","nonce":1682076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/2528943309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2528943309},{"type":"v1","addr":"172.21.15.194:6839","nonce":2528943309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-16T09:13:28.779 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 11 2024-03-16T09:13:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.778+0000 7f5f8220b640 1 -- 172.21.15.112:0/1772360953 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5f640a2800 msgr2=0x7f5f640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.778+0000 7f5f8220b640 1 --2- 172.21.15.112:0/1772360953 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5f640a2800 0x7f5f640a4cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5f70005e00 tx=0x7f5f70005c50 comp rx=0 tx=0).stop 2024-03-16T09:13:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.778+0000 7f5f8220b640 1 -- 172.21.15.112:0/1772360953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7c102300 msgr2=0x7f5f7c101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.778+0000 7f5f8220b640 1 --2- 172.21.15.112:0/1772360953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7c102300 0x7f5f7c101c70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00d950 tx=0x7f5f6c00de10 comp rx=0 tx=0).stop 2024-03-16T09:13:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.778+0000 7f5f8220b640 1 -- 172.21.15.112:0/1772360953 shutdown_connections 2024-03-16T09:13:28.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.778+0000 7f5f8220b640 1 --2- 172.21.15.112:0/1772360953 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5f640a2800 0x7f5f640a4cc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.778+0000 7f5f8220b640 1 --2- 172.21.15.112:0/1772360953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f7c107560 0x7f5f7c1007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.778+0000 7f5f8220b640 1 --2- 172.21.15.112:0/1772360953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f7c104c30 0x7f5f7c1002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.778+0000 7f5f8220b640 1 --2- 172.21.15.112:0/1772360953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f7c102300 0x7f5f7c101c70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.778+0000 7f5f8220b640 1 -- 172.21.15.112:0/1772360953 >> 172.21.15.112:0/1772360953 conn(0x7f5f7c0fbc70 msgr2=0x7f5f7c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.778+0000 7f5f8220b640 1 -- 172.21.15.112:0/1772360953 shutdown_connections 2024-03-16T09:13:28.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:28.778+0000 7f5f8220b640 1 -- 172.21.15.112:0/1772360953 wait complete. 2024-03-16T09:13:28.791 INFO:journalctl@ceph.mds.a.smithi112.stdout:Mar 16 09:13:28 smithi112 ceph-mds[74136]: -- 172.21.15.112:0/4272813397 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56280185da40 con 0x562801830800 2024-03-16T09:13:28.791 INFO:journalctl@ceph.mds.a.smithi112.stdout:Mar 16 09:13:28 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-a[74132]: starting mds.a at 2024-03-16T09:13:28.791 INFO:journalctl@ceph.mds.a.smithi112.stdout:Mar 16 09:13:28 smithi112 bash[74116]: 5b2e1e3e3a94d17fdb14bc25c39a24d53fdbbb282daecd0aff0db78b00a3465a 2024-03-16T09:13:28.791 INFO:journalctl@ceph.mds.a.smithi112.stdout:Mar 16 09:13:28 smithi112 systemd[1]: Started Ceph mds.a for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:13:29.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:29 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:29.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:29 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:29.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:29 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:29.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:29 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:29.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:29 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:29.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:29 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:29.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:29 smithi159 ceph-mon[36739]: Deploying daemon mds.b on smithi159 2024-03-16T09:13:29.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:29 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1772360953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:29.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:29 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.112:6840/3118523258,v1:172.21.15.112:6841/3118523258] up:boot 2024-03-16T09:13:29.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:29 smithi159 ceph-mon[36739]: fsmap 11 up:standby 2024-03-16T09:13:29.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:29 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-16T09:13:29.737 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:29 smithi159 ceph-mon[36739]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:29.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:29.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:29.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:29.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:29.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:29.843 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:29.843 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:29 smithi112 ceph-mon[32307]: Deploying daemon mds.b on smithi159 2024-03-16T09:13:29.843 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:29 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1772360953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:29.843 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:29 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.112:6840/3118523258,v1:172.21.15.112:6841/3118523258] up:boot 2024-03-16T09:13:29.843 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:29 smithi112 ceph-mon[32307]: fsmap 11 up:standby 2024-03-16T09:13:29.843 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:29 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-16T09:13:29.843 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:29 smithi112 ceph-mon[32307]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:29.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:29.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:29.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:29.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-16T09:13:29.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-16T09:13:29.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:29.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:29 smithi194 ceph-mon[36729]: Deploying daemon mds.b on smithi159 2024-03-16T09:13:29.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:29 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1772360953' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:29.903 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:29 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.112:6840/3118523258,v1:172.21.15.112:6841/3118523258] up:boot 2024-03-16T09:13:29.903 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:29 smithi194 ceph-mon[36729]: fsmap 11 up:standby 2024-03-16T09:13:29.903 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:29 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-16T09:13:29.903 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:29 smithi194 ceph-mon[36729]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:30.793 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:13:30.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.905+0000 7fc335732640 1 -- 172.21.15.112:0/1651919035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc330101800 msgr2=0x7fc330101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:30.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.905+0000 7fc335732640 1 --2- 172.21.15.112:0/1651919035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc330101800 0x7fc330101c20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc320009900 tx=0x7fc32002bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:30.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.905+0000 7fc335732640 1 -- 172.21.15.112:0/1651919035 shutdown_connections 2024-03-16T09:13:30.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.905+0000 7fc335732640 1 --2- 172.21.15.112:0/1651919035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3301033b0 0x7fc330109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:30.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.905+0000 7fc335732640 1 --2- 172.21.15.112:0/1651919035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3301029f0 0x7fc330102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:30.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.905+0000 7fc335732640 1 --2- 172.21.15.112:0/1651919035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc330101800 0x7fc330101c20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:30.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.905+0000 7fc335732640 1 -- 172.21.15.112:0/1651919035 >> 172.21.15.112:0/1651919035 conn(0x7fc3300fcd90 msgr2=0x7fc3300ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:30.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.905+0000 7fc335732640 1 -- 172.21.15.112:0/1651919035 shutdown_connections 2024-03-16T09:13:30.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.905+0000 7fc335732640 1 -- 172.21.15.112:0/1651919035 wait complete. 2024-03-16T09:13:30.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.906+0000 7fc335732640 1 Processor -- start 2024-03-16T09:13:30.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.906+0000 7fc335732640 1 -- start start 2024-03-16T09:13:30.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.906+0000 7fc335732640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc330101800 0x7fc330197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:30.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.906+0000 7fc335732640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3301029f0 0x7fc330198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:30.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.906+0000 7fc335732640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3301033b0 0x7fc33019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:30.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.906+0000 7fc335732640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc33010d550 con 0x7fc3301033b0 2024-03-16T09:13:30.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.906+0000 7fc335732640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc33010d3d0 con 0x7fc330101800 2024-03-16T09:13:30.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.906+0000 7fc335732640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc33010d6d0 con 0x7fc3301029f0 2024-03-16T09:13:30.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.906+0000 7fc32f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3301033b0 0x7fc33019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:30.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.906+0000 7fc32f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3301033b0 0x7fc33019f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41924/0 (socket says 172.21.15.112:41924) 2024-03-16T09:13:30.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.906+0000 7fc32f7fe640 1 -- 172.21.15.112:0/3240387589 learned_addr learned my addr 172.21.15.112:0/3240387589 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:30.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.906+0000 7fc32effd640 1 --2- 172.21.15.112:0/3240387589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc330101800 0x7fc330197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:30.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.907+0000 7fc32f7fe640 1 -- 172.21.15.112:0/3240387589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3301029f0 msgr2=0x7fc330198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:30.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.907+0000 7fc32f7fe640 1 --2- 172.21.15.112:0/3240387589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3301029f0 0x7fc330198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:30.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.907+0000 7fc32f7fe640 1 -- 172.21.15.112:0/3240387589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc330101800 msgr2=0x7fc330197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:30.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.907+0000 7fc32f7fe640 1 --2- 172.21.15.112:0/3240387589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc330101800 0x7fc330197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:30.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.907+0000 7fc32f7fe640 1 -- 172.21.15.112:0/3240387589 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33019f830 con 0x7fc3301033b0 2024-03-16T09:13:30.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.907+0000 7fc32effd640 1 --2- 172.21.15.112:0/3240387589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc330101800 0x7fc330197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:13:30.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.907+0000 7fc32f7fe640 1 --2- 172.21.15.112:0/3240387589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3301033b0 0x7fc33019f190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc32800cce0 tx=0x7fc32800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:30.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.907+0000 7fc32cff9640 1 -- 172.21.15.112:0/3240387589 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32800f450 con 0x7fc3301033b0 2024-03-16T09:13:30.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.907+0000 7fc32cff9640 1 -- 172.21.15.112:0/3240387589 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc32800faf0 con 0x7fc3301033b0 2024-03-16T09:13:30.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.907+0000 7fc335732640 1 -- 172.21.15.112:0/3240387589 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc33019fb20 con 0x7fc3301033b0 2024-03-16T09:13:30.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.907+0000 7fc32cff9640 1 -- 172.21.15.112:0/3240387589 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328018cc0 con 0x7fc3301033b0 2024-03-16T09:13:30.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.907+0000 7fc335732640 1 -- 172.21.15.112:0/3240387589 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3301a0020 con 0x7fc3301033b0 2024-03-16T09:13:30.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.908+0000 7fc335732640 1 -- 172.21.15.112:0/3240387589 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc330108ac0 con 0x7fc3301033b0 2024-03-16T09:13:30.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.909+0000 7fc32cff9640 1 -- 172.21.15.112:0/3240387589 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fc32800f5f0 con 0x7fc3301033b0 2024-03-16T09:13:30.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.910+0000 7fc32cff9640 1 --2- 172.21.15.112:0/3240387589 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc31c0a2800 0x7fc31c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:30.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.910+0000 7fc32effd640 1 --2- 172.21.15.112:0/3240387589 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc31c0a2800 0x7fc31c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:30.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.910+0000 7fc32cff9640 1 -- 172.21.15.112:0/3240387589 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc3280cec80 con 0x7fc3301033b0 2024-03-16T09:13:30.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.910+0000 7fc32effd640 1 --2- 172.21.15.112:0/3240387589 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc31c0a2800 0x7fc31c0a4cc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc320009230 tx=0x7fc32002c9d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:30.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:30.913+0000 7fc32cff9640 1 -- 172.21.15.112:0/3240387589 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc328014030 con 0x7fc3301033b0 2024-03-16T09:13:31.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:31.125+0000 7fc335732640 1 -- 172.21.15.112:0/3240387589 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc33002d1b0 con 0x7fc3301033b0 2024-03-16T09:13:31.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:31.126+0000 7fc32cff9640 1 -- 172.21.15.112:0/3240387589 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9668 (secure 0 0 0) 0x7fc32809c1c0 con 0x7fc3301033b0 2024-03-16T09:13:31.129 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:31.129 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3939748854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3939748854},{"type":"v1","addr":"172.21.15.112:6839","nonce":3939748854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3118523258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3118523258},{"type":"v1","addr":"172.21.15.112:6841","nonce":3118523258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1682076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":1682076},{"type":"v1","addr":"172.21.15.194:6837","nonce":1682076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/2528943309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2528943309},{"type":"v1","addr":"172.21.15.194:6839","nonce":2528943309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-16T09:13:31.129 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 12 2024-03-16T09:13:31.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:31.128+0000 7fc335732640 1 -- 172.21.15.112:0/3240387589 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc31c0a2800 msgr2=0x7fc31c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:31.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:31.128+0000 7fc335732640 1 --2- 172.21.15.112:0/3240387589 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc31c0a2800 0x7fc31c0a4cc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc320009230 tx=0x7fc32002c9d0 comp rx=0 tx=0).stop 2024-03-16T09:13:31.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:31.128+0000 7fc335732640 1 -- 172.21.15.112:0/3240387589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3301033b0 msgr2=0x7fc33019f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:31.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:31.128+0000 7fc335732640 1 --2- 172.21.15.112:0/3240387589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3301033b0 0x7fc33019f190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc32800cce0 tx=0x7fc32800a620 comp rx=0 tx=0).stop 2024-03-16T09:13:31.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:31.128+0000 7fc335732640 1 -- 172.21.15.112:0/3240387589 shutdown_connections 2024-03-16T09:13:31.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:31.128+0000 7fc335732640 1 --2- 172.21.15.112:0/3240387589 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc31c0a2800 0x7fc31c0a4cc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:31.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:31.128+0000 7fc335732640 1 --2- 172.21.15.112:0/3240387589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3301033b0 0x7fc33019f190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:31.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:31.128+0000 7fc335732640 1 --2- 172.21.15.112:0/3240387589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3301029f0 0x7fc330198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:31.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:31.128+0000 7fc335732640 1 --2- 172.21.15.112:0/3240387589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc330101800 0x7fc330197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:31.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:31.128+0000 7fc335732640 1 -- 172.21.15.112:0/3240387589 >> 172.21.15.112:0/3240387589 conn(0x7fc3300fcd90 msgr2=0x7fc330105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:31.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:31.128+0000 7fc335732640 1 -- 172.21.15.112:0/3240387589 shutdown_connections 2024-03-16T09:13:31.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:31.128+0000 7fc335732640 1 -- 172.21.15.112:0/3240387589 wait complete. 2024-03-16T09:13:31.157 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:13:31 smithi159 podman[66489]: 2024-03-16 09:13:31.019417288 +0000 UTC m=+0.017740908 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:13:31.157 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:13:31 smithi159 podman[66489]: 2024-03-16 09:13:31.156221021 +0000 UTC m=+0.154544631 container create 404cd1638c026731e0a48e118967429336a2f3ba033c5698f950644b61a80aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-b, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:13:31.736 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:13:31 smithi159 podman[66489]: 2024-03-16 09:13:31.409038581 +0000 UTC m=+0.407362190 container init 404cd1638c026731e0a48e118967429336a2f3ba033c5698f950644b61a80aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-16T09:13:31.736 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:13:31 smithi159 podman[66489]: 2024-03-16 09:13:31.411606696 +0000 UTC m=+0.409930306 container start 404cd1638c026731e0a48e118967429336a2f3ba033c5698f950644b61a80aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-16T09:13:31.737 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:13:31 smithi159 ceph-mds[66508]: -- 172.21.15.159:0/3759586665 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5632c1ea9a40 con 0x5632c1e7cc00 2024-03-16T09:13:31.737 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:13:31 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-b[66504]: starting mds.b at 2024-03-16T09:13:31.737 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:13:31 smithi159 bash[66489]: 404cd1638c026731e0a48e118967429336a2f3ba033c5698f950644b61a80aa9 2024-03-16T09:13:31.737 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:13:31 smithi159 systemd[1]: Started Ceph mds.b for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:13:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:31 smithi159 ceph-mon[36739]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:31 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3240387589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:32.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:31 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:13:32.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:31 smithi112 ceph-mon[32307]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:32.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:31 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3240387589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:32.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:32.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:32.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:32.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:32.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:32.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:31 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:13:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:31 smithi194 ceph-mon[36729]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:31 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3240387589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:31 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:13:33.143 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:13:33.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:32 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] up:boot 2024-03-16T09:13:33.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:32 smithi159 ceph-mon[36739]: fsmap 12 up:standby 2024-03-16T09:13:33.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:32 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T09:13:33.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.256+0000 7f00ab0b2640 1 -- 172.21.15.112:0/3329493503 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00a4100780 msgr2=0x7f00a4100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:33.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.256+0000 7f00ab0b2640 1 --2- 172.21.15.112:0/3329493503 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00a4100780 0x7f00a4100ba0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0098009900 tx=0x7f009802bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:33.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.256+0000 7f00ab0b2640 1 -- 172.21.15.112:0/3329493503 shutdown_connections 2024-03-16T09:13:33.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.256+0000 7f00ab0b2640 1 --2- 172.21.15.112:0/3329493503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00a4102330 0x7f00a4108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.256+0000 7f00ab0b2640 1 --2- 172.21.15.112:0/3329493503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00a4101970 0x7f00a4101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.256+0000 7f00ab0b2640 1 --2- 172.21.15.112:0/3329493503 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00a4100780 0x7f00a4100ba0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.256+0000 7f00ab0b2640 1 -- 172.21.15.112:0/3329493503 >> 172.21.15.112:0/3329493503 conn(0x7f00a40fbd10 msgr2=0x7f00a40fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:33.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.256+0000 7f00ab0b2640 1 -- 172.21.15.112:0/3329493503 shutdown_connections 2024-03-16T09:13:33.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.256+0000 7f00ab0b2640 1 -- 172.21.15.112:0/3329493503 wait complete. 2024-03-16T09:13:33.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.258+0000 7f00ab0b2640 1 Processor -- start 2024-03-16T09:13:33.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.258+0000 7f00ab0b2640 1 -- start start 2024-03-16T09:13:33.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.258+0000 7f00ab0b2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00a4100780 0x7f00a406a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:33.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.258+0000 7f00ab0b2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00a4101970 0x7f00a419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:33.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.258+0000 7f00ab0b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00a4102330 0x7f00a419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:33.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.258+0000 7f00ab0b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f00a410c470 con 0x7f00a4102330 2024-03-16T09:13:33.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.258+0000 7f00ab0b2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f00a410c2f0 con 0x7f00a4101970 2024-03-16T09:13:33.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.258+0000 7f00ab0b2640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f00a410c5f0 con 0x7f00a4100780 2024-03-16T09:13:33.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.258+0000 7f00a964f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00a4102330 0x7f00a419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:33.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.258+0000 7f00a8e4e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00a4100780 0x7f00a406a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:33.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.258+0000 7f00a964f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00a4102330 0x7f00a419ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33782/0 (socket says 172.21.15.112:33782) 2024-03-16T09:13:33.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.258+0000 7f00a964f640 1 -- 172.21.15.112:0/1174572406 learned_addr learned my addr 172.21.15.112:0/1174572406 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:33.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.258+0000 7f00a3fff640 1 --2- 172.21.15.112:0/1174572406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00a4101970 0x7f00a419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:33.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.258+0000 7f00a8e4e640 1 -- 172.21.15.112:0/1174572406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00a4101970 msgr2=0x7f00a419c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:33.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.258+0000 7f00a8e4e640 1 --2- 172.21.15.112:0/1174572406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00a4101970 0x7f00a419c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.258+0000 7f00a8e4e640 1 -- 172.21.15.112:0/1174572406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00a4102330 msgr2=0x7f00a419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.258+0000 7f00a8e4e640 1 --2- 172.21.15.112:0/1174572406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00a4102330 0x7f00a419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.258+0000 7f00a8e4e640 1 -- 172.21.15.112:0/1174572406 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a419f870 con 0x7f00a4100780 2024-03-16T09:13:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.258+0000 7f00a964f640 1 --2- 172.21.15.112:0/1174572406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00a4102330 0x7f00a419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:13:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.259+0000 7f00a8e4e640 1 --2- 172.21.15.112:0/1174572406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00a4100780 0x7f00a406a980 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f009802c170 tx=0x7f0098004a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.259+0000 7f00a1ffb640 1 -- 172.21.15.112:0/1174572406 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0098043020 con 0x7f00a4100780 2024-03-16T09:13:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.259+0000 7f00ab0b2640 1 -- 172.21.15.112:0/1174572406 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a419faa0 con 0x7f00a4100780 2024-03-16T09:13:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.259+0000 7f00a1ffb640 1 -- 172.21.15.112:0/1174572406 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0098004250 con 0x7f00a4100780 2024-03-16T09:13:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.259+0000 7f00a1ffb640 1 -- 172.21.15.112:0/1174572406 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0098003ac0 con 0x7f00a4100780 2024-03-16T09:13:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.259+0000 7f00ab0b2640 1 -- 172.21.15.112:0/1174572406 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a419ffc0 con 0x7f00a4100780 2024-03-16T09:13:33.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.260+0000 7f00ab0b2640 1 -- 172.21.15.112:0/1174572406 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00a4068a20 con 0x7f00a4100780 2024-03-16T09:13:33.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.261+0000 7f00a1ffb640 1 -- 172.21.15.112:0/1174572406 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f0098046020 con 0x7f00a4100780 2024-03-16T09:13:33.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.262+0000 7f00a1ffb640 1 --2- 172.21.15.112:0/1174572406 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f00940a2800 0x7f00940a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:33.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.262+0000 7f00a3fff640 1 --2- 172.21.15.112:0/1174572406 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f00940a2800 0x7f00940a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:33.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.262+0000 7f00a1ffb640 1 -- 172.21.15.112:0/1174572406 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f00980f06c0 con 0x7f00a4100780 2024-03-16T09:13:33.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.262+0000 7f00a3fff640 1 --2- 172.21.15.112:0/1174572406 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f00940a2800 0x7f00940a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00900095f0 tx=0x7f0090005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:33.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.266+0000 7f00a1ffb640 1 -- 172.21.15.112:0/1174572406 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f009804d020 con 0x7f00a4100780 2024-03-16T09:13:33.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:32 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] up:boot 2024-03-16T09:13:33.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:32 smithi112 ceph-mon[32307]: fsmap 12 up:standby 2024-03-16T09:13:33.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:32 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T09:13:33.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:32 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] up:boot 2024-03-16T09:13:33.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:32 smithi194 ceph-mon[36729]: fsmap 12 up:standby 2024-03-16T09:13:33.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:32 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T09:13:33.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.512+0000 7f00ab0b2640 1 -- 172.21.15.112:0/1174572406 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00a40623e0 con 0x7f00a4100780 2024-03-16T09:13:33.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.513+0000 7f00a1ffb640 1 -- 172.21.15.112:0/1174572406 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10494 (secure 0 0 0) 0x7f00980bdb60 con 0x7f00a4100780 2024-03-16T09:13:33.515 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:33.515 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3939748854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3939748854},{"type":"v1","addr":"172.21.15.112:6839","nonce":3939748854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3118523258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3118523258},{"type":"v1","addr":"172.21.15.112:6841","nonce":3118523258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/1682076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":1682076},{"type":"v1","addr":"172.21.15.194:6837","nonce":1682076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/2528943309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2528943309},{"type":"v1","addr":"172.21.15.194:6839","nonce":2528943309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/272895965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":272895965},{"type":"v1","addr":"172.21.15.159:6839","nonce":272895965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-03-16T09:13:33.515 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 13 2024-03-16T09:13:33.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.515+0000 7f00ab0b2640 1 -- 172.21.15.112:0/1174572406 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f00940a2800 msgr2=0x7f00940a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:33.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.515+0000 7f00ab0b2640 1 --2- 172.21.15.112:0/1174572406 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f00940a2800 0x7f00940a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00900095f0 tx=0x7f0090005f90 comp rx=0 tx=0).stop 2024-03-16T09:13:33.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.515+0000 7f00ab0b2640 1 -- 172.21.15.112:0/1174572406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00a4100780 msgr2=0x7f00a406a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:33.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.515+0000 7f00ab0b2640 1 --2- 172.21.15.112:0/1174572406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00a4100780 0x7f00a406a980 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f009802c170 tx=0x7f0098004a30 comp rx=0 tx=0).stop 2024-03-16T09:13:33.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.515+0000 7f00ab0b2640 1 -- 172.21.15.112:0/1174572406 shutdown_connections 2024-03-16T09:13:33.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.515+0000 7f00ab0b2640 1 --2- 172.21.15.112:0/1174572406 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f00940a2800 0x7f00940a4cc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.515+0000 7f00ab0b2640 1 --2- 172.21.15.112:0/1174572406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00a4102330 0x7f00a419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.515+0000 7f00ab0b2640 1 --2- 172.21.15.112:0/1174572406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00a4101970 0x7f00a419c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.515+0000 7f00ab0b2640 1 --2- 172.21.15.112:0/1174572406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f00a4100780 0x7f00a406a980 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.515+0000 7f00ab0b2640 1 -- 172.21.15.112:0/1174572406 >> 172.21.15.112:0/1174572406 conn(0x7f00a40fbd10 msgr2=0x7f00a4104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:33.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.515+0000 7f00ab0b2640 1 -- 172.21.15.112:0/1174572406 shutdown_connections 2024-03-16T09:13:33.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.515+0000 7f00ab0b2640 1 -- 172.21.15.112:0/1174572406 wait complete. 2024-03-16T09:13:33.528 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:13:33.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.633+0000 7f2c76744640 1 -- 172.21.15.112:0/3998077218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c701022b0 msgr2=0x7f2c70108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:33.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.633+0000 7f2c76744640 1 --2- 172.21.15.112:0/3998077218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c701022b0 0x7f2c70108b40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009ca0 tx=0x7f2c6802c0f0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.633+0000 7f2c76744640 1 -- 172.21.15.112:0/3998077218 shutdown_connections 2024-03-16T09:13:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.633+0000 7f2c76744640 1 --2- 172.21.15.112:0/3998077218 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c701022b0 0x7f2c70108b40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.633+0000 7f2c76744640 1 --2- 172.21.15.112:0/3998077218 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c701018f0 0x7f2c70101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.633+0000 7f2c76744640 1 --2- 172.21.15.112:0/3998077218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c70100700 0x7f2c70100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.633+0000 7f2c76744640 1 -- 172.21.15.112:0/3998077218 >> 172.21.15.112:0/3998077218 conn(0x7f2c700fbc70 msgr2=0x7f2c700fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.633+0000 7f2c76744640 1 -- 172.21.15.112:0/3998077218 shutdown_connections 2024-03-16T09:13:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.633+0000 7f2c76744640 1 -- 172.21.15.112:0/3998077218 wait complete. 2024-03-16T09:13:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.634+0000 7f2c76744640 1 Processor -- start 2024-03-16T09:13:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.634+0000 7f2c76744640 1 -- start start 2024-03-16T09:13:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.634+0000 7f2c76744640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c70100700 0x7f2c700694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.634+0000 7f2c76744640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c701018f0 0x7f2c7019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.634+0000 7f2c76744640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c701022b0 0x7f2c701a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.634+0000 7f2c76744640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7010c4b0 con 0x7f2c701018f0 2024-03-16T09:13:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.634+0000 7f2c76744640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7010c330 con 0x7f2c701022b0 2024-03-16T09:13:33.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.634+0000 7f2c76744640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7010c630 con 0x7f2c70100700 2024-03-16T09:13:33.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.634+0000 7f2c6f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c701018f0 0x7f2c7019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:33.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.634+0000 7f2c6f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c701018f0 0x7f2c7019ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33812/0 (socket says 172.21.15.112:33812) 2024-03-16T09:13:33.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.634+0000 7f2c6f7fe640 1 -- 172.21.15.112:0/811377188 learned_addr learned my addr 172.21.15.112:0/811377188 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:33.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.634+0000 7f2c6ffff640 1 --2- 172.21.15.112:0/811377188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c70100700 0x7f2c700694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:33.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.635+0000 7f2c74ce1640 1 --2- 172.21.15.112:0/811377188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c701022b0 0x7f2c701a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:33.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.635+0000 7f2c6f7fe640 1 -- 172.21.15.112:0/811377188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c70100700 msgr2=0x7f2c700694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:33.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.635+0000 7f2c6f7fe640 1 --2- 172.21.15.112:0/811377188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c70100700 0x7f2c700694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.635+0000 7f2c6f7fe640 1 -- 172.21.15.112:0/811377188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c701022b0 msgr2=0x7f2c701a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:33.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.635+0000 7f2c6f7fe640 1 --2- 172.21.15.112:0/811377188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c701022b0 0x7f2c701a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.635+0000 7f2c6f7fe640 1 -- 172.21.15.112:0/811377188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c701a1990 con 0x7f2c701018f0 2024-03-16T09:13:33.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.635+0000 7f2c6f7fe640 1 --2- 172.21.15.112:0/811377188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c701018f0 0x7f2c7019ec70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400c920 tx=0x7f2c6400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:33.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.635+0000 7f2c6d7fa640 1 -- 172.21.15.112:0/811377188 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c64007c70 con 0x7f2c701018f0 2024-03-16T09:13:33.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.635+0000 7f2c6d7fa640 1 -- 172.21.15.112:0/811377188 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2c64007e10 con 0x7f2c701018f0 2024-03-16T09:13:33.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.635+0000 7f2c76744640 1 -- 172.21.15.112:0/811377188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c701a1c80 con 0x7f2c701018f0 2024-03-16T09:13:33.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.635+0000 7f2c6d7fa640 1 -- 172.21.15.112:0/811377188 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c64018940 con 0x7f2c701018f0 2024-03-16T09:13:33.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.635+0000 7f2c76744640 1 -- 172.21.15.112:0/811377188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c701a21d0 con 0x7f2c701018f0 2024-03-16T09:13:33.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.636+0000 7f2c76744640 1 -- 172.21.15.112:0/811377188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c70107950 con 0x7f2c701018f0 2024-03-16T09:13:33.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.637+0000 7f2c6d7fa640 1 -- 172.21.15.112:0/811377188 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f2c64018b20 con 0x7f2c701018f0 2024-03-16T09:13:33.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.638+0000 7f2c6d7fa640 1 --2- 172.21.15.112:0/811377188 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2c580a2800 0x7f2c580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:33.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.638+0000 7f2c6ffff640 1 --2- 172.21.15.112:0/811377188 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2c580a2800 0x7f2c580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:33.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.638+0000 7f2c6d7fa640 1 -- 172.21.15.112:0/811377188 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f2c640ce660 con 0x7f2c701018f0 2024-03-16T09:13:33.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.638+0000 7f2c6ffff640 1 --2- 172.21.15.112:0/811377188 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2c580a2800 0x7f2c580a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2c70101750 tx=0x7f2c6000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:33.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.642+0000 7f2c6d7fa640 1 -- 172.21.15.112:0/811377188 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2c64014030 con 0x7f2c701018f0 2024-03-16T09:13:33.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.830+0000 7f2c76744640 1 -- 172.21.15.112:0/811377188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c7002d3b0 con 0x7f2c701018f0 2024-03-16T09:13:33.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.831+0000 7f2c6d7fa640 1 -- 172.21.15.112:0/811377188 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18528 (secure 0 0 0) 0x7f2c6409bba0 con 0x7f2c701018f0 2024-03-16T09:13:33.834 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:33.834 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":81,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:12:48.106465+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:13:33.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.833+0000 7f2c76744640 1 -- 172.21.15.112:0/811377188 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2c580a2800 msgr2=0x7f2c580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:33.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.833+0000 7f2c76744640 1 --2- 172.21.15.112:0/811377188 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2c580a2800 0x7f2c580a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2c70101750 tx=0x7f2c6000b390 comp rx=0 tx=0).stop 2024-03-16T09:13:33.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.833+0000 7f2c76744640 1 -- 172.21.15.112:0/811377188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c701018f0 msgr2=0x7f2c7019ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:33.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.833+0000 7f2c76744640 1 --2- 172.21.15.112:0/811377188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c701018f0 0x7f2c7019ec70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400c920 tx=0x7f2c6400cde0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.834+0000 7f2c76744640 1 -- 172.21.15.112:0/811377188 shutdown_connections 2024-03-16T09:13:33.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.834+0000 7f2c76744640 1 --2- 172.21.15.112:0/811377188 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2c580a2800 0x7f2c580a4cc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.834+0000 7f2c76744640 1 --2- 172.21.15.112:0/811377188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c701022b0 0x7f2c701a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.834+0000 7f2c76744640 1 --2- 172.21.15.112:0/811377188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c701018f0 0x7f2c7019ec70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.834+0000 7f2c76744640 1 --2- 172.21.15.112:0/811377188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c70100700 0x7f2c700694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.834+0000 7f2c76744640 1 -- 172.21.15.112:0/811377188 >> 172.21.15.112:0/811377188 conn(0x7f2c700fbc70 msgr2=0x7f2c70104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:33.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.834+0000 7f2c76744640 1 -- 172.21.15.112:0/811377188 shutdown_connections 2024-03-16T09:13:33.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.834+0000 7f2c76744640 1 -- 172.21.15.112:0/811377188 wait complete. 2024-03-16T09:13:33.847 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T09:08:57.135312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T09:13:33.847 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T09:13:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.952+0000 7fb8f7122640 1 -- 172.21.15.112:0/3431928768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f0102a00 msgr2=0x7fb8f0102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.952+0000 7fb8f7122640 1 --2- 172.21.15.112:0/3431928768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f0102a00 0x7fb8f0102e80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009900 tx=0x7fb8e002bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.952+0000 7fb8f7122640 1 -- 172.21.15.112:0/3431928768 shutdown_connections 2024-03-16T09:13:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.952+0000 7fb8f7122640 1 --2- 172.21.15.112:0/3431928768 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8f01033c0 0x7fb8f0109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.952+0000 7fb8f7122640 1 --2- 172.21.15.112:0/3431928768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f0102a00 0x7fb8f0102e80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.952+0000 7fb8f7122640 1 --2- 172.21.15.112:0/3431928768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8f0101810 0x7fb8f0101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.952+0000 7fb8f7122640 1 -- 172.21.15.112:0/3431928768 >> 172.21.15.112:0/3431928768 conn(0x7fb8f00fcd80 msgr2=0x7fb8f00ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.952+0000 7fb8f7122640 1 -- 172.21.15.112:0/3431928768 shutdown_connections 2024-03-16T09:13:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.952+0000 7fb8f7122640 1 -- 172.21.15.112:0/3431928768 wait complete. 2024-03-16T09:13:33.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.953+0000 7fb8f7122640 1 Processor -- start 2024-03-16T09:13:33.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.953+0000 7fb8f7122640 1 -- start start 2024-03-16T09:13:33.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.953+0000 7fb8f7122640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8f0101810 0x7fb8f0197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:33.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.953+0000 7fb8f7122640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f0102a00 0x7fb8f0198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:33.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.953+0000 7fb8f7122640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8f01033c0 0x7fb8f019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:33.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.953+0000 7fb8f7122640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f010d570 con 0x7fb8f0102a00 2024-03-16T09:13:33.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.953+0000 7fb8f7122640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f010d3f0 con 0x7fb8f01033c0 2024-03-16T09:13:33.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.953+0000 7fb8f7122640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f010d6f0 con 0x7fb8f0101810 2024-03-16T09:13:33.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.953+0000 7fb8e7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f0102a00 0x7fb8f0198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:33.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.953+0000 7fb8e7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f0102a00 0x7fb8f0198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33824/0 (socket says 172.21.15.112:33824) 2024-03-16T09:13:33.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.953+0000 7fb8e7fff640 1 -- 172.21.15.112:0/966798990 learned_addr learned my addr 172.21.15.112:0/966798990 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:33.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.953+0000 7fb8f4ebe640 1 --2- 172.21.15.112:0/966798990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8f0101810 0x7fb8f0197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:33.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.953+0000 7fb8f56bf640 1 --2- 172.21.15.112:0/966798990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8f01033c0 0x7fb8f019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:33.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.953+0000 7fb8e7fff640 1 -- 172.21.15.112:0/966798990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8f0101810 msgr2=0x7fb8f0197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:33.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.953+0000 7fb8e7fff640 1 --2- 172.21.15.112:0/966798990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8f0101810 0x7fb8f0197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.953+0000 7fb8e7fff640 1 -- 172.21.15.112:0/966798990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8f01033c0 msgr2=0x7fb8f019f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:33.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.953+0000 7fb8e7fff640 1 --2- 172.21.15.112:0/966798990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8f01033c0 0x7fb8f019f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:33.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.953+0000 7fb8e7fff640 1 -- 172.21.15.112:0/966798990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f019f7e0 con 0x7fb8f0102a00 2024-03-16T09:13:33.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.953+0000 7fb8e7fff640 1 --2- 172.21.15.112:0/966798990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f0102a00 0x7fb8f0198100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009230 tx=0x7fb8e0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:33.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.954+0000 7fb8e5ffb640 1 -- 172.21.15.112:0/966798990 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0043020 con 0x7fb8f0102a00 2024-03-16T09:13:33.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.954+0000 7fb8f7122640 1 -- 172.21.15.112:0/966798990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f019fa70 con 0x7fb8f0102a00 2024-03-16T09:13:33.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.954+0000 7fb8e5ffb640 1 -- 172.21.15.112:0/966798990 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb8e0004250 con 0x7fb8f0102a00 2024-03-16T09:13:33.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.954+0000 7fb8e5ffb640 1 -- 172.21.15.112:0/966798990 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0003b50 con 0x7fb8f0102a00 2024-03-16T09:13:33.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.954+0000 7fb8f7122640 1 -- 172.21.15.112:0/966798990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f01a0010 con 0x7fb8f0102a00 2024-03-16T09:13:33.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.955+0000 7fb8f7122640 1 -- 172.21.15.112:0/966798990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8f0108b30 con 0x7fb8f0102a00 2024-03-16T09:13:33.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.956+0000 7fb8e5ffb640 1 -- 172.21.15.112:0/966798990 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fb8e0046020 con 0x7fb8f0102a00 2024-03-16T09:13:33.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.957+0000 7fb8e5ffb640 1 --2- 172.21.15.112:0/966798990 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb8d80a2800 0x7fb8d80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:33.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.957+0000 7fb8f4ebe640 1 --2- 172.21.15.112:0/966798990 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb8d80a2800 0x7fb8d80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:33.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.957+0000 7fb8e5ffb640 1 -- 172.21.15.112:0/966798990 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb8e00f0a80 con 0x7fb8f0102a00 2024-03-16T09:13:33.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.957+0000 7fb8f4ebe640 1 --2- 172.21.15.112:0/966798990 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb8d80a2800 0x7fb8d80a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0102860 tx=0x7fb8ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:33.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:33.961+0000 7fb8e5ffb640 1 -- 172.21.15.112:0/966798990 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb8e004d020 con 0x7fb8f0102a00 2024-03-16T09:13:34.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.147+0000 7fb8f7122640 1 -- 172.21.15.112:0/966798990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb8f002d3d0 con 0x7fb8f0102a00 2024-03-16T09:13:34.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.147+0000 7fb8e5ffb640 1 -- 172.21.15.112:0/966798990 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fb8e00bdfc0 con 0x7fb8f0102a00 2024-03-16T09:13:34.149 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-03-16T09:13:34.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.149+0000 7fb8f7122640 1 -- 172.21.15.112:0/966798990 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb8d80a2800 msgr2=0x7fb8d80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.149+0000 7fb8f7122640 1 --2- 172.21.15.112:0/966798990 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb8d80a2800 0x7fb8d80a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0102860 tx=0x7fb8ec005f90 comp rx=0 tx=0).stop 2024-03-16T09:13:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.149+0000 7fb8f7122640 1 -- 172.21.15.112:0/966798990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f0102a00 msgr2=0x7fb8f0198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.149+0000 7fb8f7122640 1 --2- 172.21.15.112:0/966798990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f0102a00 0x7fb8f0198100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009230 tx=0x7fb8e0004ab0 comp rx=0 tx=0).stop 2024-03-16T09:13:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.149+0000 7fb8f7122640 1 -- 172.21.15.112:0/966798990 shutdown_connections 2024-03-16T09:13:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.149+0000 7fb8f7122640 1 --2- 172.21.15.112:0/966798990 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb8d80a2800 0x7fb8d80a4cc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.149+0000 7fb8f7122640 1 --2- 172.21.15.112:0/966798990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8f01033c0 0x7fb8f019f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.149+0000 7fb8f7122640 1 --2- 172.21.15.112:0/966798990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8f0102a00 0x7fb8f0198100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.149+0000 7fb8f7122640 1 --2- 172.21.15.112:0/966798990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8f0101810 0x7fb8f0197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.149+0000 7fb8f7122640 1 -- 172.21.15.112:0/966798990 >> 172.21.15.112:0/966798990 conn(0x7fb8f00fcd80 msgr2=0x7fb8f0105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.150+0000 7fb8f7122640 1 -- 172.21.15.112:0/966798990 shutdown_connections 2024-03-16T09:13:34.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.150+0000 7fb8f7122640 1 -- 172.21.15.112:0/966798990 wait complete. 2024-03-16T09:13:34.162 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:33 smithi112 ceph-mon[32307]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:34.162 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.162 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.162 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.162 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.163 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:33 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1174572406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:34.163 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.163 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.163 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:34.163 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:13:34.163 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.163 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:33 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:13:34.163 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:33 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/811377188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:34.164 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-16T09:13:34.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:33 smithi159 ceph-mon[36739]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:34.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:33 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1174572406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:34.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:34.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:13:34.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:33 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:13:34.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:33 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/811377188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:34.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.291+0000 7f9194759640 1 -- 172.21.15.112:0/118335584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f918c104bb0 msgr2=0x7f918c106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:34.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.291+0000 7f9194759640 1 --2- 172.21.15.112:0/118335584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f918c104bb0 0x7f918c106fa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9188009900 tx=0x7f918802bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:34.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.291+0000 7f9194759640 1 -- 172.21.15.112:0/118335584 shutdown_connections 2024-03-16T09:13:34.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.291+0000 7f9194759640 1 --2- 172.21.15.112:0/118335584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f918c1074e0 0x7f918c109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:34.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.291+0000 7f9194759640 1 --2- 172.21.15.112:0/118335584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f918c104bb0 0x7f918c106fa0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:34.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.291+0000 7f9194759640 1 --2- 172.21.15.112:0/118335584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f918c102280 0x7f918c104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:34.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.291+0000 7f9194759640 1 -- 172.21.15.112:0/118335584 >> 172.21.15.112:0/118335584 conn(0x7f918c0fbc70 msgr2=0x7f918c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:34.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.291+0000 7f9194759640 1 -- 172.21.15.112:0/118335584 shutdown_connections 2024-03-16T09:13:34.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.291+0000 7f9194759640 1 -- 172.21.15.112:0/118335584 wait complete. 2024-03-16T09:13:34.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.292+0000 7f9194759640 1 Processor -- start 2024-03-16T09:13:34.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.292+0000 7f9194759640 1 -- start start 2024-03-16T09:13:34.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.292+0000 7f9194759640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f918c102280 0x7f918c197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:34.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.292+0000 7f9194759640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f918c104bb0 0x7f918c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:34.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.292+0000 7f9194759640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f918c1074e0 0x7f918c19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:34.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.292+0000 7f9194759640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f918c10d560 con 0x7f918c1074e0 2024-03-16T09:13:34.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.292+0000 7f9194759640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f918c10d3e0 con 0x7f918c102280 2024-03-16T09:13:34.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.292+0000 7f9194759640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f918c10d6e0 con 0x7f918c104bb0 2024-03-16T09:13:34.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.292+0000 7f9192cf6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f918c1074e0 0x7f918c19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:34.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.292+0000 7f9192cf6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f918c1074e0 0x7f918c19f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33850/0 (socket says 172.21.15.112:33850) 2024-03-16T09:13:34.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.292+0000 7f9192cf6640 1 -- 172.21.15.112:0/1592096244 learned_addr learned my addr 172.21.15.112:0/1592096244 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:34.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.292+0000 7f91924f5640 1 --2- 172.21.15.112:0/1592096244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f918c102280 0x7f918c197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:34.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.292+0000 7f9191cf4640 1 --2- 172.21.15.112:0/1592096244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f918c104bb0 0x7f918c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:34.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.292+0000 7f9192cf6640 1 -- 172.21.15.112:0/1592096244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f918c104bb0 msgr2=0x7f918c198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:34.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.292+0000 7f9192cf6640 1 --2- 172.21.15.112:0/1592096244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f918c104bb0 0x7f918c198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:34.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.292+0000 7f9192cf6640 1 -- 172.21.15.112:0/1592096244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f918c102280 msgr2=0x7f918c197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:34.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.292+0000 7f9192cf6640 1 --2- 172.21.15.112:0/1592096244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f918c102280 0x7f918c197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:34.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.292+0000 7f9192cf6640 1 -- 172.21.15.112:0/1592096244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c19f840 con 0x7f918c1074e0 2024-03-16T09:13:34.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.292+0000 7f91924f5640 1 --2- 172.21.15.112:0/1592096244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f918c102280 0x7f918c197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:13:34.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.293+0000 7f9192cf6640 1 --2- 172.21.15.112:0/1592096244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f918c1074e0 0x7f918c19f1a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f918400eaa0 tx=0x7f918400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:34.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.293+0000 7f91837fe640 1 -- 172.21.15.112:0/1592096244 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918400cdb0 con 0x7f918c1074e0 2024-03-16T09:13:34.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.293+0000 7f9194759640 1 -- 172.21.15.112:0/1592096244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918c19fb30 con 0x7f918c1074e0 2024-03-16T09:13:34.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.293+0000 7f91837fe640 1 -- 172.21.15.112:0/1592096244 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9184004b20 con 0x7f918c1074e0 2024-03-16T09:13:34.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.293+0000 7f91837fe640 1 -- 172.21.15.112:0/1592096244 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9184011760 con 0x7f918c1074e0 2024-03-16T09:13:34.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.293+0000 7f9194759640 1 -- 172.21.15.112:0/1592096244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f918c1a00d0 con 0x7f918c1074e0 2024-03-16T09:13:34.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.294+0000 7f9194759640 1 -- 172.21.15.112:0/1592096244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f918c0fd760 con 0x7f918c1074e0 2024-03-16T09:13:34.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.295+0000 7f91837fe640 1 -- 172.21.15.112:0/1592096244 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f9184011900 con 0x7f918c1074e0 2024-03-16T09:13:34.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.296+0000 7f91837fe640 1 --2- 172.21.15.112:0/1592096244 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f91780a2800 0x7f91780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:34.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.296+0000 7f91837fe640 1 -- 172.21.15.112:0/1592096244 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f91840ceb90 con 0x7f918c1074e0 2024-03-16T09:13:34.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.296+0000 7f91924f5640 1 --2- 172.21.15.112:0/1592096244 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f91780a2800 0x7f91780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:34.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.296+0000 7f91924f5640 1 --2- 172.21.15.112:0/1592096244 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f91780a2800 0x7f91780a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f918c199100 tx=0x7f917c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:34.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.301+0000 7f91837fe640 1 -- 172.21.15.112:0/1592096244 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9184016030 con 0x7f918c1074e0 2024-03-16T09:13:34.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:33 smithi194 ceph-mon[36729]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:34.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:33 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1174572406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:34.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:34.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:13:34.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:34.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:33 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:13:34.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:33 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/811377188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:34.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.484+0000 7f9194759640 1 -- 172.21.15.112:0/1592096244 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f918c02d330 con 0x7f918c1074e0 2024-03-16T09:13:34.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.484+0000 7f91837fe640 1 -- 172.21.15.112:0/1592096244 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f918409ba80 con 0x7f918c1074e0 2024-03-16T09:13:34.486 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:34.486 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-03-16T09:13:34.486 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:13:34.486 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2024-03-16T09:13:34.486 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2024-03-16T09:13:34.486 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:13:34.486 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-03-16T09:13:34.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.486+0000 7f9194759640 1 -- 172.21.15.112:0/1592096244 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f91780a2800 msgr2=0x7f91780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:34.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.486+0000 7f9194759640 1 --2- 172.21.15.112:0/1592096244 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f91780a2800 0x7f91780a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f918c199100 tx=0x7f917c005f90 comp rx=0 tx=0).stop 2024-03-16T09:13:34.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.486+0000 7f9194759640 1 -- 172.21.15.112:0/1592096244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f918c1074e0 msgr2=0x7f918c19f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:34.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.486+0000 7f9194759640 1 --2- 172.21.15.112:0/1592096244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f918c1074e0 0x7f918c19f1a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f918400eaa0 tx=0x7f918400ef60 comp rx=0 tx=0).stop 2024-03-16T09:13:34.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.486+0000 7f9194759640 1 -- 172.21.15.112:0/1592096244 shutdown_connections 2024-03-16T09:13:34.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.486+0000 7f9194759640 1 --2- 172.21.15.112:0/1592096244 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f91780a2800 0x7f91780a4cc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:34.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.486+0000 7f9194759640 1 --2- 172.21.15.112:0/1592096244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f918c1074e0 0x7f918c19f1a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:34.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.486+0000 7f9194759640 1 --2- 172.21.15.112:0/1592096244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f918c104bb0 0x7f918c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:34.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.486+0000 7f9194759640 1 --2- 172.21.15.112:0/1592096244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f918c102280 0x7f918c197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:34.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.486+0000 7f9194759640 1 -- 172.21.15.112:0/1592096244 >> 172.21.15.112:0/1592096244 conn(0x7f918c0fbc70 msgr2=0x7f918c102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:34.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.486+0000 7f9194759640 1 -- 172.21.15.112:0/1592096244 shutdown_connections 2024-03-16T09:13:34.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.486+0000 7f9194759640 1 -- 172.21.15.112:0/1592096244 wait complete. 2024-03-16T09:13:34.499 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-16T09:13:34.499 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-16T09:13:34.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.606+0000 7f56cdee8640 1 -- 172.21.15.112:0/170420548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c81018d0 msgr2=0x7f56c8101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:34.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.606+0000 7f56cdee8640 1 --2- 172.21.15.112:0/170420548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c81018d0 0x7f56c8101d50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009900 tx=0x7f56bc02bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:34.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.606+0000 7f56cdee8640 1 -- 172.21.15.112:0/170420548 shutdown_connections 2024-03-16T09:13:34.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.606+0000 7f56cdee8640 1 --2- 172.21.15.112:0/170420548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56c8102290 0x7f56c8108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:34.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.606+0000 7f56cdee8640 1 --2- 172.21.15.112:0/170420548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c81018d0 0x7f56c8101d50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:34.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.606+0000 7f56cdee8640 1 --2- 172.21.15.112:0/170420548 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56c81006e0 0x7f56c8100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:34.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.606+0000 7f56cdee8640 1 -- 172.21.15.112:0/170420548 >> 172.21.15.112:0/170420548 conn(0x7f56c80fbc70 msgr2=0x7f56c80fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.606+0000 7f56cdee8640 1 -- 172.21.15.112:0/170420548 shutdown_connections 2024-03-16T09:13:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.606+0000 7f56cdee8640 1 -- 172.21.15.112:0/170420548 wait complete. 2024-03-16T09:13:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.607+0000 7f56cdee8640 1 Processor -- start 2024-03-16T09:13:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.607+0000 7f56cdee8640 1 -- start start 2024-03-16T09:13:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.607+0000 7f56cdee8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56c81006e0 0x7f56c8069510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.607+0000 7f56cdee8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56c81018d0 0x7f56c819c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.607+0000 7f56cdee8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c8102290 0x7f56c819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.607+0000 7f56cdee8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f56c810c440 con 0x7f56c8102290 2024-03-16T09:13:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.607+0000 7f56cdee8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56c810c2c0 con 0x7f56c81006e0 2024-03-16T09:13:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.607+0000 7f56cdee8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f56c810c5c0 con 0x7f56c81018d0 2024-03-16T09:13:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.607+0000 7f56c7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c8102290 0x7f56c819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.607+0000 7f56c7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c8102290 0x7f56c819ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33872/0 (socket says 172.21.15.112:33872) 2024-03-16T09:13:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.607+0000 7f56c7fff640 1 -- 172.21.15.112:0/600750448 learned_addr learned my addr 172.21.15.112:0/600750448 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.607+0000 7f56c77fe640 1 --2- 172.21.15.112:0/600750448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56c81006e0 0x7f56c8069510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:34.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.607+0000 7f56c6ffd640 1 --2- 172.21.15.112:0/600750448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56c81018d0 0x7f56c819c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:34.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.607+0000 7f56c7fff640 1 -- 172.21.15.112:0/600750448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56c81018d0 msgr2=0x7f56c819c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:34.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.607+0000 7f56c7fff640 1 --2- 172.21.15.112:0/600750448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56c81018d0 0x7f56c819c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:34.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.607+0000 7f56c7fff640 1 -- 172.21.15.112:0/600750448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56c81006e0 msgr2=0x7f56c8069510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:34.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.607+0000 7f56c7fff640 1 --2- 172.21.15.112:0/600750448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56c81006e0 0x7f56c8069510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:34.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.607+0000 7f56c7fff640 1 -- 172.21.15.112:0/600750448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c819f7e0 con 0x7f56c8102290 2024-03-16T09:13:34.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.608+0000 7f56c7fff640 1 --2- 172.21.15.112:0/600750448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c8102290 0x7f56c819ef20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f56c000eaa0 tx=0x7f56c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:34.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.608+0000 7f56c4ff9640 1 -- 172.21.15.112:0/600750448 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c000cdb0 con 0x7f56c8102290 2024-03-16T09:13:34.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.608+0000 7f56cdee8640 1 -- 172.21.15.112:0/600750448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c819fad0 con 0x7f56c8102290 2024-03-16T09:13:34.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.608+0000 7f56c4ff9640 1 -- 172.21.15.112:0/600750448 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f56c0004b20 con 0x7f56c8102290 2024-03-16T09:13:34.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.608+0000 7f56c4ff9640 1 -- 172.21.15.112:0/600750448 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0011760 con 0x7f56c8102290 2024-03-16T09:13:34.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.608+0000 7f56cdee8640 1 -- 172.21.15.112:0/600750448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c819ffa0 con 0x7f56c8102290 2024-03-16T09:13:34.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.609+0000 7f56cdee8640 1 -- 172.21.15.112:0/600750448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c8107a00 con 0x7f56c8102290 2024-03-16T09:13:34.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.610+0000 7f56c4ff9640 1 -- 172.21.15.112:0/600750448 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f56c0004cd0 con 0x7f56c8102290 2024-03-16T09:13:34.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.610+0000 7f56c4ff9640 1 --2- 172.21.15.112:0/600750448 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f56b00a2800 0x7f56b00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:34.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.611+0000 7f56c77fe640 1 --2- 172.21.15.112:0/600750448 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f56b00a2800 0x7f56b00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:34.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.611+0000 7f56c4ff9640 1 -- 172.21.15.112:0/600750448 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f56c00ce700 con 0x7f56c8102290 2024-03-16T09:13:34.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.611+0000 7f56c77fe640 1 --2- 172.21.15.112:0/600750448 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f56b00a2800 0x7f56b00a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f56b8005fd0 tx=0x7f56b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:34.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.615+0000 7f56c4ff9640 1 -- 172.21.15.112:0/600750448 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f56c0016030 con 0x7f56c8102290 2024-03-16T09:13:34.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:34.799+0000 7f56cdee8640 1 -- 172.21.15.112:0/600750448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f56c802d280 con 0x7f56c8102290 2024-03-16T09:13:35.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:35 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/966798990' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:13:35.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:35 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1592096244' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-16T09:13:35.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.093+0000 7f56c4ff9640 1 -- 172.21.15.112:0/600750448 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f56c009b5f0 con 0x7f56c8102290 2024-03-16T09:13:35.095 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2024-03-16T09:13:35.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.096+0000 7f56cdee8640 1 -- 172.21.15.112:0/600750448 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f56b00a2800 msgr2=0x7f56b00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:35.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.096+0000 7f56cdee8640 1 --2- 172.21.15.112:0/600750448 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f56b00a2800 0x7f56b00a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f56b8005fd0 tx=0x7f56b800b410 comp rx=0 tx=0).stop 2024-03-16T09:13:35.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.096+0000 7f56cdee8640 1 -- 172.21.15.112:0/600750448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c8102290 msgr2=0x7f56c819ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:35.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.096+0000 7f56cdee8640 1 --2- 172.21.15.112:0/600750448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c8102290 0x7f56c819ef20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f56c000eaa0 tx=0x7f56c000ef60 comp rx=0 tx=0).stop 2024-03-16T09:13:35.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.096+0000 7f56cdee8640 1 -- 172.21.15.112:0/600750448 shutdown_connections 2024-03-16T09:13:35.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.096+0000 7f56cdee8640 1 --2- 172.21.15.112:0/600750448 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f56b00a2800 0x7f56b00a4cc0 secure :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f56b8005fd0 tx=0x7f56b800b410 comp rx=0 tx=0).stop 2024-03-16T09:13:35.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.096+0000 7f56cdee8640 1 --2- 172.21.15.112:0/600750448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56c8102290 0x7f56c819ef20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:35.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.096+0000 7f56cdee8640 1 --2- 172.21.15.112:0/600750448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56c81018d0 0x7f56c819c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:35.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.096+0000 7f56cdee8640 1 --2- 172.21.15.112:0/600750448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56c81006e0 0x7f56c8069510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:35.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.096+0000 7f56cdee8640 1 -- 172.21.15.112:0/600750448 >> 172.21.15.112:0/600750448 conn(0x7f56c80fbc70 msgr2=0x7f56c8104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:35.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.096+0000 7f56cdee8640 1 -- 172.21.15.112:0/600750448 shutdown_connections 2024-03-16T09:13:35.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.096+0000 7f56cdee8640 1 -- 172.21.15.112:0/600750448 wait complete. 2024-03-16T09:13:35.113 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-16T09:13:35.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.233+0000 7f85af085640 1 -- 172.21.15.112:0/3590332624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85a81018d0 msgr2=0x7f85a8101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:35.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.233+0000 7f85af085640 1 --2- 172.21.15.112:0/3590332624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85a81018d0 0x7f85a8101d50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8594009900 tx=0x7f859402bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.233+0000 7f85af085640 1 -- 172.21.15.112:0/3590332624 shutdown_connections 2024-03-16T09:13:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.233+0000 7f85af085640 1 --2- 172.21.15.112:0/3590332624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a8102290 0x7f85a8108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.233+0000 7f85af085640 1 --2- 172.21.15.112:0/3590332624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85a81018d0 0x7f85a8101d50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.233+0000 7f85af085640 1 --2- 172.21.15.112:0/3590332624 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85a81006e0 0x7f85a8100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.233+0000 7f85af085640 1 -- 172.21.15.112:0/3590332624 >> 172.21.15.112:0/3590332624 conn(0x7f85a80fbc70 msgr2=0x7f85a80fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.233+0000 7f85af085640 1 -- 172.21.15.112:0/3590332624 shutdown_connections 2024-03-16T09:13:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.233+0000 7f85af085640 1 -- 172.21.15.112:0/3590332624 wait complete. 2024-03-16T09:13:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.233+0000 7f85af085640 1 Processor -- start 2024-03-16T09:13:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.233+0000 7f85af085640 1 -- start start 2024-03-16T09:13:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.233+0000 7f85af085640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a81006e0 0x7f85a80694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.233+0000 7f85af085640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85a81018d0 0x7f85a819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.233+0000 7f85af085640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85a8102290 0x7f85a819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.233+0000 7f85af085640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85a810c440 con 0x7f85a8102290 2024-03-16T09:13:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.233+0000 7f85af085640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f85a810c2c0 con 0x7f85a81006e0 2024-03-16T09:13:35.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.233+0000 7f85af085640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f85a810c5c0 con 0x7f85a81018d0 2024-03-16T09:13:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.234+0000 7f85ace21640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a81006e0 0x7f85a80694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.234+0000 7f85ad622640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85a8102290 0x7f85a819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.234+0000 7f85a7fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85a81018d0 0x7f85a819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.234+0000 7f85ace21640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a81006e0 0x7f85a80694b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:60360/0 (socket says 172.21.15.112:60360) 2024-03-16T09:13:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.234+0000 7f85ace21640 1 -- 172.21.15.112:0/3888618842 learned_addr learned my addr 172.21.15.112:0/3888618842 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.234+0000 7f85ad622640 1 -- 172.21.15.112:0/3888618842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85a81018d0 msgr2=0x7f85a819c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.234+0000 7f85ad622640 1 --2- 172.21.15.112:0/3888618842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85a81018d0 0x7f85a819c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.234+0000 7f85ad622640 1 -- 172.21.15.112:0/3888618842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a81006e0 msgr2=0x7f85a80694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.234+0000 7f85ad622640 1 --2- 172.21.15.112:0/3888618842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a81006e0 0x7f85a80694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.234+0000 7f85ad622640 1 -- 172.21.15.112:0/3888618842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a819f890 con 0x7f85a8102290 2024-03-16T09:13:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.234+0000 7f85a7fff640 1 --2- 172.21.15.112:0/3888618842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85a81018d0 0x7f85a819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:13:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.234+0000 7f85ace21640 1 --2- 172.21.15.112:0/3888618842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a81006e0 0x7f85a80694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:13:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.234+0000 7f85ad622640 1 --2- 172.21.15.112:0/3888618842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85a8102290 0x7f85a819ef80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009fd0 tx=0x7f85a000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.234+0000 7f85a5ffb640 1 -- 172.21.15.112:0/3888618842 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a000ce20 con 0x7f85a8102290 2024-03-16T09:13:35.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.234+0000 7f85a5ffb640 1 -- 172.21.15.112:0/3888618842 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f85a0004b20 con 0x7f85a8102290 2024-03-16T09:13:35.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.234+0000 7f85af085640 1 -- 172.21.15.112:0/3888618842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a819fb20 con 0x7f85a8102290 2024-03-16T09:13:35.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.234+0000 7f85a5ffb640 1 -- 172.21.15.112:0/3888618842 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0010790 con 0x7f85a8102290 2024-03-16T09:13:35.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.234+0000 7f85af085640 1 -- 172.21.15.112:0/3888618842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a819fff0 con 0x7f85a8102290 2024-03-16T09:13:35.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.236+0000 7f85af085640 1 -- 172.21.15.112:0/3888618842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a8107a50 con 0x7f85a8102290 2024-03-16T09:13:35.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.237+0000 7f85a5ffb640 1 -- 172.21.15.112:0/3888618842 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f85a0005020 con 0x7f85a8102290 2024-03-16T09:13:35.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.237+0000 7f85a5ffb640 1 --2- 172.21.15.112:0/3888618842 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f85980a2800 0x7f85980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:35.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.237+0000 7f85ace21640 1 --2- 172.21.15.112:0/3888618842 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f85980a2800 0x7f85980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:35.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.238+0000 7f85a5ffb640 1 -- 172.21.15.112:0/3888618842 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f85a00cee50 con 0x7f85a8102290 2024-03-16T09:13:35.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.238+0000 7f85ace21640 1 --2- 172.21.15.112:0/3888618842 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f85980a2800 0x7f85980a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f859c0058c0 tx=0x7f859c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:35.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.243+0000 7f85a5ffb640 1 -- 172.21.15.112:0/3888618842 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f85a009c220 con 0x7f85a8102290 2024-03-16T09:13:35.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:35 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/966798990' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:13:35.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:35 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1592096244' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-16T09:13:35.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:35.453+0000 7f85af085640 1 -- 172.21.15.112:0/3888618842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f85a8062480 con 0x7f85a8102290 2024-03-16T09:13:35.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:35 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/966798990' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:13:35.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:35 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1592096244' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-16T09:13:36.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.095+0000 7f85a5ffb640 1 -- 172.21.15.112:0/3888618842 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7f85a009bb20 con 0x7f85a8102290 2024-03-16T09:13:36.097 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2024-03-16T09:13:36.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.098+0000 7f85af085640 1 -- 172.21.15.112:0/3888618842 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f85980a2800 msgr2=0x7f85980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:36.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.098+0000 7f85af085640 1 --2- 172.21.15.112:0/3888618842 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f85980a2800 0x7f85980a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f859c0058c0 tx=0x7f859c005f90 comp rx=0 tx=0).stop 2024-03-16T09:13:36.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.098+0000 7f85af085640 1 -- 172.21.15.112:0/3888618842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85a8102290 msgr2=0x7f85a819ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:36.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.098+0000 7f85af085640 1 --2- 172.21.15.112:0/3888618842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85a8102290 0x7f85a819ef80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009fd0 tx=0x7f85a000ef10 comp rx=0 tx=0).stop 2024-03-16T09:13:36.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.099+0000 7f85af085640 1 -- 172.21.15.112:0/3888618842 shutdown_connections 2024-03-16T09:13:36.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.099+0000 7f85af085640 1 --2- 172.21.15.112:0/3888618842 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f85980a2800 0x7f85980a4cc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:36.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.099+0000 7f85af085640 1 --2- 172.21.15.112:0/3888618842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85a8102290 0x7f85a819ef80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:36.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.099+0000 7f85af085640 1 --2- 172.21.15.112:0/3888618842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85a81018d0 0x7f85a819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:36.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.099+0000 7f85af085640 1 --2- 172.21.15.112:0/3888618842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a81006e0 0x7f85a80694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:36.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.099+0000 7f85af085640 1 -- 172.21.15.112:0/3888618842 >> 172.21.15.112:0/3888618842 conn(0x7f85a80fbc70 msgr2=0x7f85a8104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:36.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.099+0000 7f85af085640 1 -- 172.21.15.112:0/3888618842 shutdown_connections 2024-03-16T09:13:36.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.099+0000 7f85af085640 1 -- 172.21.15.112:0/3888618842 wait complete. 2024-03-16T09:13:36.124 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-16T09:13:36.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.226+0000 7fb2efb40640 1 -- 172.21.15.112:0/2100539458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e81006f0 msgr2=0x7fb2e8100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:36.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.226+0000 7fb2efb40640 1 --2- 172.21.15.112:0/2100539458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e81006f0 0x7fb2e8100b10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009900 tx=0x7fb2e402bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:36.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.226+0000 7fb2efb40640 1 -- 172.21.15.112:0/2100539458 shutdown_connections 2024-03-16T09:13:36.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.226+0000 7fb2efb40640 1 --2- 172.21.15.112:0/2100539458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2e81022a0 0x7fb2e8108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:36.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.226+0000 7fb2efb40640 1 --2- 172.21.15.112:0/2100539458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2e81018e0 0x7fb2e8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:36.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.226+0000 7fb2efb40640 1 --2- 172.21.15.112:0/2100539458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e81006f0 0x7fb2e8100b10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:36.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.226+0000 7fb2efb40640 1 -- 172.21.15.112:0/2100539458 >> 172.21.15.112:0/2100539458 conn(0x7fb2e80fbc80 msgr2=0x7fb2e80fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:36.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.226+0000 7fb2efb40640 1 -- 172.21.15.112:0/2100539458 shutdown_connections 2024-03-16T09:13:36.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.226+0000 7fb2efb40640 1 -- 172.21.15.112:0/2100539458 wait complete. 2024-03-16T09:13:36.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.227+0000 7fb2efb40640 1 Processor -- start 2024-03-16T09:13:36.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.227+0000 7fb2efb40640 1 -- start start 2024-03-16T09:13:36.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.227+0000 7fb2efb40640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2e81006f0 0x7fb2e8197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:36.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.227+0000 7fb2efb40640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e81018e0 0x7fb2e8198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:36.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.227+0000 7fb2efb40640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2e81022a0 0x7fb2e819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:36.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.227+0000 7fb2efb40640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e810c4b0 con 0x7fb2e81018e0 2024-03-16T09:13:36.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.227+0000 7fb2efb40640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e810c330 con 0x7fb2e81022a0 2024-03-16T09:13:36.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.227+0000 7fb2efb40640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e810c630 con 0x7fb2e81006f0 2024-03-16T09:13:36.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.227+0000 7fb2ed0db640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e81018e0 0x7fb2e8198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:36.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.227+0000 7fb2ed0db640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e81018e0 0x7fb2e8198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33898/0 (socket says 172.21.15.112:33898) 2024-03-16T09:13:36.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.227+0000 7fb2ed0db640 1 -- 172.21.15.112:0/836267163 learned_addr learned my addr 172.21.15.112:0/836267163 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:36.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.227+0000 7fb2ed8dc640 1 --2- 172.21.15.112:0/836267163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2e81006f0 0x7fb2e8197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:36.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.227+0000 7fb2ee0dd640 1 --2- 172.21.15.112:0/836267163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2e81022a0 0x7fb2e819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:36.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.227+0000 7fb2ed0db640 1 -- 172.21.15.112:0/836267163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2e81006f0 msgr2=0x7fb2e8197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:36.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.227+0000 7fb2ed0db640 1 --2- 172.21.15.112:0/836267163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2e81006f0 0x7fb2e8197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:36.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.227+0000 7fb2ed0db640 1 -- 172.21.15.112:0/836267163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2e81022a0 msgr2=0x7fb2e819f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:36.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.227+0000 7fb2ed0db640 1 --2- 172.21.15.112:0/836267163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2e81022a0 0x7fb2e819f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:36.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.227+0000 7fb2ed0db640 1 -- 172.21.15.112:0/836267163 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e819f900 con 0x7fb2e81018e0 2024-03-16T09:13:36.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.227+0000 7fb2ed8dc640 1 --2- 172.21.15.112:0/836267163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2e81006f0 0x7fb2e8197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:13:36.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.228+0000 7fb2ed0db640 1 --2- 172.21.15.112:0/836267163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e81018e0 0x7fb2e8198200 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800d950 tx=0x7fb2d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:36.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.228+0000 7fb2deffd640 1 -- 172.21.15.112:0/836267163 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8017070 con 0x7fb2e81018e0 2024-03-16T09:13:36.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.228+0000 7fb2efb40640 1 -- 172.21.15.112:0/836267163 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e819fbf0 con 0x7fb2e81018e0 2024-03-16T09:13:36.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.228+0000 7fb2deffd640 1 -- 172.21.15.112:0/836267163 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb2d8009c30 con 0x7fb2e81018e0 2024-03-16T09:13:36.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.228+0000 7fb2deffd640 1 -- 172.21.15.112:0/836267163 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d800f740 con 0x7fb2e81018e0 2024-03-16T09:13:36.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.228+0000 7fb2efb40640 1 -- 172.21.15.112:0/836267163 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e81a0190 con 0x7fb2e81018e0 2024-03-16T09:13:36.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.229+0000 7fb2efb40640 1 -- 172.21.15.112:0/836267163 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e81a03c0 con 0x7fb2e81018e0 2024-03-16T09:13:36.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.230+0000 7fb2deffd640 1 -- 172.21.15.112:0/836267163 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fb2d8010a80 con 0x7fb2e81018e0 2024-03-16T09:13:36.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.230+0000 7fb2deffd640 1 --2- 172.21.15.112:0/836267163 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb2d40a2800 0x7fb2d40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:36.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.231+0000 7fb2ed8dc640 1 --2- 172.21.15.112:0/836267163 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb2d40a2800 0x7fb2d40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:36.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.231+0000 7fb2deffd640 1 -- 172.21.15.112:0/836267163 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7fb2d80ce580 con 0x7fb2e81018e0 2024-03-16T09:13:36.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.231+0000 7fb2ed8dc640 1 --2- 172.21.15.112:0/836267163 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb2d40a2800 0x7fb2d40a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8198b90 tx=0x7fb2e402c770 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:36.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.234+0000 7fb2deffd640 1 -- 172.21.15.112:0/836267163 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb2d8009de0 con 0x7fb2e81018e0 2024-03-16T09:13:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:36 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/600750448' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-16T09:13:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:36 smithi112 ceph-mon[32307]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:36 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/600750448' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-16T09:13:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:36 smithi112 ceph-mon[32307]: osdmap e82: 12 total, 12 up, 12 in 2024-03-16T09:13:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:36 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:36 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:36 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:36 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:36 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3888618842' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-16T09:13:36.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:36 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:36 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:36 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:36.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:36 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:13:36.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:36 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:36 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:36 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/600750448' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-16T09:13:36.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:36 smithi194 ceph-mon[36729]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:36.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:36 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/600750448' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-16T09:13:36.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:36 smithi194 ceph-mon[36729]: osdmap e82: 12 total, 12 up, 12 in 2024-03-16T09:13:36.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:36 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:36 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:36 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:36 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:36 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3888618842' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-16T09:13:36.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:36 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:36 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:36 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:36.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:36 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:13:36.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:36 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:36 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:36.448+0000 7fb2efb40640 1 -- 172.21.15.112:0/836267163 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb2e81a0590 con 0x7fb2e81018e0 2024-03-16T09:13:36.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:36 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/600750448' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-16T09:13:36.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:36 smithi159 ceph-mon[36739]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:36.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:36 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/600750448' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-16T09:13:36.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:36 smithi159 ceph-mon[36739]: osdmap e82: 12 total, 12 up, 12 in 2024-03-16T09:13:36.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:36 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:36 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:36 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:36 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:36 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3888618842' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-16T09:13:36.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:36 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:36 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:36 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:13:36.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:36 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:13:36.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:36 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:36.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:36 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:13:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.104+0000 7fb2deffd640 1 -- 172.21.15.112:0/836267163 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fb2d809b7d0 con 0x7fb2e81018e0 2024-03-16T09:13:37.106 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 2 and data pool 3 2024-03-16T09:13:37.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.108+0000 7fb2efb40640 1 -- 172.21.15.112:0/836267163 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb2d40a2800 msgr2=0x7fb2d40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:37.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.108+0000 7fb2efb40640 1 --2- 172.21.15.112:0/836267163 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb2d40a2800 0x7fb2d40a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8198b90 tx=0x7fb2e402c770 comp rx=0 tx=0).stop 2024-03-16T09:13:37.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.108+0000 7fb2efb40640 1 -- 172.21.15.112:0/836267163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e81018e0 msgr2=0x7fb2e8198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:37.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.108+0000 7fb2efb40640 1 --2- 172.21.15.112:0/836267163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e81018e0 0x7fb2e8198200 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800d950 tx=0x7fb2d800de10 comp rx=0 tx=0).stop 2024-03-16T09:13:37.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.108+0000 7fb2efb40640 1 -- 172.21.15.112:0/836267163 shutdown_connections 2024-03-16T09:13:37.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.108+0000 7fb2efb40640 1 --2- 172.21.15.112:0/836267163 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb2d40a2800 0x7fb2d40a4cc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.108+0000 7fb2efb40640 1 --2- 172.21.15.112:0/836267163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2e81022a0 0x7fb2e819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.108+0000 7fb2efb40640 1 --2- 172.21.15.112:0/836267163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2e81018e0 0x7fb2e8198200 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.108+0000 7fb2efb40640 1 --2- 172.21.15.112:0/836267163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2e81006f0 0x7fb2e8197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.108+0000 7fb2efb40640 1 -- 172.21.15.112:0/836267163 >> 172.21.15.112:0/836267163 conn(0x7fb2e80fbc80 msgr2=0x7fb2e8069db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:37.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.108+0000 7fb2efb40640 1 -- 172.21.15.112:0/836267163 shutdown_connections 2024-03-16T09:13:37.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.108+0000 7fb2efb40640 1 -- 172.21.15.112:0/836267163 wait complete. 2024-03-16T09:13:37.129 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:13:37.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.230+0000 7fa5274ff640 1 -- 172.21.15.112:0/1018294363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201022b0 msgr2=0x7fa520108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:37.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.230+0000 7fa5274ff640 1 --2- 172.21.15.112:0/1018294363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201022b0 0x7fa520108b40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa510009900 tx=0x7fa51002bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:37.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.230+0000 7fa5274ff640 1 -- 172.21.15.112:0/1018294363 shutdown_connections 2024-03-16T09:13:37.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.230+0000 7fa5274ff640 1 --2- 172.21.15.112:0/1018294363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201022b0 0x7fa520108b40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.230+0000 7fa5274ff640 1 --2- 172.21.15.112:0/1018294363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5201018f0 0x7fa520101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.230+0000 7fa5274ff640 1 --2- 172.21.15.112:0/1018294363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa520100700 0x7fa520100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.230+0000 7fa5274ff640 1 -- 172.21.15.112:0/1018294363 >> 172.21.15.112:0/1018294363 conn(0x7fa5200fbc70 msgr2=0x7fa5200fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:37.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.231+0000 7fa5274ff640 1 -- 172.21.15.112:0/1018294363 shutdown_connections 2024-03-16T09:13:37.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.231+0000 7fa5274ff640 1 -- 172.21.15.112:0/1018294363 wait complete. 2024-03-16T09:13:37.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.231+0000 7fa5274ff640 1 Processor -- start 2024-03-16T09:13:37.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.231+0000 7fa5274ff640 1 -- start start 2024-03-16T09:13:37.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.231+0000 7fa5274ff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa520100700 0x7fa520196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:37.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.231+0000 7fa5274ff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5201018f0 0x7fa520196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:37.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.231+0000 7fa5274ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201022b0 0x7fa52019e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:37.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.231+0000 7fa5274ff640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa52010c460 con 0x7fa5201022b0 2024-03-16T09:13:37.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.231+0000 7fa5274ff640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa52010c2e0 con 0x7fa5201018f0 2024-03-16T09:13:37.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.231+0000 7fa5274ff640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa52010c5e0 con 0x7fa520100700 2024-03-16T09:13:37.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.231+0000 7fa525a9c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201022b0 0x7fa52019e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:37.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.231+0000 7fa525a9c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201022b0 0x7fa52019e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33914/0 (socket says 172.21.15.112:33914) 2024-03-16T09:13:37.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.231+0000 7fa525a9c640 1 -- 172.21.15.112:0/1000672656 learned_addr learned my addr 172.21.15.112:0/1000672656 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:37.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.231+0000 7fa524a9a640 1 --2- 172.21.15.112:0/1000672656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5201018f0 0x7fa520196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:37.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.232+0000 7fa52529b640 1 --2- 172.21.15.112:0/1000672656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa520100700 0x7fa520196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:37.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.232+0000 7fa525a9c640 1 -- 172.21.15.112:0/1000672656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa520100700 msgr2=0x7fa520196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:37.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.232+0000 7fa525a9c640 1 --2- 172.21.15.112:0/1000672656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa520100700 0x7fa520196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.232+0000 7fa525a9c640 1 -- 172.21.15.112:0/1000672656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5201018f0 msgr2=0x7fa520196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:37.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.232+0000 7fa525a9c640 1 --2- 172.21.15.112:0/1000672656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5201018f0 0x7fa520196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.232+0000 7fa525a9c640 1 -- 172.21.15.112:0/1000672656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52019e6b0 con 0x7fa5201022b0 2024-03-16T09:13:37.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.232+0000 7fa52529b640 1 --2- 172.21.15.112:0/1000672656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa520100700 0x7fa520196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:13:37.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.232+0000 7fa525a9c640 1 --2- 172.21.15.112:0/1000672656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201022b0 0x7fa52019e040 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa510005ab0 tx=0x7fa51002c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:37.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.232+0000 7fa51e7fc640 1 -- 172.21.15.112:0/1000672656 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510043020 con 0x7fa5201022b0 2024-03-16T09:13:37.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.232+0000 7fa5274ff640 1 -- 172.21.15.112:0/1000672656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52019e940 con 0x7fa5201022b0 2024-03-16T09:13:37.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.232+0000 7fa51e7fc640 1 -- 172.21.15.112:0/1000672656 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa5100043d0 con 0x7fa5201022b0 2024-03-16T09:13:37.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.232+0000 7fa51e7fc640 1 -- 172.21.15.112:0/1000672656 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510003cc0 con 0x7fa5201022b0 2024-03-16T09:13:37.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.232+0000 7fa5274ff640 1 -- 172.21.15.112:0/1000672656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52019eee0 con 0x7fa5201022b0 2024-03-16T09:13:37.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.233+0000 7fa5274ff640 1 -- 172.21.15.112:0/1000672656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa520107950 con 0x7fa5201022b0 2024-03-16T09:13:37.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.234+0000 7fa51e7fc640 1 -- 172.21.15.112:0/1000672656 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fa510046020 con 0x7fa5201022b0 2024-03-16T09:13:37.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.234+0000 7fa51e7fc640 1 --2- 172.21.15.112:0/1000672656 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa5080a2800 0x7fa5080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:37.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.235+0000 7fa52529b640 1 --2- 172.21.15.112:0/1000672656 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa5080a2800 0x7fa5080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:37.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.235+0000 7fa51e7fc640 1 -- 172.21.15.112:0/1000672656 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fa5100f0c90 con 0x7fa5201022b0 2024-03-16T09:13:37.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.235+0000 7fa52529b640 1 --2- 172.21.15.112:0/1000672656 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa5080a2800 0x7fa5080a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa520101750 tx=0x7fa51800a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:37.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.239+0000 7fa51e7fc640 1 -- 172.21.15.112:0/1000672656 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa51004d020 con 0x7fa5201022b0 2024-03-16T09:13:37.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:37 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3888618842' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-16T09:13:37.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:37 smithi194 ceph-mon[36729]: osdmap e83: 12 total, 12 up, 12 in 2024-03-16T09:13:37.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:37 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/836267163' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-16T09:13:37.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:37 smithi194 ceph-mon[36729]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-16T09:13:37.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:37 smithi194 ceph-mon[36729]: pgmap v286: 97 pgs: 6 creating+peering, 1 active+clean, 90 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:37.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.416+0000 7fa5274ff640 1 -- 172.21.15.112:0/1000672656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa52002d360 con 0x7fa5201022b0 2024-03-16T09:13:37.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.418+0000 7fa51e7fc640 1 -- 172.21.15.112:0/1000672656 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22304 (secure 0 0 0) 0x7fa5100bde80 con 0x7fa5201022b0 2024-03-16T09:13:37.420 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:37.420 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":84,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:13:36.449840+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T09:13:35.040628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T09:13:35.689419+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:13:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.419+0000 7fa5274ff640 1 -- 172.21.15.112:0/1000672656 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa5080a2800 msgr2=0x7fa5080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.419+0000 7fa5274ff640 1 --2- 172.21.15.112:0/1000672656 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa5080a2800 0x7fa5080a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa520101750 tx=0x7fa51800a5e0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.419+0000 7fa5274ff640 1 -- 172.21.15.112:0/1000672656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201022b0 msgr2=0x7fa52019e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.419+0000 7fa5274ff640 1 --2- 172.21.15.112:0/1000672656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201022b0 0x7fa52019e040 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa510005ab0 tx=0x7fa51002c7b0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.419+0000 7fa5274ff640 1 -- 172.21.15.112:0/1000672656 shutdown_connections 2024-03-16T09:13:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.419+0000 7fa5274ff640 1 --2- 172.21.15.112:0/1000672656 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa5080a2800 0x7fa5080a4cc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.419+0000 7fa5274ff640 1 --2- 172.21.15.112:0/1000672656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201022b0 0x7fa52019e040 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.420+0000 7fa5274ff640 1 --2- 172.21.15.112:0/1000672656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5201018f0 0x7fa520196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.420+0000 7fa5274ff640 1 --2- 172.21.15.112:0/1000672656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa520100700 0x7fa520196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.420+0000 7fa5274ff640 1 -- 172.21.15.112:0/1000672656 >> 172.21.15.112:0/1000672656 conn(0x7fa5200fbc70 msgr2=0x7fa520104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:37.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.420+0000 7fa5274ff640 1 -- 172.21.15.112:0/1000672656 shutdown_connections 2024-03-16T09:13:37.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.420+0000 7fa5274ff640 1 -- 172.21.15.112:0/1000672656 wait complete. 2024-03-16T09:13:37.431 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:37 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3888618842' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-16T09:13:37.431 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:37 smithi112 ceph-mon[32307]: osdmap e83: 12 total, 12 up, 12 in 2024-03-16T09:13:37.431 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:37 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/836267163' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-16T09:13:37.431 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:37 smithi112 ceph-mon[32307]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-16T09:13:37.431 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:37 smithi112 ceph-mon[32307]: pgmap v286: 97 pgs: 6 creating+peering, 1 active+clean, 90 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:37.432 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:13:37.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:37 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3888618842' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-16T09:13:37.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:37 smithi159 ceph-mon[36739]: osdmap e83: 12 total, 12 up, 12 in 2024-03-16T09:13:37.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:37 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/836267163' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-16T09:13:37.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:37 smithi159 ceph-mon[36739]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-16T09:13:37.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:37 smithi159 ceph-mon[36739]: pgmap v286: 97 pgs: 6 creating+peering, 1 active+clean, 90 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-16T09:13:37.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.532+0000 7f88f1cde640 1 -- 172.21.15.112:0/72215543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88ec102e00 msgr2=0x7f88ec103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:37.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.532+0000 7f88f1cde640 1 --2- 172.21.15.112:0/72215543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88ec102e00 0x7f88ec103220 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009900 tx=0x7f88dc02bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:37.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.532+0000 7f88f1cde640 1 -- 172.21.15.112:0/72215543 shutdown_connections 2024-03-16T09:13:37.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.532+0000 7f88f1cde640 1 --2- 172.21.15.112:0/72215543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88ec100230 0x7f88ec109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.532+0000 7f88f1cde640 1 --2- 172.21.15.112:0/72215543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88ec0ff870 0x7f88ec0ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.532+0000 7f88f1cde640 1 --2- 172.21.15.112:0/72215543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88ec102e00 0x7f88ec103220 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.532+0000 7f88f1cde640 1 -- 172.21.15.112:0/72215543 >> 172.21.15.112:0/72215543 conn(0x7f88ec0fb410 msgr2=0x7f88ec0fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:37.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.532+0000 7f88f1cde640 1 -- 172.21.15.112:0/72215543 shutdown_connections 2024-03-16T09:13:37.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.532+0000 7f88f1cde640 1 -- 172.21.15.112:0/72215543 wait complete. 2024-03-16T09:13:37.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.533+0000 7f88f1cde640 1 Processor -- start 2024-03-16T09:13:37.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.533+0000 7f88f1cde640 1 -- start start 2024-03-16T09:13:37.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.533+0000 7f88f1cde640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88ec0ff870 0x7f88ec197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:37.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.533+0000 7f88f1cde640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88ec100230 0x7f88ec198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:37.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.533+0000 7f88f1cde640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88ec102e00 0x7f88ec19f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:37.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.533+0000 7f88f1cde640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec10d570 con 0x7f88ec0ff870 2024-03-16T09:13:37.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.533+0000 7f88f1cde640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec10d3f0 con 0x7f88ec100230 2024-03-16T09:13:37.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.533+0000 7f88f1cde640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f88ec10d6f0 con 0x7f88ec102e00 2024-03-16T09:13:37.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.533+0000 7f88eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88ec0ff870 0x7f88ec197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:37.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.533+0000 7f88eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88ec0ff870 0x7f88ec197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33918/0 (socket says 172.21.15.112:33918) 2024-03-16T09:13:37.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.533+0000 7f88eb7fe640 1 -- 172.21.15.112:0/594413266 learned_addr learned my addr 172.21.15.112:0/594413266 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:37.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.533+0000 7f88ebfff640 1 --2- 172.21.15.112:0/594413266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88ec102e00 0x7f88ec19f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:37.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.533+0000 7f88eaffd640 1 --2- 172.21.15.112:0/594413266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88ec100230 0x7f88ec198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:37.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.533+0000 7f88eb7fe640 1 -- 172.21.15.112:0/594413266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88ec102e00 msgr2=0x7f88ec19f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:37.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.533+0000 7f88eb7fe640 1 --2- 172.21.15.112:0/594413266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88ec102e00 0x7f88ec19f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.533+0000 7f88eb7fe640 1 -- 172.21.15.112:0/594413266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88ec100230 msgr2=0x7f88ec198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:37.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.533+0000 7f88eb7fe640 1 --2- 172.21.15.112:0/594413266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88ec100230 0x7f88ec198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.534+0000 7f88eb7fe640 1 -- 172.21.15.112:0/594413266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ec19f850 con 0x7f88ec0ff870 2024-03-16T09:13:37.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.534+0000 7f88eaffd640 1 --2- 172.21.15.112:0/594413266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88ec100230 0x7f88ec198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:13:37.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.534+0000 7f88eb7fe640 1 --2- 172.21.15.112:0/594413266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88ec0ff870 0x7f88ec197c40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f88dc005ab0 tx=0x7f88dc02c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:37.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.534+0000 7f88e8ff9640 1 -- 172.21.15.112:0/594413266 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc043020 con 0x7f88ec0ff870 2024-03-16T09:13:37.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.534+0000 7f88e8ff9640 1 -- 172.21.15.112:0/594413266 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f88dc004350 con 0x7f88ec0ff870 2024-03-16T09:13:37.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.534+0000 7f88f1cde640 1 -- 172.21.15.112:0/594413266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ec19fae0 con 0x7f88ec0ff870 2024-03-16T09:13:37.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.534+0000 7f88e8ff9640 1 -- 172.21.15.112:0/594413266 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc003d70 con 0x7f88ec0ff870 2024-03-16T09:13:37.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.534+0000 7f88f1cde640 1 -- 172.21.15.112:0/594413266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ec1a0030 con 0x7f88ec0ff870 2024-03-16T09:13:37.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.535+0000 7f88f1cde640 1 -- 172.21.15.112:0/594413266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88ec108d30 con 0x7f88ec0ff870 2024-03-16T09:13:37.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.536+0000 7f88e8ff9640 1 -- 172.21.15.112:0/594413266 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f88dc046020 con 0x7f88ec0ff870 2024-03-16T09:13:37.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.536+0000 7f88e8ff9640 1 --2- 172.21.15.112:0/594413266 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88d40a2800 0x7f88d40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:37.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.537+0000 7f88eaffd640 1 --2- 172.21.15.112:0/594413266 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88d40a2800 0x7f88d40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:37.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.537+0000 7f88e8ff9640 1 -- 172.21.15.112:0/594413266 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f88dc0f0d00 con 0x7f88ec0ff870 2024-03-16T09:13:37.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.537+0000 7f88eaffd640 1 --2- 172.21.15.112:0/594413266 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88d40a2800 0x7f88d40a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f88e0009b40 tx=0x7f88e0005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:37.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.541+0000 7f88e8ff9640 1 -- 172.21.15.112:0/594413266 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f88dc04d020 con 0x7f88ec0ff870 2024-03-16T09:13:37.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.718+0000 7f88f1cde640 1 -- 172.21.15.112:0/594413266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88ec02d3d0 con 0x7f88ec0ff870 2024-03-16T09:13:37.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.719+0000 7f88e8ff9640 1 -- 172.21.15.112:0/594413266 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22304 (secure 0 0 0) 0x7f88dc0bdef0 con 0x7f88ec0ff870 2024-03-16T09:13:37.722 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:37.722 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":84,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:13:36.449840+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T09:13:35.040628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T09:13:35.689419+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:13:37.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.722+0000 7f88f1cde640 1 -- 172.21.15.112:0/594413266 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88d40a2800 msgr2=0x7f88d40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:37.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.722+0000 7f88f1cde640 1 --2- 172.21.15.112:0/594413266 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88d40a2800 0x7f88d40a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f88e0009b40 tx=0x7f88e0005e50 comp rx=0 tx=0).stop 2024-03-16T09:13:37.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.722+0000 7f88f1cde640 1 -- 172.21.15.112:0/594413266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88ec0ff870 msgr2=0x7f88ec197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:37.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.722+0000 7f88f1cde640 1 --2- 172.21.15.112:0/594413266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88ec0ff870 0x7f88ec197c40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f88dc005ab0 tx=0x7f88dc02c7a0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.722+0000 7f88f1cde640 1 -- 172.21.15.112:0/594413266 shutdown_connections 2024-03-16T09:13:37.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.722+0000 7f88f1cde640 1 --2- 172.21.15.112:0/594413266 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88d40a2800 0x7f88d40a4cc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.722+0000 7f88f1cde640 1 --2- 172.21.15.112:0/594413266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88ec102e00 0x7f88ec19f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.722+0000 7f88f1cde640 1 --2- 172.21.15.112:0/594413266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88ec100230 0x7f88ec198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.722+0000 7f88f1cde640 1 --2- 172.21.15.112:0/594413266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88ec0ff870 0x7f88ec197c40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.722+0000 7f88f1cde640 1 -- 172.21.15.112:0/594413266 >> 172.21.15.112:0/594413266 conn(0x7f88ec0fb410 msgr2=0x7f88ec105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:37.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.722+0000 7f88f1cde640 1 -- 172.21.15.112:0/594413266 shutdown_connections 2024-03-16T09:13:37.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.722+0000 7f88f1cde640 1 -- 172.21.15.112:0/594413266 wait complete. 2024-03-16T09:13:37.736 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-16T09:13:37.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.840+0000 7fc20faea640 1 -- 172.21.15.112:0/3521341526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc208101350 msgr2=0x7fc2081017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:37.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.840+0000 7fc20faea640 1 --2- 172.21.15.112:0/3521341526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc208101350 0x7fc2081017d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc200009900 tx=0x7fc20002bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:37.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.840+0000 7fc20faea640 1 -- 172.21.15.112:0/3521341526 shutdown_connections 2024-03-16T09:13:37.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.840+0000 7fc20faea640 1 --2- 172.21.15.112:0/3521341526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc208101350 0x7fc2081017d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.840+0000 7fc20faea640 1 --2- 172.21.15.112:0/3521341526 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc2081009b0 0x7fc208100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.840+0000 7fc20faea640 1 --2- 172.21.15.112:0/3521341526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc208100050 0x7fc208100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.840+0000 7fc20faea640 1 -- 172.21.15.112:0/3521341526 >> 172.21.15.112:0/3521341526 conn(0x7fc2080fbc70 msgr2=0x7fc2080fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:37.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.841+0000 7fc20faea640 1 -- 172.21.15.112:0/3521341526 shutdown_connections 2024-03-16T09:13:37.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.841+0000 7fc20faea640 1 -- 172.21.15.112:0/3521341526 wait complete. 2024-03-16T09:13:37.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.841+0000 7fc20faea640 1 Processor -- start 2024-03-16T09:13:37.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.841+0000 7fc20faea640 1 -- start start 2024-03-16T09:13:37.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.841+0000 7fc20faea640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc208100050 0x7fc208199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:37.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.841+0000 7fc20faea640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2081009b0 0x7fc20819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:37.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.841+0000 7fc20faea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc208101350 0x7fc2081a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:37.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.841+0000 7fc20faea640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc20810f750 con 0x7fc208101350 2024-03-16T09:13:37.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.841+0000 7fc20faea640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc20810f5d0 con 0x7fc2081009b0 2024-03-16T09:13:37.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.841+0000 7fc20faea640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc20810f8d0 con 0x7fc208100050 2024-03-16T09:13:37.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.841+0000 7fc20e087640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc208101350 0x7fc2081a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:37.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.841+0000 7fc20e087640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc208101350 0x7fc2081a1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33924/0 (socket says 172.21.15.112:33924) 2024-03-16T09:13:37.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.841+0000 7fc20e087640 1 -- 172.21.15.112:0/2673007488 learned_addr learned my addr 172.21.15.112:0/2673007488 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:37.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.841+0000 7fc20d886640 1 --2- 172.21.15.112:0/2673007488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc208100050 0x7fc208199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:37.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.841+0000 7fc20d085640 1 --2- 172.21.15.112:0/2673007488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2081009b0 0x7fc20819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:37.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.842+0000 7fc20e087640 1 -- 172.21.15.112:0/2673007488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc208100050 msgr2=0x7fc208199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:37.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.842+0000 7fc20e087640 1 --2- 172.21.15.112:0/2673007488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc208100050 0x7fc208199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.842+0000 7fc20e087640 1 -- 172.21.15.112:0/2673007488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2081009b0 msgr2=0x7fc20819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:37.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.842+0000 7fc20e087640 1 --2- 172.21.15.112:0/2673007488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2081009b0 0x7fc20819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:37.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.842+0000 7fc20e087640 1 -- 172.21.15.112:0/2673007488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2081a1990 con 0x7fc208101350 2024-03-16T09:13:37.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.842+0000 7fc20d085640 1 --2- 172.21.15.112:0/2673007488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2081009b0 0x7fc20819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:13:37.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.842+0000 7fc20e087640 1 --2- 172.21.15.112:0/2673007488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc208101350 0x7fc2081a1320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc200005ab0 tx=0x7fc20002c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:37.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.842+0000 7fc206ffd640 1 -- 172.21.15.112:0/2673007488 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc200043020 con 0x7fc208101350 2024-03-16T09:13:37.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.842+0000 7fc206ffd640 1 -- 172.21.15.112:0/2673007488 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc2000043d0 con 0x7fc208101350 2024-03-16T09:13:37.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.842+0000 7fc20faea640 1 -- 172.21.15.112:0/2673007488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2081a1c20 con 0x7fc208101350 2024-03-16T09:13:37.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.842+0000 7fc206ffd640 1 -- 172.21.15.112:0/2673007488 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc200003cc0 con 0x7fc208101350 2024-03-16T09:13:37.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.842+0000 7fc20faea640 1 -- 172.21.15.112:0/2673007488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2081a2170 con 0x7fc208101350 2024-03-16T09:13:37.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.843+0000 7fc20faea640 1 -- 172.21.15.112:0/2673007488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc20810af10 con 0x7fc208101350 2024-03-16T09:13:37.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.844+0000 7fc206ffd640 1 -- 172.21.15.112:0/2673007488 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fc200046020 con 0x7fc208101350 2024-03-16T09:13:37.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.845+0000 7fc206ffd640 1 --2- 172.21.15.112:0/2673007488 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc1f00a2800 0x7fc1f00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:37.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.845+0000 7fc20d886640 1 --2- 172.21.15.112:0/2673007488 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc1f00a2800 0x7fc1f00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:37.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.845+0000 7fc206ffd640 1 -- 172.21.15.112:0/2673007488 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fc2000f0fe0 con 0x7fc208101350 2024-03-16T09:13:37.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.845+0000 7fc20d886640 1 --2- 172.21.15.112:0/2673007488 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc1f00a2800 0x7fc1f00a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8005c40 tx=0x7fc1f800a590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:37.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:37.848+0000 7fc206ffd640 1 -- 172.21.15.112:0/2673007488 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc20004d020 con 0x7fc208101350 2024-03-16T09:13:38.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.058+0000 7fc20faea640 1 -- 172.21.15.112:0/2673007488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc20802d1b0 con 0x7fc208101350 2024-03-16T09:13:38.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.112+0000 7fc206ffd640 1 -- 172.21.15.112:0/2673007488 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7fc2000be2f0 con 0x7fc208101350 2024-03-16T09:13:38.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.114+0000 7fc20faea640 1 -- 172.21.15.112:0/2673007488 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc1f00a2800 msgr2=0x7fc1f00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:38.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.114+0000 7fc20faea640 1 --2- 172.21.15.112:0/2673007488 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc1f00a2800 0x7fc1f00a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8005c40 tx=0x7fc1f800a590 comp rx=0 tx=0).stop 2024-03-16T09:13:38.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.114+0000 7fc20faea640 1 -- 172.21.15.112:0/2673007488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc208101350 msgr2=0x7fc2081a1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:38.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.114+0000 7fc20faea640 1 --2- 172.21.15.112:0/2673007488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc208101350 0x7fc2081a1320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc200005ab0 tx=0x7fc20002c7b0 comp rx=0 tx=0).stop 2024-03-16T09:13:38.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.114+0000 7fc20faea640 1 -- 172.21.15.112:0/2673007488 shutdown_connections 2024-03-16T09:13:38.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.114+0000 7fc20faea640 1 --2- 172.21.15.112:0/2673007488 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc1f00a2800 0x7fc1f00a4cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:38.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.114+0000 7fc20faea640 1 --2- 172.21.15.112:0/2673007488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc208101350 0x7fc2081a1320 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:38.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.114+0000 7fc20faea640 1 --2- 172.21.15.112:0/2673007488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2081009b0 0x7fc20819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:38.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.114+0000 7fc20faea640 1 --2- 172.21.15.112:0/2673007488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc208100050 0x7fc208199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:38.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.114+0000 7fc20faea640 1 -- 172.21.15.112:0/2673007488 >> 172.21.15.112:0/2673007488 conn(0x7fc2080fbc70 msgr2=0x7fc2080fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:38.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.114+0000 7fc20faea640 1 -- 172.21.15.112:0/2673007488 shutdown_connections 2024-03-16T09:13:38.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.114+0000 7fc20faea640 1 -- 172.21.15.112:0/2673007488 wait complete. 2024-03-16T09:13:38.128 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-03-16T09:13:38.128 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-03-16T09:13:38.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.232+0000 7f196b71b640 1 -- 172.21.15.112:0/2558197815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 msgr2=0x7f1964108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:38.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.232+0000 7f196b71b640 1 --2- 172.21.15.112:0/2558197815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 0x7f1964108b40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f195c009ca0 tx=0x7f195c02c0f0 comp rx=0 tx=0).stop 2024-03-16T09:13:38.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.232+0000 7f196b71b640 1 -- 172.21.15.112:0/2558197815 shutdown_connections 2024-03-16T09:13:38.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.232+0000 7f196b71b640 1 --2- 172.21.15.112:0/2558197815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 0x7f1964108b40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:38.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.232+0000 7f196b71b640 1 --2- 172.21.15.112:0/2558197815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19641018f0 0x7f1964101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:38.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.232+0000 7f196b71b640 1 --2- 172.21.15.112:0/2558197815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1964100700 0x7f1964100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:38.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.232+0000 7f196b71b640 1 -- 172.21.15.112:0/2558197815 >> 172.21.15.112:0/2558197815 conn(0x7f19640fbc90 msgr2=0x7f19640fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:38.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.232+0000 7f196b71b640 1 -- 172.21.15.112:0/2558197815 shutdown_connections 2024-03-16T09:13:38.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.232+0000 7f196b71b640 1 -- 172.21.15.112:0/2558197815 wait complete. 2024-03-16T09:13:38.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.232+0000 7f196b71b640 1 Processor -- start 2024-03-16T09:13:38.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.232+0000 7f196b71b640 1 -- start start 2024-03-16T09:13:38.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f196b71b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1964100700 0x7f196406a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:38.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f196b71b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19641018f0 0x7f196419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:38.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f196b71b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 0x7f196419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:38.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f196b71b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f196410c4b0 con 0x7f19641022b0 2024-03-16T09:13:38.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f196b71b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f196410c330 con 0x7f1964100700 2024-03-16T09:13:38.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f196b71b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f196410c630 con 0x7f19641018f0 2024-03-16T09:13:38.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f1969cb8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 0x7f196419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:38.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f1969cb8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 0x7f196419eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33932/0 (socket says 172.21.15.112:33932) 2024-03-16T09:13:38.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f1969cb8640 1 -- 172.21.15.112:0/1424276652 learned_addr learned my addr 172.21.15.112:0/1424276652 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:38.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f19694b7640 1 --2- 172.21.15.112:0/1424276652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1964100700 0x7f196406a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:38.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f1968cb6640 1 --2- 172.21.15.112:0/1424276652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19641018f0 0x7f196419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:38.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f1969cb8640 1 -- 172.21.15.112:0/1424276652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19641018f0 msgr2=0x7f196419c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:38.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f1969cb8640 1 --2- 172.21.15.112:0/1424276652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19641018f0 0x7f196419c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:38.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f1969cb8640 1 -- 172.21.15.112:0/1424276652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1964100700 msgr2=0x7f196406a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:38.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f1969cb8640 1 --2- 172.21.15.112:0/1424276652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1964100700 0x7f196406a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:38.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f1969cb8640 1 -- 172.21.15.112:0/1424276652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196419f7e0 con 0x7f19641022b0 2024-03-16T09:13:38.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f1969cb8640 1 --2- 172.21.15.112:0/1424276652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 0x7f196419eed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f195c02c700 tx=0x7f195c0346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:38.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f195a7fc640 1 -- 172.21.15.112:0/1424276652 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c039040 con 0x7f19641022b0 2024-03-16T09:13:38.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f195a7fc640 1 -- 172.21.15.112:0/1424276652 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f195c044070 con 0x7f19641022b0 2024-03-16T09:13:38.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f195a7fc640 1 -- 172.21.15.112:0/1424276652 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c0044d0 con 0x7f19641022b0 2024-03-16T09:13:38.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f196b71b640 1 -- 172.21.15.112:0/1424276652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196419fa10 con 0x7f19641022b0 2024-03-16T09:13:38.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.233+0000 7f196b71b640 1 -- 172.21.15.112:0/1424276652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f196419ff00 con 0x7f19641022b0 2024-03-16T09:13:38.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.234+0000 7f196b71b640 1 -- 172.21.15.112:0/1424276652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19641079c0 con 0x7f19641022b0 2024-03-16T09:13:38.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.235+0000 7f195a7fc640 1 -- 172.21.15.112:0/1424276652 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f195c004670 con 0x7f19641022b0 2024-03-16T09:13:38.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.236+0000 7f195a7fc640 1 --2- 172.21.15.112:0/1424276652 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f19500a2800 0x7f19500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:38.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.236+0000 7f19694b7640 1 --2- 172.21.15.112:0/1424276652 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f19500a2800 0x7f19500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:38.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.236+0000 7f195a7fc640 1 -- 172.21.15.112:0/1424276652 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f195c0f1770 con 0x7f19641022b0 2024-03-16T09:13:38.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.236+0000 7f19694b7640 1 --2- 172.21.15.112:0/1424276652 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f19500a2800 0x7f19500a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1964101750 tx=0x7f1960005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:38.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.240+0000 7f195a7fc640 1 -- 172.21.15.112:0/1424276652 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f195c0be960 con 0x7f19641022b0 2024-03-16T09:13:38.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:38 smithi194 ceph-mon[36729]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-16T09:13:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:38 smithi194 ceph-mon[36729]: osdmap e84: 12 total, 12 up, 12 in 2024-03-16T09:13:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:38 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/836267163' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-16T09:13:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:38 smithi194 ceph-mon[36729]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-16T09:13:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:38 smithi194 ceph-mon[36729]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-16T09:13:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:38 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1000672656' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:38 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/594413266' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:38 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2673007488' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-16T09:13:38.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:38.452+0000 7f196b71b640 1 -- 172.21.15.112:0/1424276652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f196402d400 con 0x7f19641022b0 2024-03-16T09:13:38.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:38 smithi159 ceph-mon[36739]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-16T09:13:38.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:38 smithi159 ceph-mon[36739]: osdmap e84: 12 total, 12 up, 12 in 2024-03-16T09:13:38.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:38 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/836267163' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-16T09:13:38.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:38 smithi159 ceph-mon[36739]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-16T09:13:38.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:38 smithi159 ceph-mon[36739]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-16T09:13:38.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:38 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1000672656' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:38.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:38 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/594413266' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:38.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:38 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2673007488' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-16T09:13:38.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:38 smithi112 ceph-mon[32307]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-16T09:13:38.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:38 smithi112 ceph-mon[32307]: osdmap e84: 12 total, 12 up, 12 in 2024-03-16T09:13:38.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:38 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/836267163' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-16T09:13:38.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:38 smithi112 ceph-mon[32307]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-16T09:13:38.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:38 smithi112 ceph-mon[32307]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-16T09:13:38.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:38 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1000672656' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:38.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:38 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/594413266' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:38.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:38 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2673007488' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-16T09:13:39.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.113+0000 7f195a7fc640 1 -- 172.21.15.112:0/1424276652 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7f195c0be260 con 0x7f19641022b0 2024-03-16T09:13:39.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.115+0000 7f196b71b640 1 -- 172.21.15.112:0/1424276652 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f19500a2800 msgr2=0x7f19500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:39.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.115+0000 7f196b71b640 1 --2- 172.21.15.112:0/1424276652 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f19500a2800 0x7f19500a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1964101750 tx=0x7f1960005f90 comp rx=0 tx=0).stop 2024-03-16T09:13:39.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.115+0000 7f196b71b640 1 -- 172.21.15.112:0/1424276652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 msgr2=0x7f196419eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:39.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.115+0000 7f196b71b640 1 --2- 172.21.15.112:0/1424276652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 0x7f196419eed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f195c02c700 tx=0x7f195c0346d0 comp rx=0 tx=0).stop 2024-03-16T09:13:39.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.116+0000 7f196b71b640 1 -- 172.21.15.112:0/1424276652 shutdown_connections 2024-03-16T09:13:39.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.116+0000 7f196b71b640 1 --2- 172.21.15.112:0/1424276652 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f19500a2800 0x7f19500a4cc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:39.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.116+0000 7f196b71b640 1 --2- 172.21.15.112:0/1424276652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19641022b0 0x7f196419eed0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:39.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.116+0000 7f196b71b640 1 --2- 172.21.15.112:0/1424276652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19641018f0 0x7f196419c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:39.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.116+0000 7f196b71b640 1 --2- 172.21.15.112:0/1424276652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1964100700 0x7f196406a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:39.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.116+0000 7f196b71b640 1 -- 172.21.15.112:0/1424276652 >> 172.21.15.112:0/1424276652 conn(0x7f19640fbc90 msgr2=0x7f1964104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:39.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.116+0000 7f196b71b640 1 -- 172.21.15.112:0/1424276652 shutdown_connections 2024-03-16T09:13:39.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.116+0000 7f196b71b640 1 -- 172.21.15.112:0/1424276652 wait complete. 2024-03-16T09:13:39.134 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-16T09:13:39.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.239+0000 7f02f9162640 1 -- 172.21.15.112:0/715630399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f41029e0 msgr2=0x7f02f4102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:39.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.239+0000 7f02f9162640 1 --2- 172.21.15.112:0/715630399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f41029e0 0x7f02f4102e60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009900 tx=0x7f02e802bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:39.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.239+0000 7f02f9162640 1 -- 172.21.15.112:0/715630399 shutdown_connections 2024-03-16T09:13:39.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.239+0000 7f02f9162640 1 --2- 172.21.15.112:0/715630399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02f41033a0 0x7f02f4109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:39.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.239+0000 7f02f9162640 1 --2- 172.21.15.112:0/715630399 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f41029e0 0x7f02f4102e60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:39.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.239+0000 7f02f9162640 1 --2- 172.21.15.112:0/715630399 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02f41017f0 0x7f02f4101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:39.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.239+0000 7f02f9162640 1 -- 172.21.15.112:0/715630399 >> 172.21.15.112:0/715630399 conn(0x7f02f40fcd80 msgr2=0x7f02f40ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:39.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.239+0000 7f02f9162640 1 -- 172.21.15.112:0/715630399 shutdown_connections 2024-03-16T09:13:39.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.239+0000 7f02f9162640 1 -- 172.21.15.112:0/715630399 wait complete. 2024-03-16T09:13:39.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.240+0000 7f02f9162640 1 Processor -- start 2024-03-16T09:13:39.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.240+0000 7f02f9162640 1 -- start start 2024-03-16T09:13:39.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.240+0000 7f02f9162640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f41017f0 0x7f02f4197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:39.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.240+0000 7f02f9162640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02f41029e0 0x7f02f4198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:39.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.240+0000 7f02f9162640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02f41033a0 0x7f02f419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:39.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.240+0000 7f02f9162640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02f410d550 con 0x7f02f41017f0 2024-03-16T09:13:39.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.240+0000 7f02f9162640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f02f410d3d0 con 0x7f02f41029e0 2024-03-16T09:13:39.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.240+0000 7f02f9162640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f02f410d6d0 con 0x7f02f41033a0 2024-03-16T09:13:39.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.240+0000 7f02f2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f41017f0 0x7f02f4197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:39.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.240+0000 7f02f2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f41017f0 0x7f02f4197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33944/0 (socket says 172.21.15.112:33944) 2024-03-16T09:13:39.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.240+0000 7f02f2d9d640 1 -- 172.21.15.112:0/3501430590 learned_addr learned my addr 172.21.15.112:0/3501430590 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:39.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.240+0000 7f02f259c640 1 --2- 172.21.15.112:0/3501430590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02f41029e0 0x7f02f4198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:39.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.241+0000 7f02f359e640 1 --2- 172.21.15.112:0/3501430590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02f41033a0 0x7f02f419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:39.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.241+0000 7f02f2d9d640 1 -- 172.21.15.112:0/3501430590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02f41033a0 msgr2=0x7f02f419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:39.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.241+0000 7f02f2d9d640 1 --2- 172.21.15.112:0/3501430590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02f41033a0 0x7f02f419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:39.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.241+0000 7f02f2d9d640 1 -- 172.21.15.112:0/3501430590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02f41029e0 msgr2=0x7f02f4198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:39.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.241+0000 7f02f2d9d640 1 --2- 172.21.15.112:0/3501430590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02f41029e0 0x7f02f4198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:39.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.241+0000 7f02f2d9d640 1 -- 172.21.15.112:0/3501430590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f419f830 con 0x7f02f41017f0 2024-03-16T09:13:39.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.241+0000 7f02f259c640 1 --2- 172.21.15.112:0/3501430590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02f41029e0 0x7f02f4198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:13:39.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.241+0000 7f02f2d9d640 1 --2- 172.21.15.112:0/3501430590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f41017f0 0x7f02f4197c20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f02e400d950 tx=0x7f02e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:39.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.241+0000 7f02dbfff640 1 -- 172.21.15.112:0/3501430590 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e4017070 con 0x7f02f41017f0 2024-03-16T09:13:39.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.241+0000 7f02dbfff640 1 -- 172.21.15.112:0/3501430590 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f02e4009c30 con 0x7f02f41017f0 2024-03-16T09:13:39.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.241+0000 7f02f9162640 1 -- 172.21.15.112:0/3501430590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f419fb20 con 0x7f02f41017f0 2024-03-16T09:13:39.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.241+0000 7f02dbfff640 1 -- 172.21.15.112:0/3501430590 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e400f6b0 con 0x7f02f41017f0 2024-03-16T09:13:39.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.241+0000 7f02f9162640 1 -- 172.21.15.112:0/3501430590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f41a0020 con 0x7f02f41017f0 2024-03-16T09:13:39.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.242+0000 7f02f9162640 1 -- 172.21.15.112:0/3501430590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02f4108b90 con 0x7f02f41017f0 2024-03-16T09:13:39.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.243+0000 7f02dbfff640 1 -- 172.21.15.112:0/3501430590 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f02e401a070 con 0x7f02f41017f0 2024-03-16T09:13:39.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.244+0000 7f02dbfff640 1 --2- 172.21.15.112:0/3501430590 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f02dc0a2800 0x7f02dc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:39.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.244+0000 7f02f259c640 1 --2- 172.21.15.112:0/3501430590 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f02dc0a2800 0x7f02dc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:39.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.244+0000 7f02dbfff640 1 -- 172.21.15.112:0/3501430590 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f02e40ce320 con 0x7f02f41017f0 2024-03-16T09:13:39.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.244+0000 7f02f259c640 1 --2- 172.21.15.112:0/3501430590 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f02dc0a2800 0x7f02dc0a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f02e802c900 tx=0x7f02e8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:39.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.248+0000 7f02dbfff640 1 -- 172.21.15.112:0/3501430590 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f02e4009de0 con 0x7f02f41017f0 2024-03-16T09:13:39.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:39 smithi194 ceph-mon[36729]: osdmap e85: 12 total, 12 up, 12 in 2024-03-16T09:13:39.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:39 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2673007488' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-16T09:13:39.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:39 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] up:active 2024-03-16T09:13:39.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:39 smithi194 ceph-mon[36729]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T09:13:39.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:39 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1424276652' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-16T09:13:39.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:39 smithi194 ceph-mon[36729]: pgmap v289: 97 pgs: 11 creating+peering, 51 active+clean, 35 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-03-16T09:13:39.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:39.461+0000 7f02f9162640 1 -- 172.21.15.112:0/3501430590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f02f402d420 con 0x7f02f41017f0 2024-03-16T09:13:39.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:39 smithi159 ceph-mon[36739]: osdmap e85: 12 total, 12 up, 12 in 2024-03-16T09:13:39.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:39 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2673007488' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-16T09:13:39.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:39 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] up:active 2024-03-16T09:13:39.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:39 smithi159 ceph-mon[36739]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T09:13:39.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:39 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1424276652' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-16T09:13:39.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:39 smithi159 ceph-mon[36739]: pgmap v289: 97 pgs: 11 creating+peering, 51 active+clean, 35 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-03-16T09:13:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:39 smithi112 ceph-mon[32307]: osdmap e85: 12 total, 12 up, 12 in 2024-03-16T09:13:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:39 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2673007488' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-16T09:13:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:39 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] up:active 2024-03-16T09:13:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:39 smithi112 ceph-mon[32307]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T09:13:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:39 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1424276652' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-16T09:13:39.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:39 smithi112 ceph-mon[32307]: pgmap v289: 97 pgs: 11 creating+peering, 51 active+clean, 35 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-03-16T09:13:40.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.125+0000 7f02dbfff640 1 -- 172.21.15.112:0/3501430590 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f02e409aec0 con 0x7f02f41017f0 2024-03-16T09:13:40.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.127+0000 7f02f9162640 1 -- 172.21.15.112:0/3501430590 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f02dc0a2800 msgr2=0x7f02dc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:40.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.127+0000 7f02f9162640 1 --2- 172.21.15.112:0/3501430590 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f02dc0a2800 0x7f02dc0a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f02e802c900 tx=0x7f02e8005fd0 comp rx=0 tx=0).stop 2024-03-16T09:13:40.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.127+0000 7f02f9162640 1 -- 172.21.15.112:0/3501430590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f41017f0 msgr2=0x7f02f4197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:40.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.127+0000 7f02f9162640 1 --2- 172.21.15.112:0/3501430590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f41017f0 0x7f02f4197c20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f02e400d950 tx=0x7f02e400de10 comp rx=0 tx=0).stop 2024-03-16T09:13:40.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.127+0000 7f02f9162640 1 -- 172.21.15.112:0/3501430590 shutdown_connections 2024-03-16T09:13:40.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.127+0000 7f02f9162640 1 --2- 172.21.15.112:0/3501430590 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f02dc0a2800 0x7f02dc0a4cc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:40.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.127+0000 7f02f9162640 1 --2- 172.21.15.112:0/3501430590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02f41033a0 0x7f02f419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:40.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.127+0000 7f02f9162640 1 --2- 172.21.15.112:0/3501430590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02f41029e0 0x7f02f4198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:40.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.127+0000 7f02f9162640 1 --2- 172.21.15.112:0/3501430590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f41017f0 0x7f02f4197c20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:40.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.127+0000 7f02f9162640 1 -- 172.21.15.112:0/3501430590 >> 172.21.15.112:0/3501430590 conn(0x7f02f40fcd80 msgr2=0x7f02f4105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:40.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.128+0000 7f02f9162640 1 -- 172.21.15.112:0/3501430590 shutdown_connections 2024-03-16T09:13:40.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.128+0000 7f02f9162640 1 -- 172.21.15.112:0/3501430590 wait complete. 2024-03-16T09:13:40.191 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-16T09:13:40.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.296+0000 7f66c8adc640 1 -- 172.21.15.112:0/2541421176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66c40ffe80 msgr2=0x7f66c41002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:40.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.296+0000 7f66c8adc640 1 --2- 172.21.15.112:0/2541421176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66c40ffe80 0x7f66c41002a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009900 tx=0x7f66b402bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:40.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.296+0000 7f66c8adc640 1 -- 172.21.15.112:0/2541421176 shutdown_connections 2024-03-16T09:13:40.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.296+0000 7f66c8adc640 1 --2- 172.21.15.112:0/2541421176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66c4101990 0x7f66c41082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:40.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.296+0000 7f66c8adc640 1 --2- 172.21.15.112:0/2541421176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66c4100fd0 0x7f66c4101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:40.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.296+0000 7f66c8adc640 1 --2- 172.21.15.112:0/2541421176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66c40ffe80 0x7f66c41002a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:40.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.296+0000 7f66c8adc640 1 -- 172.21.15.112:0/2541421176 >> 172.21.15.112:0/2541421176 conn(0x7f66c40fb410 msgr2=0x7f66c40fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:40.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.296+0000 7f66c8adc640 1 -- 172.21.15.112:0/2541421176 shutdown_connections 2024-03-16T09:13:40.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.296+0000 7f66c8adc640 1 -- 172.21.15.112:0/2541421176 wait complete. 2024-03-16T09:13:40.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.297+0000 7f66c8adc640 1 Processor -- start 2024-03-16T09:13:40.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.297+0000 7f66c8adc640 1 -- start start 2024-03-16T09:13:40.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.297+0000 7f66c8adc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66c40ffe80 0x7f66c4197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:40.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.297+0000 7f66c8adc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66c4100fd0 0x7f66c4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:40.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.297+0000 7f66c8adc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66c4101990 0x7f66c419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:40.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.297+0000 7f66c8adc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66c410c4a0 con 0x7f66c4101990 2024-03-16T09:13:40.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.297+0000 7f66c8adc640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f66c410c320 con 0x7f66c40ffe80 2024-03-16T09:13:40.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.297+0000 7f66c8adc640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f66c410c620 con 0x7f66c4100fd0 2024-03-16T09:13:40.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.297+0000 7f66c2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66c4101990 0x7f66c419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:40.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.297+0000 7f66c2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66c4101990 0x7f66c419f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33962/0 (socket says 172.21.15.112:33962) 2024-03-16T09:13:40.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.297+0000 7f66c2d9d640 1 -- 172.21.15.112:0/3092386313 learned_addr learned my addr 172.21.15.112:0/3092386313 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:40.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.297+0000 7f66c1d9b640 1 --2- 172.21.15.112:0/3092386313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66c4100fd0 0x7f66c4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:40.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.297+0000 7f66c259c640 1 --2- 172.21.15.112:0/3092386313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66c40ffe80 0x7f66c4197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:40.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.297+0000 7f66c2d9d640 1 -- 172.21.15.112:0/3092386313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66c4100fd0 msgr2=0x7f66c4198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:40.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.297+0000 7f66c2d9d640 1 --2- 172.21.15.112:0/3092386313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66c4100fd0 0x7f66c4198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:40.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.297+0000 7f66c2d9d640 1 -- 172.21.15.112:0/3092386313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66c40ffe80 msgr2=0x7f66c4197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:40.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.297+0000 7f66c2d9d640 1 --2- 172.21.15.112:0/3092386313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66c40ffe80 0x7f66c4197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:40.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.297+0000 7f66c2d9d640 1 -- 172.21.15.112:0/3092386313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c419f840 con 0x7f66c4101990 2024-03-16T09:13:40.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.297+0000 7f66c1d9b640 1 --2- 172.21.15.112:0/3092386313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66c4100fd0 0x7f66c4198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:13:40.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.298+0000 7f66c2d9d640 1 --2- 172.21.15.112:0/3092386313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66c4101990 0x7f66c419f1a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00eb10 tx=0x7f66bc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:40.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.298+0000 7f66b37fe640 1 -- 172.21.15.112:0/3092386313 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc004b20 con 0x7f66c4101990 2024-03-16T09:13:40.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.298+0000 7f66b37fe640 1 -- 172.21.15.112:0/3092386313 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f66bc004cc0 con 0x7f66c4101990 2024-03-16T09:13:40.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.298+0000 7f66c8adc640 1 -- 172.21.15.112:0/3092386313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c419fb30 con 0x7f66c4101990 2024-03-16T09:13:40.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.298+0000 7f66b37fe640 1 -- 172.21.15.112:0/3092386313 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0107f0 con 0x7f66c4101990 2024-03-16T09:13:40.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.298+0000 7f66c8adc640 1 -- 172.21.15.112:0/3092386313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c41a00d0 con 0x7f66c4101990 2024-03-16T09:13:40.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.299+0000 7f66c8adc640 1 -- 172.21.15.112:0/3092386313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66c41a02a0 con 0x7f66c4101990 2024-03-16T09:13:40.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.300+0000 7f66b37fe640 1 -- 172.21.15.112:0/3092386313 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f66bc005000 con 0x7f66c4101990 2024-03-16T09:13:40.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.300+0000 7f66b37fe640 1 --2- 172.21.15.112:0/3092386313 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f66ac0a2880 0x7f66ac0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:40.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.301+0000 7f66c259c640 1 --2- 172.21.15.112:0/3092386313 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f66ac0a2880 0x7f66ac0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:40.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.301+0000 7f66b37fe640 1 -- 172.21.15.112:0/3092386313 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f66bc0ceff0 con 0x7f66c4101990 2024-03-16T09:13:40.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.301+0000 7f66c259c640 1 --2- 172.21.15.112:0/3092386313 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f66ac0a2880 0x7f66ac0a4d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f66c4199100 tx=0x7f66b4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:40.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.304+0000 7f66b37fe640 1 -- 172.21.15.112:0/3092386313 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f66bc053740 con 0x7f66c4101990 2024-03-16T09:13:40.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:40 smithi194 ceph-mon[36729]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-16T09:13:40.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:40 smithi194 ceph-mon[36729]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-16T09:13:40.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:40 smithi194 ceph-mon[36729]: Cluster is now healthy 2024-03-16T09:13:40.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:40 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1424276652' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-16T09:13:40.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:40 smithi194 ceph-mon[36729]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-16T09:13:40.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:40 smithi194 ceph-mon[36729]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T09:13:40.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:40 smithi194 ceph-mon[36729]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-16T09:13:40.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:40 smithi194 ceph-mon[36729]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-16T09:13:40.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:40 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3501430590' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-16T09:13:40.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:40 smithi159 ceph-mon[36739]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-16T09:13:40.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:40 smithi159 ceph-mon[36739]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-16T09:13:40.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:40 smithi159 ceph-mon[36739]: Cluster is now healthy 2024-03-16T09:13:40.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:40 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1424276652' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-16T09:13:40.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:40 smithi159 ceph-mon[36739]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-16T09:13:40.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:40 smithi159 ceph-mon[36739]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T09:13:40.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:40 smithi159 ceph-mon[36739]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-16T09:13:40.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:40 smithi159 ceph-mon[36739]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-16T09:13:40.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:40 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3501430590' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-16T09:13:40.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:40.526+0000 7f66c8adc640 1 -- 172.21.15.112:0/3092386313 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f66c41a0470 con 0x7f66c4101990 2024-03-16T09:13:40.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:40 smithi112 ceph-mon[32307]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-16T09:13:40.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:40 smithi112 ceph-mon[32307]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-16T09:13:40.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:40 smithi112 ceph-mon[32307]: Cluster is now healthy 2024-03-16T09:13:40.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:40 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1424276652' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-16T09:13:40.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:40 smithi112 ceph-mon[32307]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-16T09:13:40.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:40 smithi112 ceph-mon[32307]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-16T09:13:40.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:40 smithi112 ceph-mon[32307]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-16T09:13:40.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:40 smithi112 ceph-mon[32307]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-16T09:13:40.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:40 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3501430590' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-16T09:13:41.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.326+0000 7f66b37fe640 1 -- 172.21.15.112:0/3092386313 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f66bc09c1e0 con 0x7f66c4101990 2024-03-16T09:13:41.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.329+0000 7f66c8adc640 1 -- 172.21.15.112:0/3092386313 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f66ac0a2880 msgr2=0x7f66ac0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:41.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.329+0000 7f66c8adc640 1 --2- 172.21.15.112:0/3092386313 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f66ac0a2880 0x7f66ac0a4d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f66c4199100 tx=0x7f66b4005fd0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.329+0000 7f66c8adc640 1 -- 172.21.15.112:0/3092386313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66c4101990 msgr2=0x7f66c419f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:41.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.329+0000 7f66c8adc640 1 --2- 172.21.15.112:0/3092386313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66c4101990 0x7f66c419f1a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00eb10 tx=0x7f66bc00c2d0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.329+0000 7f66c8adc640 1 -- 172.21.15.112:0/3092386313 shutdown_connections 2024-03-16T09:13:41.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.329+0000 7f66c8adc640 1 --2- 172.21.15.112:0/3092386313 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f66ac0a2880 0x7f66ac0a4d40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.329+0000 7f66c8adc640 1 --2- 172.21.15.112:0/3092386313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66c4101990 0x7f66c419f1a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.329+0000 7f66c8adc640 1 --2- 172.21.15.112:0/3092386313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66c4100fd0 0x7f66c4198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.329+0000 7f66c8adc640 1 --2- 172.21.15.112:0/3092386313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66c40ffe80 0x7f66c4197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.329+0000 7f66c8adc640 1 -- 172.21.15.112:0/3092386313 >> 172.21.15.112:0/3092386313 conn(0x7f66c40fb410 msgr2=0x7f66c4108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:41.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.329+0000 7f66c8adc640 1 -- 172.21.15.112:0/3092386313 shutdown_connections 2024-03-16T09:13:41.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.329+0000 7f66c8adc640 1 -- 172.21.15.112:0/3092386313 wait complete. 2024-03-16T09:13:41.344 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-03-16T09:13:41.345 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-16T09:13:41.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:41 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3501430590' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-16T09:13:41.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:41 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] up:active 2024-03-16T09:13:41.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:41 smithi194 ceph-mon[36729]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-16T09:13:41.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:41 smithi194 ceph-mon[36729]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-16T09:13:41.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:41 smithi194 ceph-mon[36729]: Cluster is now healthy 2024-03-16T09:13:41.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:41 smithi194 ceph-mon[36729]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-16T09:13:41.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:41 smithi194 ceph-mon[36729]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-16T09:13:41.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:41 smithi194 ceph-mon[36729]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-16T09:13:41.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:41 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] up:active 2024-03-16T09:13:41.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:41 smithi194 ceph-mon[36729]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:13:41.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:41 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3092386313' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-16T09:13:41.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:41 smithi194 ceph-mon[36729]: pgmap v290: 97 pgs: 11 creating+peering, 80 active+clean, 6 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 5 op/s 2024-03-16T09:13:41.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.452+0000 7fef13236640 1 -- 172.21.15.112:0/109291963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef0c103450 msgr2=0x7fef0c109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:41.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.452+0000 7fef13236640 1 --2- 172.21.15.112:0/109291963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef0c103450 0x7fef0c109ce0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef04009cd0 tx=0x7fef0402c0f0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.452+0000 7fef13236640 1 -- 172.21.15.112:0/109291963 shutdown_connections 2024-03-16T09:13:41.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.452+0000 7fef13236640 1 --2- 172.21.15.112:0/109291963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef0c103450 0x7fef0c109ce0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.452+0000 7fef13236640 1 --2- 172.21.15.112:0/109291963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef0c102a90 0x7fef0c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.452+0000 7fef13236640 1 --2- 172.21.15.112:0/109291963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef0c0ffac0 0x7fef0c0ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.452+0000 7fef13236640 1 -- 172.21.15.112:0/109291963 >> 172.21.15.112:0/109291963 conn(0x7fef0c0fb3f0 msgr2=0x7fef0c0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:41.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.452+0000 7fef13236640 1 -- 172.21.15.112:0/109291963 shutdown_connections 2024-03-16T09:13:41.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.452+0000 7fef13236640 1 -- 172.21.15.112:0/109291963 wait complete. 2024-03-16T09:13:41.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef13236640 1 Processor -- start 2024-03-16T09:13:41.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef13236640 1 -- start start 2024-03-16T09:13:41.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef13236640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef0c0ffac0 0x7fef0c197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:41.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef13236640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef0c102a90 0x7fef0c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:41.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef13236640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef0c103450 0x7fef0c19f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:41.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef13236640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c10d5d0 con 0x7fef0c103450 2024-03-16T09:13:41.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef13236640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c10d450 con 0x7fef0c102a90 2024-03-16T09:13:41.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef13236640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c10d750 con 0x7fef0c0ffac0 2024-03-16T09:13:41.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef117d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef0c103450 0x7fef0c19f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:41.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef117d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef0c103450 0x7fef0c19f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59050/0 (socket says 172.21.15.112:59050) 2024-03-16T09:13:41.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef117d3640 1 -- 172.21.15.112:0/724707320 learned_addr learned my addr 172.21.15.112:0/724707320 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:41.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef0bfff640 1 --2- 172.21.15.112:0/724707320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef0c102a90 0x7fef0c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:41.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef10fd2640 1 --2- 172.21.15.112:0/724707320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef0c0ffac0 0x7fef0c197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:41.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef117d3640 1 -- 172.21.15.112:0/724707320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef0c0ffac0 msgr2=0x7fef0c197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:41.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef117d3640 1 --2- 172.21.15.112:0/724707320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef0c0ffac0 0x7fef0c197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef117d3640 1 -- 172.21.15.112:0/724707320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef0c102a90 msgr2=0x7fef0c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:41.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef117d3640 1 --2- 172.21.15.112:0/724707320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef0c102a90 0x7fef0c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef117d3640 1 -- 172.21.15.112:0/724707320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0c19f870 con 0x7fef0c103450 2024-03-16T09:13:41.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef117d3640 1 --2- 172.21.15.112:0/724707320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef0c103450 0x7fef0c19f1d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fef04009ca0 tx=0x7fef04035d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:41.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef09ffb640 1 -- 172.21.15.112:0/724707320 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef04039040 con 0x7fef0c103450 2024-03-16T09:13:41.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef09ffb640 1 -- 172.21.15.112:0/724707320 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fef04044070 con 0x7fef0c103450 2024-03-16T09:13:41.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.453+0000 7fef09ffb640 1 -- 172.21.15.112:0/724707320 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef040044a0 con 0x7fef0c103450 2024-03-16T09:13:41.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.454+0000 7fef13236640 1 -- 172.21.15.112:0/724707320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0c19fb00 con 0x7fef0c103450 2024-03-16T09:13:41.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.454+0000 7fef13236640 1 -- 172.21.15.112:0/724707320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef0c19fff0 con 0x7fef0c103450 2024-03-16T09:13:41.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.455+0000 7fef13236640 1 -- 172.21.15.112:0/724707320 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef0c108af0 con 0x7fef0c103450 2024-03-16T09:13:41.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.456+0000 7fef09ffb640 1 -- 172.21.15.112:0/724707320 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fef04004640 con 0x7fef0c103450 2024-03-16T09:13:41.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.456+0000 7fef09ffb640 1 --2- 172.21.15.112:0/724707320 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feefc0a2800 0x7feefc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:41.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.456+0000 7fef10fd2640 1 --2- 172.21.15.112:0/724707320 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feefc0a2800 0x7feefc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:41.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.456+0000 7fef09ffb640 1 -- 172.21.15.112:0/724707320 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fef040f15a0 con 0x7fef0c103450 2024-03-16T09:13:41.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.457+0000 7fef10fd2640 1 --2- 172.21.15.112:0/724707320 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feefc0a2800 0x7feefc0a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fef0c199130 tx=0x7fef00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:41.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.460+0000 7fef09ffb640 1 -- 172.21.15.112:0/724707320 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fef040be790 con 0x7fef0c103450 2024-03-16T09:13:41.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:41 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3501430590' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-16T09:13:41.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:41 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] up:active 2024-03-16T09:13:41.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:41 smithi159 ceph-mon[36739]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-16T09:13:41.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:41 smithi159 ceph-mon[36739]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-16T09:13:41.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:41 smithi159 ceph-mon[36739]: Cluster is now healthy 2024-03-16T09:13:41.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:41 smithi159 ceph-mon[36739]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-16T09:13:41.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:41 smithi159 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-16T09:13:41.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:41 smithi159 ceph-mon[36739]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-16T09:13:41.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:41 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] up:active 2024-03-16T09:13:41.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:41 smithi159 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:13:41.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:41 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3092386313' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-16T09:13:41.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:41 smithi159 ceph-mon[36739]: pgmap v290: 97 pgs: 11 creating+peering, 80 active+clean, 6 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 5 op/s 2024-03-16T09:13:41.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:41 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3501430590' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-16T09:13:41.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:41 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] up:active 2024-03-16T09:13:41.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:41 smithi112 ceph-mon[32307]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-16T09:13:41.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:41 smithi112 ceph-mon[32307]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-16T09:13:41.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:41 smithi112 ceph-mon[32307]: Cluster is now healthy 2024-03-16T09:13:41.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:41 smithi112 ceph-mon[32307]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-16T09:13:41.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:41 smithi112 ceph-mon[32307]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-16T09:13:41.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:41 smithi112 ceph-mon[32307]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-16T09:13:41.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:41 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] up:active 2024-03-16T09:13:41.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:41 smithi112 ceph-mon[32307]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:13:41.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:41 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3092386313' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-16T09:13:41.593 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:41 smithi112 ceph-mon[32307]: pgmap v290: 97 pgs: 11 creating+peering, 80 active+clean, 6 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 5 op/s 2024-03-16T09:13:41.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.658+0000 7fef13236640 1 -- 172.21.15.112:0/724707320 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fef0c061c20 con 0x7feefc0a2800 2024-03-16T09:13:41.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.675+0000 7fef09ffb640 1 -- 172.21.15.112:0/724707320 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fef0c061c20 con 0x7feefc0a2800 2024-03-16T09:13:41.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.677+0000 7fef13236640 1 -- 172.21.15.112:0/724707320 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feefc0a2800 msgr2=0x7feefc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:41.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.677+0000 7fef13236640 1 --2- 172.21.15.112:0/724707320 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feefc0a2800 0x7feefc0a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fef0c199130 tx=0x7fef00005f90 comp rx=0 tx=0).stop 2024-03-16T09:13:41.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.677+0000 7fef13236640 1 -- 172.21.15.112:0/724707320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef0c103450 msgr2=0x7fef0c19f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:41.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.677+0000 7fef13236640 1 --2- 172.21.15.112:0/724707320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef0c103450 0x7fef0c19f1d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fef04009ca0 tx=0x7fef04035d30 comp rx=0 tx=0).stop 2024-03-16T09:13:41.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.677+0000 7fef13236640 1 -- 172.21.15.112:0/724707320 shutdown_connections 2024-03-16T09:13:41.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.677+0000 7fef13236640 1 --2- 172.21.15.112:0/724707320 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feefc0a2800 0x7feefc0a4cc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.677+0000 7fef13236640 1 --2- 172.21.15.112:0/724707320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef0c103450 0x7fef0c19f1d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.677+0000 7fef13236640 1 --2- 172.21.15.112:0/724707320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef0c102a90 0x7fef0c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.677+0000 7fef13236640 1 --2- 172.21.15.112:0/724707320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef0c0ffac0 0x7fef0c197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.677+0000 7fef13236640 1 -- 172.21.15.112:0/724707320 >> 172.21.15.112:0/724707320 conn(0x7fef0c0fb3f0 msgr2=0x7fef0c105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:41.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.677+0000 7fef13236640 1 -- 172.21.15.112:0/724707320 shutdown_connections 2024-03-16T09:13:41.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.677+0000 7fef13236640 1 -- 172.21.15.112:0/724707320 wait complete. 2024-03-16T09:13:41.691 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-16T09:13:41.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.795+0000 7fa6e76c0640 1 -- 172.21.15.112:0/4129184437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6e01018f0 msgr2=0x7fa6e0101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.795+0000 7fa6e76c0640 1 --2- 172.21.15.112:0/4129184437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6e01018f0 0x7fa6e0101d70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009860 tx=0x7fa6d002bb70 comp rx=0 tx=0).stop 2024-03-16T09:13:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.795+0000 7fa6e76c0640 1 -- 172.21.15.112:0/4129184437 shutdown_connections 2024-03-16T09:13:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.795+0000 7fa6e76c0640 1 --2- 172.21.15.112:0/4129184437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6e01022b0 0x7fa6e0108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.795+0000 7fa6e76c0640 1 --2- 172.21.15.112:0/4129184437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6e01018f0 0x7fa6e0101d70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.795+0000 7fa6e76c0640 1 --2- 172.21.15.112:0/4129184437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e0100700 0x7fa6e0100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.795+0000 7fa6e76c0640 1 -- 172.21.15.112:0/4129184437 >> 172.21.15.112:0/4129184437 conn(0x7fa6e00fbc90 msgr2=0x7fa6e00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.796+0000 7fa6e76c0640 1 -- 172.21.15.112:0/4129184437 shutdown_connections 2024-03-16T09:13:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.796+0000 7fa6e76c0640 1 -- 172.21.15.112:0/4129184437 wait complete. 2024-03-16T09:13:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.796+0000 7fa6e76c0640 1 Processor -- start 2024-03-16T09:13:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.796+0000 7fa6e76c0640 1 -- start start 2024-03-16T09:13:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.796+0000 7fa6e76c0640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6e0100700 0x7fa6e006a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.796+0000 7fa6e76c0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6e01018f0 0x7fa6e019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.796+0000 7fa6e76c0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e01022b0 0x7fa6e019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.796+0000 7fa6e76c0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e010c4b0 con 0x7fa6e01022b0 2024-03-16T09:13:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.796+0000 7fa6e76c0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e010c330 con 0x7fa6e01018f0 2024-03-16T09:13:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.796+0000 7fa6e76c0640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e010c630 con 0x7fa6e0100700 2024-03-16T09:13:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.796+0000 7fa6e5c5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e01022b0 0x7fa6e019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.796+0000 7fa6e545c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6e0100700 0x7fa6e006a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.796+0000 7fa6e5c5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e01022b0 0x7fa6e019ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59070/0 (socket says 172.21.15.112:59070) 2024-03-16T09:13:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.796+0000 7fa6e5c5d640 1 -- 172.21.15.112:0/1262767734 learned_addr learned my addr 172.21.15.112:0/1262767734 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.796+0000 7fa6e4c5b640 1 --2- 172.21.15.112:0/1262767734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6e01018f0 0x7fa6e019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.797+0000 7fa6e5c5d640 1 -- 172.21.15.112:0/1262767734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6e0100700 msgr2=0x7fa6e006a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.797+0000 7fa6e5c5d640 1 --2- 172.21.15.112:0/1262767734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6e0100700 0x7fa6e006a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.797+0000 7fa6e5c5d640 1 -- 172.21.15.112:0/1262767734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6e01018f0 msgr2=0x7fa6e019c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.797+0000 7fa6e5c5d640 1 --2- 172.21.15.112:0/1262767734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6e01018f0 0x7fa6e019c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.797+0000 7fa6e5c5d640 1 -- 172.21.15.112:0/1262767734 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e019f830 con 0x7fa6e01022b0 2024-03-16T09:13:41.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.797+0000 7fa6e545c640 1 --2- 172.21.15.112:0/1262767734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6e0100700 0x7fa6e006a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:13:41.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.797+0000 7fa6e4c5b640 1 --2- 172.21.15.112:0/1262767734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6e01018f0 0x7fa6e019c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:13:41.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.797+0000 7fa6e5c5d640 1 --2- 172.21.15.112:0/1262767734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e01022b0 0x7fa6e019ef20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800eb10 tx=0x7fa6d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:41.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.797+0000 7fa6d67fc640 1 -- 172.21.15.112:0/1262767734 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8004b20 con 0x7fa6e01022b0 2024-03-16T09:13:41.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.797+0000 7fa6d67fc640 1 -- 172.21.15.112:0/1262767734 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa6d8004cc0 con 0x7fa6e01022b0 2024-03-16T09:13:41.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.797+0000 7fa6e76c0640 1 -- 172.21.15.112:0/1262767734 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e019fac0 con 0x7fa6e01022b0 2024-03-16T09:13:41.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.797+0000 7fa6d67fc640 1 -- 172.21.15.112:0/1262767734 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d80107f0 con 0x7fa6e01022b0 2024-03-16T09:13:41.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.797+0000 7fa6e76c0640 1 -- 172.21.15.112:0/1262767734 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e019ff90 con 0x7fa6e01022b0 2024-03-16T09:13:41.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.798+0000 7fa6e76c0640 1 -- 172.21.15.112:0/1262767734 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6e0107a20 con 0x7fa6e01022b0 2024-03-16T09:13:41.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.799+0000 7fa6d67fc640 1 -- 172.21.15.112:0/1262767734 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fa6d8010990 con 0x7fa6e01022b0 2024-03-16T09:13:41.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.800+0000 7fa6d67fc640 1 --2- 172.21.15.112:0/1262767734 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa6cc0a2800 0x7fa6cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:41.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.800+0000 7fa6e545c640 1 --2- 172.21.15.112:0/1262767734 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa6cc0a2800 0x7fa6cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:41.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.800+0000 7fa6d67fc640 1 -- 172.21.15.112:0/1262767734 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fa6d80cf0d0 con 0x7fa6e01022b0 2024-03-16T09:13:41.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.800+0000 7fa6e545c640 1 --2- 172.21.15.112:0/1262767734 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa6cc0a2800 0x7fa6cc0a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0096c0 tx=0x7fa6dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:41.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:41.803+0000 7fa6d67fc640 1 -- 172.21.15.112:0/1262767734 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa6d8053740 con 0x7fa6e01022b0 2024-03-16T09:13:42.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.004+0000 7fa6e76c0640 1 -- 172.21.15.112:0/1262767734 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa6e0061cb0 con 0x7fa6cc0a2800 2024-03-16T09:13:42.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.022+0000 7fa6d67fc640 1 -- 172.21.15.112:0/1262767734 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa6e0061cb0 con 0x7fa6cc0a2800 2024-03-16T09:13:42.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.024+0000 7fa6e76c0640 1 -- 172.21.15.112:0/1262767734 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa6cc0a2800 msgr2=0x7fa6cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:42.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.024+0000 7fa6e76c0640 1 --2- 172.21.15.112:0/1262767734 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa6cc0a2800 0x7fa6cc0a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0096c0 tx=0x7fa6dc009260 comp rx=0 tx=0).stop 2024-03-16T09:13:42.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.024+0000 7fa6e76c0640 1 -- 172.21.15.112:0/1262767734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e01022b0 msgr2=0x7fa6e019ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:42.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.024+0000 7fa6e76c0640 1 --2- 172.21.15.112:0/1262767734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e01022b0 0x7fa6e019ef20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800eb10 tx=0x7fa6d800c2d0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.024+0000 7fa6e76c0640 1 -- 172.21.15.112:0/1262767734 shutdown_connections 2024-03-16T09:13:42.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.025+0000 7fa6e76c0640 1 --2- 172.21.15.112:0/1262767734 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa6cc0a2800 0x7fa6cc0a4cc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.025+0000 7fa6e76c0640 1 --2- 172.21.15.112:0/1262767734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6e01022b0 0x7fa6e019ef20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.025+0000 7fa6e76c0640 1 --2- 172.21.15.112:0/1262767734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6e01018f0 0x7fa6e019c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.025+0000 7fa6e76c0640 1 --2- 172.21.15.112:0/1262767734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6e0100700 0x7fa6e006a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.025+0000 7fa6e76c0640 1 -- 172.21.15.112:0/1262767734 >> 172.21.15.112:0/1262767734 conn(0x7fa6e00fbc90 msgr2=0x7fa6e0104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:42.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.025+0000 7fa6e76c0640 1 -- 172.21.15.112:0/1262767734 shutdown_connections 2024-03-16T09:13:42.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.025+0000 7fa6e76c0640 1 -- 172.21.15.112:0/1262767734 wait complete. 2024-03-16T09:13:42.036 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:13:41 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:13:41.663+0000 7f4f8b2fb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T09:13:42.036 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:13:41 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:13:41.663+0000 7f4f8b2fb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T09:13:42.036 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:13:41 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:13:41.663+0000 7f4f8b2fb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T09:13:42.037 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:13:41 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:13:41.663+0000 7f4f8b2fb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T09:13:42.037 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:13:41 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:13:41.663+0000 7f4f8b2fb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T09:13:42.038 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-16T09:13:42.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.142+0000 7fa0fcba9640 1 -- 172.21.15.112:0/2823519512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f8102290 msgr2=0x7fa0f8108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:42.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.142+0000 7fa0fcba9640 1 --2- 172.21.15.112:0/2823519512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f8102290 0x7fa0f8108b20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009ca0 tx=0x7fa0f002c0f0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.142+0000 7fa0fcba9640 1 -- 172.21.15.112:0/2823519512 shutdown_connections 2024-03-16T09:13:42.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.142+0000 7fa0fcba9640 1 --2- 172.21.15.112:0/2823519512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f8102290 0x7fa0f8108b20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.142+0000 7fa0fcba9640 1 --2- 172.21.15.112:0/2823519512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0f81018d0 0x7fa0f8101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.142+0000 7fa0fcba9640 1 --2- 172.21.15.112:0/2823519512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0f81006e0 0x7fa0f8100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.142+0000 7fa0fcba9640 1 -- 172.21.15.112:0/2823519512 >> 172.21.15.112:0/2823519512 conn(0x7fa0f80fbc70 msgr2=0x7fa0f80fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:42.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.142+0000 7fa0fcba9640 1 -- 172.21.15.112:0/2823519512 shutdown_connections 2024-03-16T09:13:42.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.142+0000 7fa0fcba9640 1 -- 172.21.15.112:0/2823519512 wait complete. 2024-03-16T09:13:42.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.143+0000 7fa0fcba9640 1 Processor -- start 2024-03-16T09:13:42.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.143+0000 7fa0fcba9640 1 -- start start 2024-03-16T09:13:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.143+0000 7fa0fcba9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0f81006e0 0x7fa0f80694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.143+0000 7fa0fcba9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f81018d0 0x7fa0f819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.143+0000 7fa0fcba9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0f8102290 0x7fa0f819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.143+0000 7fa0fcba9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f810c480 con 0x7fa0f81018d0 2024-03-16T09:13:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.143+0000 7fa0fcba9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f810c300 con 0x7fa0f8102290 2024-03-16T09:13:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.143+0000 7fa0fcba9640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f810c600 con 0x7fa0f81006e0 2024-03-16T09:13:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.144+0000 7fa0edd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f81018d0 0x7fa0f819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.144+0000 7fa0edd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f81018d0 0x7fa0f819c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59094/0 (socket says 172.21.15.112:59094) 2024-03-16T09:13:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.144+0000 7fa0edd9b640 1 -- 172.21.15.112:0/22008447 learned_addr learned my addr 172.21.15.112:0/22008447 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.144+0000 7fa0f659c640 1 --2- 172.21.15.112:0/22008447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0f81006e0 0x7fa0f80694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.144+0000 7fa0f6d9d640 1 --2- 172.21.15.112:0/22008447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0f8102290 0x7fa0f819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.144+0000 7fa0edd9b640 1 -- 172.21.15.112:0/22008447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0f81006e0 msgr2=0x7fa0f80694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.144+0000 7fa0edd9b640 1 --2- 172.21.15.112:0/22008447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0f81006e0 0x7fa0f80694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.144+0000 7fa0edd9b640 1 -- 172.21.15.112:0/22008447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0f8102290 msgr2=0x7fa0f819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.144+0000 7fa0edd9b640 1 --2- 172.21.15.112:0/22008447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0f8102290 0x7fa0f819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.144+0000 7fa0edd9b640 1 -- 172.21.15.112:0/22008447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f819f810 con 0x7fa0f81018d0 2024-03-16T09:13:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.144+0000 7fa0edd9b640 1 --2- 172.21.15.112:0/22008447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f81018d0 0x7fa0f819c8e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000e9f0 tx=0x7fa0e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.144+0000 7fa0effff640 1 -- 172.21.15.112:0/22008447 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e000cd80 con 0x7fa0f81018d0 2024-03-16T09:13:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.144+0000 7fa0fcba9640 1 -- 172.21.15.112:0/22008447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f819faa0 con 0x7fa0f81018d0 2024-03-16T09:13:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.144+0000 7fa0effff640 1 -- 172.21.15.112:0/22008447 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa0e0010930 con 0x7fa0f81018d0 2024-03-16T09:13:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.144+0000 7fa0effff640 1 -- 172.21.15.112:0/22008447 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0018b00 con 0x7fa0f81018d0 2024-03-16T09:13:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.144+0000 7fa0fcba9640 1 -- 172.21.15.112:0/22008447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f819fff0 con 0x7fa0f81018d0 2024-03-16T09:13:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.145+0000 7fa0fcba9640 1 -- 172.21.15.112:0/22008447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f81079a0 con 0x7fa0f81018d0 2024-03-16T09:13:42.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.146+0000 7fa0effff640 1 -- 172.21.15.112:0/22008447 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7fa0e0018ca0 con 0x7fa0f81018d0 2024-03-16T09:13:42.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.147+0000 7fa0effff640 1 --2- 172.21.15.112:0/22008447 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa0e40a2800 0x7fa0e40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:42.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.147+0000 7fa0effff640 1 -- 172.21.15.112:0/22008447 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fa0e00cec60 con 0x7fa0f81018d0 2024-03-16T09:13:42.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.147+0000 7fa0f659c640 1 --2- 172.21.15.112:0/22008447 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa0e40a2800 0x7fa0e40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:42.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.147+0000 7fa0f659c640 1 --2- 172.21.15.112:0/22008447 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa0e40a2800 0x7fa0e40a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8101730 tx=0x7fa0e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:42.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.151+0000 7fa0effff640 1 -- 172.21.15.112:0/22008447 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa0e009b800 con 0x7fa0f81018d0 2024-03-16T09:13:42.236 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:13:41 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-b[66504]: 2024-03-16T09:13:41.932+0000 7f8bf607f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-16T09:13:42.236 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:13:41 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-b[66504]: 2024-03-16T09:13:41.932+0000 7f8bf607f700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-16T09:13:42.236 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:13:41 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-b[66504]: 2024-03-16T09:13:41.932+0000 7f8bf607f700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-16T09:13:42.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.356+0000 7fa0fcba9640 1 -- 172.21.15.112:0/22008447 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fa0f8061cb0 con 0x7fa0e40a2800 2024-03-16T09:13:42.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.360+0000 7fa0effff640 1 -- 172.21.15.112:0/22008447 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fa0f8061cb0 con 0x7fa0e40a2800 2024-03-16T09:13:42.362 INFO:teuthology.orchestra.run.smithi112.stdout:/volumes/qa/sv_0/73d9dfb0-8ac0-4e46-8084-40984305ccba 2024-03-16T09:13:42.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.362+0000 7fa0fcba9640 1 -- 172.21.15.112:0/22008447 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa0e40a2800 msgr2=0x7fa0e40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:42.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.362+0000 7fa0fcba9640 1 --2- 172.21.15.112:0/22008447 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa0e40a2800 0x7fa0e40a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8101730 tx=0x7fa0e8005f90 comp rx=0 tx=0).stop 2024-03-16T09:13:42.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.362+0000 7fa0fcba9640 1 -- 172.21.15.112:0/22008447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f81018d0 msgr2=0x7fa0f819c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:42.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.362+0000 7fa0fcba9640 1 --2- 172.21.15.112:0/22008447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f81018d0 0x7fa0f819c8e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000e9f0 tx=0x7fa0e000eeb0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.362+0000 7fa0fcba9640 1 -- 172.21.15.112:0/22008447 shutdown_connections 2024-03-16T09:13:42.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.362+0000 7fa0fcba9640 1 --2- 172.21.15.112:0/22008447 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa0e40a2800 0x7fa0e40a4cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.362+0000 7fa0fcba9640 1 --2- 172.21.15.112:0/22008447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0f8102290 0x7fa0f819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.362+0000 7fa0fcba9640 1 --2- 172.21.15.112:0/22008447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f81018d0 0x7fa0f819c8e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.362+0000 7fa0fcba9640 1 --2- 172.21.15.112:0/22008447 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa0f81006e0 0x7fa0f80694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.362+0000 7fa0fcba9640 1 -- 172.21.15.112:0/22008447 >> 172.21.15.112:0/22008447 conn(0x7fa0f80fbc70 msgr2=0x7fa0f8104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:42.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.362+0000 7fa0fcba9640 1 -- 172.21.15.112:0/22008447 shutdown_connections 2024-03-16T09:13:42.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.362+0000 7fa0fcba9640 1 -- 172.21.15.112:0/22008447 wait complete. 2024-03-16T09:13:42.375 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-16T09:13:42.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.479+0000 7efd600a1640 1 -- 172.21.15.112:0/2814721976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd58102260 msgr2=0x7efd58104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:42.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.479+0000 7efd600a1640 1 --2- 172.21.15.112:0/2814721976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd58102260 0x7efd58104650 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efd54009900 tx=0x7efd5402bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:42.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.479+0000 7efd600a1640 1 -- 172.21.15.112:0/2814721976 shutdown_connections 2024-03-16T09:13:42.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.479+0000 7efd600a1640 1 --2- 172.21.15.112:0/2814721976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd581074c0 0x7efd58109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.479+0000 7efd600a1640 1 --2- 172.21.15.112:0/2814721976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd58104b90 0x7efd58106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.479+0000 7efd600a1640 1 --2- 172.21.15.112:0/2814721976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd58102260 0x7efd58104650 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.479+0000 7efd600a1640 1 -- 172.21.15.112:0/2814721976 >> 172.21.15.112:0/2814721976 conn(0x7efd580fbc70 msgr2=0x7efd580fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:42.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.479+0000 7efd600a1640 1 -- 172.21.15.112:0/2814721976 shutdown_connections 2024-03-16T09:13:42.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.479+0000 7efd600a1640 1 -- 172.21.15.112:0/2814721976 wait complete. 2024-03-16T09:13:42.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd600a1640 1 Processor -- start 2024-03-16T09:13:42.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd600a1640 1 -- start start 2024-03-16T09:13:42.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd600a1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd58102260 0x7efd58197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:42.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd600a1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd58104b90 0x7efd581981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:42.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd600a1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd581074c0 0x7efd5819f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:42.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd600a1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd5810d540 con 0x7efd58104b90 2024-03-16T09:13:42.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd600a1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efd5810d3c0 con 0x7efd581074c0 2024-03-16T09:13:42.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd600a1640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efd5810d6c0 con 0x7efd58102260 2024-03-16T09:13:42.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd5d63c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd58104b90 0x7efd581981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd5d63c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd58104b90 0x7efd581981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59104/0 (socket says 172.21.15.112:59104) 2024-03-16T09:13:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd5d63c640 1 -- 172.21.15.112:0/640174801 learned_addr learned my addr 172.21.15.112:0/640174801 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd5de3d640 1 --2- 172.21.15.112:0/640174801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd58102260 0x7efd58197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd5e63e640 1 --2- 172.21.15.112:0/640174801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd581074c0 0x7efd5819f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd5d63c640 1 -- 172.21.15.112:0/640174801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd58102260 msgr2=0x7efd58197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd5d63c640 1 --2- 172.21.15.112:0/640174801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd58102260 0x7efd58197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd5d63c640 1 -- 172.21.15.112:0/640174801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd581074c0 msgr2=0x7efd5819f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd5d63c640 1 --2- 172.21.15.112:0/640174801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd581074c0 0x7efd5819f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd5d63c640 1 -- 172.21.15.112:0/640174801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5819f870 con 0x7efd58104b90 2024-03-16T09:13:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd5d63c640 1 --2- 172.21.15.112:0/640174801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd58104b90 0x7efd581981a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efd4800e9a0 tx=0x7efd4800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd4effd640 1 -- 172.21.15.112:0/640174801 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4800cd80 con 0x7efd58104b90 2024-03-16T09:13:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd4effd640 1 -- 172.21.15.112:0/640174801 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efd480035f0 con 0x7efd58104b90 2024-03-16T09:13:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd600a1640 1 -- 172.21.15.112:0/640174801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5819fb60 con 0x7efd58104b90 2024-03-16T09:13:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd4effd640 1 -- 172.21.15.112:0/640174801 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48010700 con 0x7efd58104b90 2024-03-16T09:13:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.480+0000 7efd600a1640 1 -- 172.21.15.112:0/640174801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd581a00d0 con 0x7efd58104b90 2024-03-16T09:13:42.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.481+0000 7efd600a1640 1 -- 172.21.15.112:0/640174801 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd580fd760 con 0x7efd58104b90 2024-03-16T09:13:42.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.483+0000 7efd4effd640 1 -- 172.21.15.112:0/640174801 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7efd480037a0 con 0x7efd58104b90 2024-03-16T09:13:42.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.483+0000 7efd4effd640 1 --2- 172.21.15.112:0/640174801 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efd440a2880 0x7efd440a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:42.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.483+0000 7efd5de3d640 1 --2- 172.21.15.112:0/640174801 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efd440a2880 0x7efd440a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:42.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.483+0000 7efd4effd640 1 -- 172.21.15.112:0/640174801 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7efd480cece0 con 0x7efd58104b90 2024-03-16T09:13:42.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.484+0000 7efd5de3d640 1 --2- 172.21.15.112:0/640174801 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efd440a2880 0x7efd440a4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd54000c00 tx=0x7efd54005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:42.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.487+0000 7efd4effd640 1 -- 172.21.15.112:0/640174801 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efd4809bed0 con 0x7efd58104b90 2024-03-16T09:13:42.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:42 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3092386313' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-16T09:13:42.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:42 smithi112 ceph-mon[32307]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:13:42.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:42 smithi112 ceph-mon[32307]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:13:42.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:42 smithi112 ceph-mon[32307]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:13:42.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:42 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3092386313' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-16T09:13:42.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:42 smithi194 ceph-mon[36729]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:13:42.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:42 smithi194 ceph-mon[36729]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:13:42.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:42 smithi194 ceph-mon[36729]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:13:42.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.688+0000 7efd600a1640 1 -- 172.21.15.112:0/640174801 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7efd58061c20 con 0x7efd440a2880 2024-03-16T09:13:42.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.711+0000 7efd4effd640 1 -- 172.21.15.112:0/640174801 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efd58061c20 con 0x7efd440a2880 2024-03-16T09:13:42.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.713+0000 7efd600a1640 1 -- 172.21.15.112:0/640174801 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efd440a2880 msgr2=0x7efd440a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:42.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.713+0000 7efd600a1640 1 --2- 172.21.15.112:0/640174801 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efd440a2880 0x7efd440a4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd54000c00 tx=0x7efd54005fd0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.713+0000 7efd600a1640 1 -- 172.21.15.112:0/640174801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd58104b90 msgr2=0x7efd581981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:42.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.713+0000 7efd600a1640 1 --2- 172.21.15.112:0/640174801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd58104b90 0x7efd581981a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efd4800e9a0 tx=0x7efd4800ee60 comp rx=0 tx=0).stop 2024-03-16T09:13:42.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.713+0000 7efd600a1640 1 -- 172.21.15.112:0/640174801 shutdown_connections 2024-03-16T09:13:42.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.713+0000 7efd600a1640 1 --2- 172.21.15.112:0/640174801 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efd440a2880 0x7efd440a4d40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.713+0000 7efd600a1640 1 --2- 172.21.15.112:0/640174801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd581074c0 0x7efd5819f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.713+0000 7efd600a1640 1 --2- 172.21.15.112:0/640174801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd58104b90 0x7efd581981a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.713+0000 7efd600a1640 1 --2- 172.21.15.112:0/640174801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd58102260 0x7efd58197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.713+0000 7efd600a1640 1 -- 172.21.15.112:0/640174801 >> 172.21.15.112:0/640174801 conn(0x7efd580fbc70 msgr2=0x7efd58102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:42.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.713+0000 7efd600a1640 1 -- 172.21.15.112:0/640174801 shutdown_connections 2024-03-16T09:13:42.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.714+0000 7efd600a1640 1 -- 172.21.15.112:0/640174801 wait complete. 2024-03-16T09:13:42.726 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-16T09:13:42.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:42 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3092386313' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-16T09:13:42.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:42 smithi159 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:13:42.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:42 smithi159 ceph-mon[36739]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:13:42.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:42 smithi159 ceph-mon[36739]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:13:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.831+0000 7f40e2863640 1 -- 172.21.15.112:0/4067090130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc1022a0 msgr2=0x7f40dc108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.831+0000 7f40e2863640 1 --2- 172.21.15.112:0/4067090130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc1022a0 0x7f40dc108b30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f40d4009ca0 tx=0x7f40d402c0f0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.831+0000 7f40e2863640 1 -- 172.21.15.112:0/4067090130 shutdown_connections 2024-03-16T09:13:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.831+0000 7f40e2863640 1 --2- 172.21.15.112:0/4067090130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc1022a0 0x7f40dc108b30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.831+0000 7f40e2863640 1 --2- 172.21.15.112:0/4067090130 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40dc1018e0 0x7f40dc101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.831+0000 7f40e2863640 1 --2- 172.21.15.112:0/4067090130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40dc1006f0 0x7f40dc100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.831+0000 7f40e2863640 1 -- 172.21.15.112:0/4067090130 >> 172.21.15.112:0/4067090130 conn(0x7f40dc0fbc80 msgr2=0x7f40dc0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.831+0000 7f40e2863640 1 -- 172.21.15.112:0/4067090130 shutdown_connections 2024-03-16T09:13:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.831+0000 7f40e2863640 1 -- 172.21.15.112:0/4067090130 wait complete. 2024-03-16T09:13:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.832+0000 7f40e2863640 1 Processor -- start 2024-03-16T09:13:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.832+0000 7f40e2863640 1 -- start start 2024-03-16T09:13:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.832+0000 7f40e2863640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc1006f0 0x7f40dc197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.832+0000 7f40e2863640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40dc1018e0 0x7f40dc1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.832+0000 7f40e2863640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40dc1022a0 0x7f40dc19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.832+0000 7f40e2863640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc10c4a0 con 0x7f40dc1006f0 2024-03-16T09:13:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.832+0000 7f40e2863640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc10c320 con 0x7f40dc1018e0 2024-03-16T09:13:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.832+0000 7f40e2863640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc10c620 con 0x7f40dc1022a0 2024-03-16T09:13:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.832+0000 7f40dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc1006f0 0x7f40dc197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.832+0000 7f40dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc1006f0 0x7f40dc197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59122/0 (socket says 172.21.15.112:59122) 2024-03-16T09:13:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.832+0000 7f40dbfff640 1 -- 172.21.15.112:0/575737087 learned_addr learned my addr 172.21.15.112:0/575737087 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.832+0000 7f40db7fe640 1 --2- 172.21.15.112:0/575737087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40dc1018e0 0x7f40dc1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.832+0000 7f40e0e00640 1 --2- 172.21.15.112:0/575737087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40dc1022a0 0x7f40dc19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.833+0000 7f40dbfff640 1 -- 172.21.15.112:0/575737087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40dc1022a0 msgr2=0x7f40dc19f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.833+0000 7f40dbfff640 1 --2- 172.21.15.112:0/575737087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40dc1022a0 0x7f40dc19f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.833+0000 7f40dbfff640 1 -- 172.21.15.112:0/575737087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40dc1018e0 msgr2=0x7f40dc1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.833+0000 7f40dbfff640 1 --2- 172.21.15.112:0/575737087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40dc1018e0 0x7f40dc1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.833+0000 7f40dbfff640 1 -- 172.21.15.112:0/575737087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40dc19f790 con 0x7f40dc1006f0 2024-03-16T09:13:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.833+0000 7f40dbfff640 1 --2- 172.21.15.112:0/575737087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc1006f0 0x7f40dc197b70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00e9c0 tx=0x7f40cc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.833+0000 7f40d97fa640 1 -- 172.21.15.112:0/575737087 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc00cd80 con 0x7f40dc1006f0 2024-03-16T09:13:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.833+0000 7f40d97fa640 1 -- 172.21.15.112:0/575737087 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f40cc0035f0 con 0x7f40dc1006f0 2024-03-16T09:13:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.833+0000 7f40e2863640 1 -- 172.21.15.112:0/575737087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40dc19fa80 con 0x7f40dc1006f0 2024-03-16T09:13:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.833+0000 7f40d97fa640 1 -- 172.21.15.112:0/575737087 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc010690 con 0x7f40dc1006f0 2024-03-16T09:13:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.833+0000 7f40e2863640 1 -- 172.21.15.112:0/575737087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40dc19ffd0 con 0x7f40dc1006f0 2024-03-16T09:13:42.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.834+0000 7f40e2863640 1 -- 172.21.15.112:0/575737087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40dc068a20 con 0x7f40dc1006f0 2024-03-16T09:13:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.835+0000 7f40d97fa640 1 -- 172.21.15.112:0/575737087 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f40cc0037a0 con 0x7f40dc1006f0 2024-03-16T09:13:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.835+0000 7f40d97fa640 1 --2- 172.21.15.112:0/575737087 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f40c40a2800 0x7f40c40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.835+0000 7f40db7fe640 1 --2- 172.21.15.112:0/575737087 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f40c40a2800 0x7f40c40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.836+0000 7f40d97fa640 1 -- 172.21.15.112:0/575737087 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f40cc0ced10 con 0x7f40dc1006f0 2024-03-16T09:13:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.836+0000 7f40db7fe640 1 --2- 172.21.15.112:0/575737087 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f40c40a2800 0x7f40c40a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f40dc199080 tx=0x7f40d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:42.840+0000 7f40d97fa640 1 -- 172.21.15.112:0/575737087 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f40cc09bf00 con 0x7f40dc1006f0 2024-03-16T09:13:43.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.043+0000 7f40e2863640 1 -- 172.21.15.112:0/575737087 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f40dc06a4e0 con 0x7f40c40a2800 2024-03-16T09:13:43.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.046+0000 7f40d97fa640 1 -- 172.21.15.112:0/575737087 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f40dc06a4e0 con 0x7f40c40a2800 2024-03-16T09:13:43.048 INFO:teuthology.orchestra.run.smithi112.stdout:/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff 2024-03-16T09:13:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.048+0000 7f40e2863640 1 -- 172.21.15.112:0/575737087 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f40c40a2800 msgr2=0x7f40c40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.048+0000 7f40e2863640 1 --2- 172.21.15.112:0/575737087 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f40c40a2800 0x7f40c40a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f40dc199080 tx=0x7f40d0005f90 comp rx=0 tx=0).stop 2024-03-16T09:13:43.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.048+0000 7f40e2863640 1 -- 172.21.15.112:0/575737087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc1006f0 msgr2=0x7f40dc197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.048+0000 7f40e2863640 1 --2- 172.21.15.112:0/575737087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc1006f0 0x7f40dc197b70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00e9c0 tx=0x7f40cc00ee80 comp rx=0 tx=0).stop 2024-03-16T09:13:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.048+0000 7f40e2863640 1 -- 172.21.15.112:0/575737087 shutdown_connections 2024-03-16T09:13:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.048+0000 7f40e2863640 1 --2- 172.21.15.112:0/575737087 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f40c40a2800 0x7f40c40a4cc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.048+0000 7f40e2863640 1 --2- 172.21.15.112:0/575737087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40dc1022a0 0x7f40dc19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.048+0000 7f40e2863640 1 --2- 172.21.15.112:0/575737087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40dc1018e0 0x7f40dc1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.048+0000 7f40e2863640 1 --2- 172.21.15.112:0/575737087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc1006f0 0x7f40dc197b70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.048+0000 7f40e2863640 1 -- 172.21.15.112:0/575737087 >> 172.21.15.112:0/575737087 conn(0x7f40dc0fbc80 msgr2=0x7f40dc069dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.048+0000 7f40e2863640 1 -- 172.21.15.112:0/575737087 shutdown_connections 2024-03-16T09:13:43.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.048+0000 7f40e2863640 1 -- 172.21.15.112:0/575737087 wait complete. 2024-03-16T09:13:43.061 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:13:43.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.161+0000 7f5dbea63640 1 -- 172.21.15.112:0/1333524222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5db8104c10 msgr2=0x7f5db8107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:43.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.161+0000 7f5dbea63640 1 --2- 172.21.15.112:0/1333524222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5db8104c10 0x7f5db8107000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009900 tx=0x7f5da002bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:43.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.162+0000 7f5dbea63640 1 -- 172.21.15.112:0/1333524222 shutdown_connections 2024-03-16T09:13:43.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.162+0000 7f5dbea63640 1 --2- 172.21.15.112:0/1333524222 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5db8107540 0x7f5db8109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.162+0000 7f5dbea63640 1 --2- 172.21.15.112:0/1333524222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5db8104c10 0x7f5db8107000 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.162+0000 7f5dbea63640 1 --2- 172.21.15.112:0/1333524222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5db81022e0 0x7f5db81046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.162+0000 7f5dbea63640 1 -- 172.21.15.112:0/1333524222 >> 172.21.15.112:0/1333524222 conn(0x7f5db80fbc70 msgr2=0x7f5db80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:43.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.162+0000 7f5dbea63640 1 -- 172.21.15.112:0/1333524222 shutdown_connections 2024-03-16T09:13:43.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.162+0000 7f5dbea63640 1 -- 172.21.15.112:0/1333524222 wait complete. 2024-03-16T09:13:43.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.162+0000 7f5dbea63640 1 Processor -- start 2024-03-16T09:13:43.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.162+0000 7f5dbea63640 1 -- start start 2024-03-16T09:13:43.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.162+0000 7f5dbea63640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5db81022e0 0x7f5db8197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:43.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.162+0000 7f5dbea63640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5db8104c10 0x7f5db8198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:43.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.162+0000 7f5dbea63640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5db8107540 0x7f5db819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:43.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.162+0000 7f5dbea63640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5db810d560 con 0x7f5db8104c10 2024-03-16T09:13:43.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.162+0000 7f5dbea63640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5db810d3e0 con 0x7f5db8107540 2024-03-16T09:13:43.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.162+0000 7f5dbea63640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5db810d6e0 con 0x7f5db81022e0 2024-03-16T09:13:43.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.162+0000 7f5daf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5db8104c10 0x7f5db8198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:43.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.162+0000 7f5daf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5db8104c10 0x7f5db8198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59146/0 (socket says 172.21.15.112:59146) 2024-03-16T09:13:43.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.162+0000 7f5daf7fe640 1 -- 172.21.15.112:0/230924235 learned_addr learned my addr 172.21.15.112:0/230924235 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:43.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.162+0000 7f5db7fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5db81022e0 0x7f5db8197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:43.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.163+0000 7f5dbd000640 1 --2- 172.21.15.112:0/230924235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5db8107540 0x7f5db819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:43.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.163+0000 7f5daf7fe640 1 -- 172.21.15.112:0/230924235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5db81022e0 msgr2=0x7f5db8197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:43.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.163+0000 7f5daf7fe640 1 --2- 172.21.15.112:0/230924235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5db81022e0 0x7f5db8197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.163+0000 7f5daf7fe640 1 -- 172.21.15.112:0/230924235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5db8107540 msgr2=0x7f5db819f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:43.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.163+0000 7f5daf7fe640 1 --2- 172.21.15.112:0/230924235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5db8107540 0x7f5db819f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.163+0000 7f5daf7fe640 1 -- 172.21.15.112:0/230924235 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db819f830 con 0x7f5db8104c10 2024-03-16T09:13:43.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.163+0000 7f5daf7fe640 1 --2- 172.21.15.112:0/230924235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5db8104c10 0x7f5db8198160 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5da0000c00 tx=0x7f5da0004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:43.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.163+0000 7f5db5ffb640 1 -- 172.21.15.112:0/230924235 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da0043020 con 0x7f5db8104c10 2024-03-16T09:13:43.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.163+0000 7f5dbea63640 1 -- 172.21.15.112:0/230924235 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db819fac0 con 0x7f5db8104c10 2024-03-16T09:13:43.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.163+0000 7f5db5ffb640 1 -- 172.21.15.112:0/230924235 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5da0004250 con 0x7f5db8104c10 2024-03-16T09:13:43.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.163+0000 7f5db5ffb640 1 -- 172.21.15.112:0/230924235 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da0003b90 con 0x7f5db8104c10 2024-03-16T09:13:43.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.163+0000 7f5dbea63640 1 -- 172.21.15.112:0/230924235 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db81a0010 con 0x7f5db8104c10 2024-03-16T09:13:43.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.164+0000 7f5dbea63640 1 -- 172.21.15.112:0/230924235 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db80fd760 con 0x7f5db8104c10 2024-03-16T09:13:43.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.165+0000 7f5db5ffb640 1 -- 172.21.15.112:0/230924235 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f5da0046020 con 0x7f5db8104c10 2024-03-16T09:13:43.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.166+0000 7f5db5ffb640 1 --2- 172.21.15.112:0/230924235 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5da40a2800 0x7f5da40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:43.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.166+0000 7f5db7fff640 1 --2- 172.21.15.112:0/230924235 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5da40a2800 0x7f5da40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:43.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.166+0000 7f5db5ffb640 1 -- 172.21.15.112:0/230924235 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f5da00f0a80 con 0x7f5db8104c10 2024-03-16T09:13:43.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.166+0000 7f5db7fff640 1 --2- 172.21.15.112:0/230924235 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5da40a2800 0x7f5da40a4cc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5db81990f0 tx=0x7f5da8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:43.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.170+0000 7f5db5ffb640 1 -- 172.21.15.112:0/230924235 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5da004d020 con 0x7f5db8104c10 2024-03-16T09:13:43.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.386+0000 7f5dbea63640 1 -- 172.21.15.112:0/230924235 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5db802d3b0 con 0x7f5db8104c10 2024-03-16T09:13:43.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.387+0000 7f5db5ffb640 1 -- 172.21.15.112:0/230924235 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11881 (secure 0 0 0) 0x7f5da00bdc70 con 0x7f5db8104c10 2024-03-16T09:13:43.389 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:43.389 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3939748854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3939748854},{"type":"v1","addr":"172.21.15.112:6839","nonce":3939748854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3118523258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3118523258},{"type":"v1","addr":"172.21.15.112:6841","nonce":3118523258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T09:13:36.450216+0000","modified":"2024-03-16T09:13:40.527486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/1682076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":1682076},{"type":"v1","addr":"172.21.15.194:6837","nonce":1682076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.194:6839/2528943309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2528943309},{"type":"v1","addr":"172.21.15.194:6839","nonce":2528943309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/272895965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":272895965},{"type":"v1","addr":"172.21.15.159:6839","nonce":272895965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24452,24466,24494]},"id":1}]} 2024-03-16T09:13:43.389 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2024-03-16T09:13:43.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.389+0000 7f5dbea63640 1 -- 172.21.15.112:0/230924235 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5da40a2800 msgr2=0x7f5da40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:43.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.389+0000 7f5dbea63640 1 --2- 172.21.15.112:0/230924235 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5da40a2800 0x7f5da40a4cc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5db81990f0 tx=0x7f5da8005f90 comp rx=0 tx=0).stop 2024-03-16T09:13:43.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.389+0000 7f5dbea63640 1 -- 172.21.15.112:0/230924235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5db8104c10 msgr2=0x7f5db8198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:43.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.389+0000 7f5dbea63640 1 --2- 172.21.15.112:0/230924235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5db8104c10 0x7f5db8198160 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5da0000c00 tx=0x7f5da0004a30 comp rx=0 tx=0).stop 2024-03-16T09:13:43.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.389+0000 7f5dbea63640 1 -- 172.21.15.112:0/230924235 shutdown_connections 2024-03-16T09:13:43.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.389+0000 7f5dbea63640 1 --2- 172.21.15.112:0/230924235 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5da40a2800 0x7f5da40a4cc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.389+0000 7f5dbea63640 1 --2- 172.21.15.112:0/230924235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5db8107540 0x7f5db819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.389+0000 7f5dbea63640 1 --2- 172.21.15.112:0/230924235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5db8104c10 0x7f5db8198160 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.389+0000 7f5dbea63640 1 --2- 172.21.15.112:0/230924235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5db81022e0 0x7f5db8197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.389+0000 7f5dbea63640 1 -- 172.21.15.112:0/230924235 >> 172.21.15.112:0/230924235 conn(0x7f5db80fbc70 msgr2=0x7f5db8102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:43.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.389+0000 7f5dbea63640 1 -- 172.21.15.112:0/230924235 shutdown_connections 2024-03-16T09:13:43.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.389+0000 7f5dbea63640 1 -- 172.21.15.112:0/230924235 wait complete. 2024-03-16T09:13:43.402 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:13:43.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.502+0000 7f7bbe863640 1 -- 172.21.15.112:0/62117534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb80741c0 msgr2=0x7f7bb8074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:43.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.502+0000 7f7bbe863640 1 --2- 172.21.15.112:0/62117534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb80741c0 0x7f7bb8074620 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009900 tx=0x7f7bac02bcd0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.502+0000 7f7bbe863640 1 -- 172.21.15.112:0/62117534 shutdown_connections 2024-03-16T09:13:43.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.502+0000 7f7bbe863640 1 --2- 172.21.15.112:0/62117534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bb81096b0 0x7f7bb810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.502+0000 7f7bbe863640 1 --2- 172.21.15.112:0/62117534 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb80741c0 0x7f7bb8074620 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.502+0000 7f7bbe863640 1 --2- 172.21.15.112:0/62117534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bb80757a0 0x7f7bb8073bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.502+0000 7f7bbe863640 1 -- 172.21.15.112:0/62117534 >> 172.21.15.112:0/62117534 conn(0x7f7bb80fcd90 msgr2=0x7f7bb80ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:43.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.502+0000 7f7bbe863640 1 -- 172.21.15.112:0/62117534 shutdown_connections 2024-03-16T09:13:43.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.502+0000 7f7bbe863640 1 -- 172.21.15.112:0/62117534 wait complete. 2024-03-16T09:13:43.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.503+0000 7f7bbe863640 1 Processor -- start 2024-03-16T09:13:43.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.503+0000 7f7bbe863640 1 -- start start 2024-03-16T09:13:43.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.503+0000 7f7bbe863640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bb80741c0 0x7f7bb8199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:43.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.503+0000 7f7bbe863640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb80757a0 0x7f7bb819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:43.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.503+0000 7f7bbe863640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bb81096b0 0x7f7bb81a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:43.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.503+0000 7f7bbe863640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb810f7c0 con 0x7f7bb80757a0 2024-03-16T09:13:43.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.503+0000 7f7bbe863640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb810f640 con 0x7f7bb81096b0 2024-03-16T09:13:43.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.503+0000 7f7bbe863640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb810f940 con 0x7f7bb80741c0 2024-03-16T09:13:43.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.503+0000 7f7bb77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb80757a0 0x7f7bb819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:43.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.503+0000 7f7bb77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb80757a0 0x7f7bb819a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59178/0 (socket says 172.21.15.112:59178) 2024-03-16T09:13:43.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.503+0000 7f7bb77fe640 1 -- 172.21.15.112:0/3251299522 learned_addr learned my addr 172.21.15.112:0/3251299522 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:43.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.503+0000 7f7bbce00640 1 --2- 172.21.15.112:0/3251299522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bb81096b0 0x7f7bb81a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:43.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.503+0000 7f7bb7fff640 1 --2- 172.21.15.112:0/3251299522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bb80741c0 0x7f7bb8199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:43.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.503+0000 7f7bb77fe640 1 -- 172.21.15.112:0/3251299522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bb80741c0 msgr2=0x7f7bb8199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:43.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.503+0000 7f7bb77fe640 1 --2- 172.21.15.112:0/3251299522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bb80741c0 0x7f7bb8199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.503+0000 7f7bb77fe640 1 -- 172.21.15.112:0/3251299522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bb81096b0 msgr2=0x7f7bb81a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:43.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.503+0000 7f7bb77fe640 1 --2- 172.21.15.112:0/3251299522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bb81096b0 0x7f7bb81a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.503+0000 7f7bb77fe640 1 -- 172.21.15.112:0/3251299522 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb81a19e0 con 0x7f7bb80757a0 2024-03-16T09:13:43.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.503+0000 7f7bbce00640 1 --2- 172.21.15.112:0/3251299522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bb81096b0 0x7f7bb81a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:13:43.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.504+0000 7f7bb77fe640 1 --2- 172.21.15.112:0/3251299522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb80757a0 0x7f7bb819a300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009230 tx=0x7f7bac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:43.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.504+0000 7f7bb57fa640 1 -- 172.21.15.112:0/3251299522 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac043020 con 0x7f7bb80757a0 2024-03-16T09:13:43.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.504+0000 7f7bb57fa640 1 -- 172.21.15.112:0/3251299522 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7bac004250 con 0x7f7bb80757a0 2024-03-16T09:13:43.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.504+0000 7f7bbe863640 1 -- 172.21.15.112:0/3251299522 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb81a1c70 con 0x7f7bb80757a0 2024-03-16T09:13:43.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.504+0000 7f7bb57fa640 1 -- 172.21.15.112:0/3251299522 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac003b90 con 0x7f7bb80757a0 2024-03-16T09:13:43.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.504+0000 7f7bbe863640 1 -- 172.21.15.112:0/3251299522 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb81a21c0 con 0x7f7bb80757a0 2024-03-16T09:13:43.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.505+0000 7f7bbe863640 1 -- 172.21.15.112:0/3251299522 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb8108470 con 0x7f7bb80757a0 2024-03-16T09:13:43.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.506+0000 7f7bb57fa640 1 -- 172.21.15.112:0/3251299522 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f7bac046020 con 0x7f7bb80757a0 2024-03-16T09:13:43.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.506+0000 7f7bb57fa640 1 --2- 172.21.15.112:0/3251299522 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7ba00a2880 0x7f7ba00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:43.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.506+0000 7f7bb7fff640 1 --2- 172.21.15.112:0/3251299522 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7ba00a2880 0x7f7ba00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:43.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.506+0000 7f7bb57fa640 1 -- 172.21.15.112:0/3251299522 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f7bac0f0930 con 0x7f7bb80757a0 2024-03-16T09:13:43.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.507+0000 7f7bb7fff640 1 --2- 172.21.15.112:0/3251299522 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7ba00a2880 0x7f7ba00a4d40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009bd0 tx=0x7f7bb0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:43.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.511+0000 7f7bb57fa640 1 -- 172.21.15.112:0/3251299522 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7bac04d020 con 0x7f7bb80757a0 2024-03-16T09:13:43.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:43 smithi112 ceph-mon[32307]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:13:43.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:43 smithi112 ceph-mon[32307]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:13:43.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:43 smithi112 ceph-mon[32307]: pgmap v291: 97 pgs: 97 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3.0 KiB/s wr, 15 op/s 2024-03-16T09:13:43.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:43 smithi112 ceph-mon[32307]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:13:43.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:43 smithi194 ceph-mon[36729]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:13:43.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:43 smithi194 ceph-mon[36729]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:13:43.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:43 smithi194 ceph-mon[36729]: pgmap v291: 97 pgs: 97 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3.0 KiB/s wr, 15 op/s 2024-03-16T09:13:43.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:43 smithi194 ceph-mon[36729]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:13:43.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.686+0000 7f7bbe863640 1 -- 172.21.15.112:0/3251299522 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7bb802d360 con 0x7f7bb80757a0 2024-03-16T09:13:43.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.687+0000 7f7bb57fa640 1 -- 172.21.15.112:0/3251299522 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22291 (secure 0 0 0) 0x7f7bac0bdb20 con 0x7f7bb80757a0 2024-03-16T09:13:43.689 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:43.690 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":85,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:13:38.104757+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T09:13:35.040628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T09:13:35.689419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:13:43.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.689+0000 7f7bbe863640 1 -- 172.21.15.112:0/3251299522 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7ba00a2880 msgr2=0x7f7ba00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:43.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.689+0000 7f7bbe863640 1 --2- 172.21.15.112:0/3251299522 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7ba00a2880 0x7f7ba00a4d40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009bd0 tx=0x7f7bb0005f90 comp rx=0 tx=0).stop 2024-03-16T09:13:43.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.689+0000 7f7bbe863640 1 -- 172.21.15.112:0/3251299522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb80757a0 msgr2=0x7f7bb819a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:43.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.689+0000 7f7bbe863640 1 --2- 172.21.15.112:0/3251299522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb80757a0 0x7f7bb819a300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009230 tx=0x7f7bac004ab0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.689+0000 7f7bbe863640 1 -- 172.21.15.112:0/3251299522 shutdown_connections 2024-03-16T09:13:43.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.689+0000 7f7bbe863640 1 --2- 172.21.15.112:0/3251299522 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f7ba00a2880 0x7f7ba00a4d40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.689+0000 7f7bbe863640 1 --2- 172.21.15.112:0/3251299522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bb81096b0 0x7f7bb81a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.689+0000 7f7bbe863640 1 --2- 172.21.15.112:0/3251299522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7bb80757a0 0x7f7bb819a300 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.689+0000 7f7bbe863640 1 --2- 172.21.15.112:0/3251299522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bb80741c0 0x7f7bb8199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.689+0000 7f7bbe863640 1 -- 172.21.15.112:0/3251299522 >> 172.21.15.112:0/3251299522 conn(0x7f7bb80fcd90 msgr2=0x7f7bb8105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:43.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.689+0000 7f7bbe863640 1 -- 172.21.15.112:0/3251299522 shutdown_connections 2024-03-16T09:13:43.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.689+0000 7f7bbe863640 1 -- 172.21.15.112:0/3251299522 wait complete. 2024-03-16T09:13:43.703 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-16T09:13:43.703 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T09:13:43.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:43 smithi159 ceph-mon[36739]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:13:43.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:43 smithi159 ceph-mon[36739]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:13:43.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:43 smithi159 ceph-mon[36739]: pgmap v291: 97 pgs: 97 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3.0 KiB/s wr, 15 op/s 2024-03-16T09:13:43.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:43 smithi159 ceph-mon[36739]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:13:43.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.803+0000 7f3db8ebf640 1 -- 172.21.15.112:0/515387922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db4101a00 msgr2=0x7f3db4108320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:43.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.803+0000 7f3db8ebf640 1 --2- 172.21.15.112:0/515387922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db4101a00 0x7f3db4108320 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3da400aa10 tx=0x7f3da402cf70 comp rx=0 tx=0).stop 2024-03-16T09:13:43.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.803+0000 7f3db8ebf640 1 -- 172.21.15.112:0/515387922 shutdown_connections 2024-03-16T09:13:43.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.803+0000 7f3db8ebf640 1 --2- 172.21.15.112:0/515387922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db4101a00 0x7f3db4108320 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.803+0000 7f3db8ebf640 1 --2- 172.21.15.112:0/515387922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3db4101040 0x7f3db41014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.803+0000 7f3db8ebf640 1 --2- 172.21.15.112:0/515387922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3db40ffe60 0x7f3db4100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.803+0000 7f3db8ebf640 1 -- 172.21.15.112:0/515387922 >> 172.21.15.112:0/515387922 conn(0x7f3db40fb420 msgr2=0x7f3db40fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:43.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.803+0000 7f3db8ebf640 1 -- 172.21.15.112:0/515387922 shutdown_connections 2024-03-16T09:13:43.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.803+0000 7f3db8ebf640 1 -- 172.21.15.112:0/515387922 wait complete. 2024-03-16T09:13:43.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.804+0000 7f3db8ebf640 1 Processor -- start 2024-03-16T09:13:43.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.804+0000 7f3db8ebf640 1 -- start start 2024-03-16T09:13:43.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.804+0000 7f3db8ebf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db40ffe60 0x7f3db4197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:43.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.804+0000 7f3db8ebf640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3db4101040 0x7f3db4198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:43.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.804+0000 7f3db8ebf640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3db4101a00 0x7f3db419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:43.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.804+0000 7f3db8ebf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3db410b3f0 con 0x7f3db40ffe60 2024-03-16T09:13:43.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.804+0000 7f3db8ebf640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3db410b270 con 0x7f3db4101a00 2024-03-16T09:13:43.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.804+0000 7f3db8ebf640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3db410b570 con 0x7f3db4101040 2024-03-16T09:13:43.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.804+0000 7f3db259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db40ffe60 0x7f3db4197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:43.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.804+0000 7f3db259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db40ffe60 0x7f3db4197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59198/0 (socket says 172.21.15.112:59198) 2024-03-16T09:13:43.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.804+0000 7f3db259c640 1 -- 172.21.15.112:0/220403410 learned_addr learned my addr 172.21.15.112:0/220403410 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:43.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.804+0000 7f3db2d9d640 1 --2- 172.21.15.112:0/220403410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3db4101a00 0x7f3db419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:43.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.804+0000 7f3db1d9b640 1 --2- 172.21.15.112:0/220403410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3db4101040 0x7f3db4198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:43.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.805+0000 7f3db259c640 1 -- 172.21.15.112:0/220403410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3db4101040 msgr2=0x7f3db4198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:43.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.805+0000 7f3db259c640 1 --2- 172.21.15.112:0/220403410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3db4101040 0x7f3db4198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.805+0000 7f3db259c640 1 -- 172.21.15.112:0/220403410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3db4101a00 msgr2=0x7f3db419f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:43.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.805+0000 7f3db259c640 1 --2- 172.21.15.112:0/220403410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3db4101a00 0x7f3db419f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:43.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.805+0000 7f3db259c640 1 -- 172.21.15.112:0/220403410 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db419f7e0 con 0x7f3db40ffe60 2024-03-16T09:13:43.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.805+0000 7f3db1d9b640 1 --2- 172.21.15.112:0/220403410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3db4101040 0x7f3db4198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:13:43.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.805+0000 7f3db259c640 1 --2- 172.21.15.112:0/220403410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db40ffe60 0x7f3db4197bc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009d20 tx=0x7f3dac00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:43.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.805+0000 7f3da37fe640 1 -- 172.21.15.112:0/220403410 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac010630 con 0x7f3db40ffe60 2024-03-16T09:13:43.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.805+0000 7f3da37fe640 1 -- 172.21.15.112:0/220403410 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3dac010cd0 con 0x7f3db40ffe60 2024-03-16T09:13:43.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.805+0000 7f3db8ebf640 1 -- 172.21.15.112:0/220403410 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db419fad0 con 0x7f3db40ffe60 2024-03-16T09:13:43.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.805+0000 7f3da37fe640 1 -- 172.21.15.112:0/220403410 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac0173d0 con 0x7f3db40ffe60 2024-03-16T09:13:43.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.805+0000 7f3db8ebf640 1 -- 172.21.15.112:0/220403410 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db41a0070 con 0x7f3db40ffe60 2024-03-16T09:13:43.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.806+0000 7f3db8ebf640 1 -- 172.21.15.112:0/220403410 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3db41a0240 con 0x7f3db40ffe60 2024-03-16T09:13:43.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.807+0000 7f3da37fe640 1 -- 172.21.15.112:0/220403410 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f3dac01f020 con 0x7f3db40ffe60 2024-03-16T09:13:43.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.808+0000 7f3da37fe640 1 --2- 172.21.15.112:0/220403410 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3d9c0a2880 0x7f3d9c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:43.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.808+0000 7f3db1d9b640 1 --2- 172.21.15.112:0/220403410 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3d9c0a2880 0x7f3d9c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:43.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.808+0000 7f3da37fe640 1 -- 172.21.15.112:0/220403410 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f3dac0cf3f0 con 0x7f3db40ffe60 2024-03-16T09:13:43.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.808+0000 7f3db1d9b640 1 --2- 172.21.15.112:0/220403410 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3d9c0a2880 0x7f3d9c0a4d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3db41990d0 tx=0x7f3da8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:43.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.811+0000 7f3da37fe640 1 -- 172.21.15.112:0/220403410 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3dac09c5e0 con 0x7f3db40ffe60 2024-03-16T09:13:43.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.985+0000 7f3db8ebf640 1 -- 172.21.15.112:0/220403410 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3db4107c30 con 0x7f3d9c0a2880 2024-03-16T09:13:43.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.992+0000 7f3da37fe640 1 -- 172.21.15.112:0/220403410 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258886 (secure 0 0 0) 0x7f3db4107c30 con 0x7f3d9c0a2880 2024-03-16T09:13:43.997 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:43.999 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-03-16T09:13:44.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.995+0000 7f3db8ebf640 1 -- 172.21.15.112:0/220403410 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3d9c0a2880 msgr2=0x7f3d9c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:44.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.995+0000 7f3db8ebf640 1 --2- 172.21.15.112:0/220403410 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3d9c0a2880 0x7f3d9c0a4d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3db41990d0 tx=0x7f3da8005f90 comp rx=0 tx=0).stop 2024-03-16T09:13:44.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.995+0000 7f3db8ebf640 1 -- 172.21.15.112:0/220403410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db40ffe60 msgr2=0x7f3db4197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:44.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.995+0000 7f3db8ebf640 1 --2- 172.21.15.112:0/220403410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db40ffe60 0x7f3db4197bc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009d20 tx=0x7f3dac00c6a0 comp rx=0 tx=0).stop 2024-03-16T09:13:44.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.996+0000 7f3db8ebf640 1 -- 172.21.15.112:0/220403410 shutdown_connections 2024-03-16T09:13:44.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.996+0000 7f3db8ebf640 1 --2- 172.21.15.112:0/220403410 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3d9c0a2880 0x7f3d9c0a4d40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:44.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.996+0000 7f3db8ebf640 1 --2- 172.21.15.112:0/220403410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3db4101a00 0x7f3db419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:44.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.996+0000 7f3db8ebf640 1 --2- 172.21.15.112:0/220403410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3db4101040 0x7f3db4198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:44.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.996+0000 7f3db8ebf640 1 --2- 172.21.15.112:0/220403410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3db40ffe60 0x7f3db4197bc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:44.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.996+0000 7f3db8ebf640 1 -- 172.21.15.112:0/220403410 >> 172.21.15.112:0/220403410 conn(0x7f3db40fb420 msgr2=0x7f3db4106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:44.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.996+0000 7f3db8ebf640 1 -- 172.21.15.112:0/220403410 shutdown_connections 2024-03-16T09:13:44.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:43.996+0000 7f3db8ebf640 1 -- 172.21.15.112:0/220403410 wait complete. 2024-03-16T09:13:44.009 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-03-16T09:13:42.875107+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594744,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":140,"num_read_kb":120,"num_write":269,"num_write_kb":2958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":235,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351832,"kb_used_data":10944,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589992,"statfs":{"total":1151940427776,"available":1151580151808,"internally_reserved":0,"allocated":11206656,"data_stored":6992954,"data_compressed":107142,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4376,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":88,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.778266"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.125141+0000","last_change":"2024-03-16T09:13:37.119969+0000","last_active":"2024-03-16T09:13:37.125141+0000","last_peered":"2024-03-16T09:13:37.125141+0000","last_clean":"2024-03-16T09:13:37.125141+0000","last_became_active":"2024-03-16T09:13:37.119177+0000","last_became_peered":"2024-03-16T09:13:37.119177+0000","last_unstale":"2024-03-16T09:13:37.125141+0000","last_undegraded":"2024-03-16T09:13:37.125141+0000","last_fullsized":"2024-03-16T09:13:37.125141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:58:30.822116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130335+0000","last_change":"2024-03-16T09:13:37.127564+0000","last_active":"2024-03-16T09:13:39.130335+0000","last_peered":"2024-03-16T09:13:39.130335+0000","last_clean":"2024-03-16T09:13:39.130335+0000","last_became_active":"2024-03-16T09:13:37.127214+0000","last_became_peered":"2024-03-16T09:13:37.127214+0000","last_unstale":"2024-03-16T09:13:39.130335+0000","last_undegraded":"2024-03-16T09:13:39.130335+0000","last_fullsized":"2024-03-16T09:13:39.130335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:46:14.655707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130296+0000","last_change":"2024-03-16T09:13:37.126288+0000","last_active":"2024-03-16T09:13:39.130296+0000","last_peered":"2024-03-16T09:13:39.130296+0000","last_clean":"2024-03-16T09:13:39.130296+0000","last_became_active":"2024-03-16T09:13:37.125383+0000","last_became_peered":"2024-03-16T09:13:37.125383+0000","last_unstale":"2024-03-16T09:13:39.130296+0000","last_undegraded":"2024-03-16T09:13:39.130296+0000","last_fullsized":"2024-03-16T09:13:39.130296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:47:09.901508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.126813+0000","last_change":"2024-03-16T09:13:37.125503+0000","last_active":"2024-03-16T09:13:37.126813+0000","last_peered":"2024-03-16T09:13:37.126813+0000","last_clean":"2024-03-16T09:13:37.126813+0000","last_became_active":"2024-03-16T09:13:37.125144+0000","last_became_peered":"2024-03-16T09:13:37.125144+0000","last_unstale":"2024-03-16T09:13:37.126813+0000","last_undegraded":"2024-03-16T09:13:37.126813+0000","last_fullsized":"2024-03-16T09:13:37.126813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:23:39.839396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121380+0000","last_change":"2024-03-16T09:13:37.114237+0000","last_active":"2024-03-16T09:13:38.121380+0000","last_peered":"2024-03-16T09:13:38.121380+0000","last_clean":"2024-03-16T09:13:38.121380+0000","last_became_active":"2024-03-16T09:13:37.113595+0000","last_became_peered":"2024-03-16T09:13:37.113595+0000","last_unstale":"2024-03-16T09:13:38.121380+0000","last_undegraded":"2024-03-16T09:13:38.121380+0000","last_fullsized":"2024-03-16T09:13:38.121380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:06:17.735301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.246871+0000","last_change":"2024-03-16T09:13:37.118843+0000","last_active":"2024-03-16T09:13:38.246871+0000","last_peered":"2024-03-16T09:13:38.246871+0000","last_clean":"2024-03-16T09:13:38.246871+0000","last_became_active":"2024-03-16T09:13:37.117086+0000","last_became_peered":"2024-03-16T09:13:37.117086+0000","last_unstale":"2024-03-16T09:13:38.246871+0000","last_undegraded":"2024-03-16T09:13:38.246871+0000","last_fullsized":"2024-03-16T09:13:38.246871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:23:12.929931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.129493+0000","last_change":"2024-03-16T09:13:37.133427+0000","last_active":"2024-03-16T09:13:39.129493+0000","last_peered":"2024-03-16T09:13:39.129493+0000","last_clean":"2024-03-16T09:13:39.129493+0000","last_became_active":"2024-03-16T09:13:37.133032+0000","last_became_peered":"2024-03-16T09:13:37.133032+0000","last_unstale":"2024-03-16T09:13:39.129493+0000","last_undegraded":"2024-03-16T09:13:39.129493+0000","last_fullsized":"2024-03-16T09:13:39.129493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:49:28.179883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.120741+0000","last_change":"2024-03-16T09:13:37.132132+0000","last_active":"2024-03-16T09:13:38.120741+0000","last_peered":"2024-03-16T09:13:38.120741+0000","last_clean":"2024-03-16T09:13:38.120741+0000","last_became_active":"2024-03-16T09:13:37.131801+0000","last_became_peered":"2024-03-16T09:13:37.131801+0000","last_unstale":"2024-03-16T09:13:38.120741+0000","last_undegraded":"2024-03-16T09:13:38.120741+0000","last_fullsized":"2024-03-16T09:13:38.120741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:45:03.891725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.109314+0000","last_change":"2024-03-16T09:13:37.116378+0000","last_active":"2024-03-16T09:13:38.109314+0000","last_peered":"2024-03-16T09:13:38.109314+0000","last_clean":"2024-03-16T09:13:38.109314+0000","last_became_active":"2024-03-16T09:13:37.116027+0000","last_became_peered":"2024-03-16T09:13:37.116027+0000","last_unstale":"2024-03-16T09:13:38.109314+0000","last_undegraded":"2024-03-16T09:13:38.109314+0000","last_fullsized":"2024-03-16T09:13:38.109314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:14:38.681613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.247275+0000","last_change":"2024-03-16T09:13:37.121417+0000","last_active":"2024-03-16T09:13:38.247275+0000","last_peered":"2024-03-16T09:13:38.247275+0000","last_clean":"2024-03-16T09:13:38.247275+0000","last_became_active":"2024-03-16T09:13:37.121112+0000","last_became_peered":"2024-03-16T09:13:37.121112+0000","last_unstale":"2024-03-16T09:13:38.247275+0000","last_undegraded":"2024-03-16T09:13:38.247275+0000","last_fullsized":"2024-03-16T09:13:38.247275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:38:07.823604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130669+0000","last_change":"2024-03-16T09:13:37.121368+0000","last_active":"2024-03-16T09:13:39.130669+0000","last_peered":"2024-03-16T09:13:39.130669+0000","last_clean":"2024-03-16T09:13:39.130669+0000","last_became_active":"2024-03-16T09:13:37.121060+0000","last_became_peered":"2024-03-16T09:13:37.121060+0000","last_unstale":"2024-03-16T09:13:39.130669+0000","last_undegraded":"2024-03-16T09:13:39.130669+0000","last_fullsized":"2024-03-16T09:13:39.130669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:48:49.168285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.128968+0000","last_change":"2024-03-16T09:13:37.127004+0000","last_active":"2024-03-16T09:13:37.128968+0000","last_peered":"2024-03-16T09:13:37.128968+0000","last_clean":"2024-03-16T09:13:37.128968+0000","last_became_active":"2024-03-16T09:13:37.126579+0000","last_became_peered":"2024-03-16T09:13:37.126579+0000","last_unstale":"2024-03-16T09:13:37.128968+0000","last_undegraded":"2024-03-16T09:13:37.128968+0000","last_fullsized":"2024-03-16T09:13:37.128968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:44:40.229978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.105639+0000","last_change":"2024-03-16T09:13:37.104534+0000","last_active":"2024-03-16T09:13:37.105639+0000","last_peered":"2024-03-16T09:13:37.105639+0000","last_clean":"2024-03-16T09:13:37.105639+0000","last_became_active":"2024-03-16T09:13:37.104202+0000","last_became_peered":"2024-03-16T09:13:37.104202+0000","last_unstale":"2024-03-16T09:13:37.105639+0000","last_undegraded":"2024-03-16T09:13:37.105639+0000","last_fullsized":"2024-03-16T09:13:37.105639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:30:31.196479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130427+0000","last_change":"2024-03-16T09:13:37.117166+0000","last_active":"2024-03-16T09:13:39.130427+0000","last_peered":"2024-03-16T09:13:39.130427+0000","last_clean":"2024-03-16T09:13:39.130427+0000","last_became_active":"2024-03-16T09:13:37.116747+0000","last_became_peered":"2024-03-16T09:13:37.116747+0000","last_unstale":"2024-03-16T09:13:39.130427+0000","last_undegraded":"2024-03-16T09:13:39.130427+0000","last_fullsized":"2024-03-16T09:13:39.130427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:50:58.705683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130296+0000","last_change":"2024-03-16T09:13:37.124898+0000","last_active":"2024-03-16T09:13:39.130296+0000","last_peered":"2024-03-16T09:13:39.130296+0000","last_clean":"2024-03-16T09:13:39.130296+0000","last_became_active":"2024-03-16T09:13:37.124623+0000","last_became_peered":"2024-03-16T09:13:37.124623+0000","last_unstale":"2024-03-16T09:13:39.130296+0000","last_undegraded":"2024-03-16T09:13:39.130296+0000","last_fullsized":"2024-03-16T09:13:39.130296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:52:35.159740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.108690+0000","last_change":"2024-03-16T09:13:37.120583+0000","last_active":"2024-03-16T09:13:38.108690+0000","last_peered":"2024-03-16T09:13:38.108690+0000","last_clean":"2024-03-16T09:13:38.108690+0000","last_became_active":"2024-03-16T09:13:37.120217+0000","last_became_peered":"2024-03-16T09:13:37.120217+0000","last_unstale":"2024-03-16T09:13:38.108690+0000","last_undegraded":"2024-03-16T09:13:38.108690+0000","last_fullsized":"2024-03-16T09:13:38.108690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:00:17.653736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.125163+0000","last_change":"2024-03-16T09:13:37.121179+0000","last_active":"2024-03-16T09:13:37.125163+0000","last_peered":"2024-03-16T09:13:37.125163+0000","last_clean":"2024-03-16T09:13:37.125163+0000","last_became_active":"2024-03-16T09:13:37.120070+0000","last_became_peered":"2024-03-16T09:13:37.120070+0000","last_unstale":"2024-03-16T09:13:37.125163+0000","last_undegraded":"2024-03-16T09:13:37.125163+0000","last_fullsized":"2024-03-16T09:13:37.125163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:59:30.359969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.119335+0000","last_change":"2024-03-16T09:13:37.110074+0000","last_active":"2024-03-16T09:13:38.119335+0000","last_peered":"2024-03-16T09:13:38.119335+0000","last_clean":"2024-03-16T09:13:38.119335+0000","last_became_active":"2024-03-16T09:13:37.109767+0000","last_became_peered":"2024-03-16T09:13:37.109767+0000","last_unstale":"2024-03-16T09:13:38.119335+0000","last_undegraded":"2024-03-16T09:13:38.119335+0000","last_fullsized":"2024-03-16T09:13:38.119335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:20:37.408262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.109132+0000","last_change":"2024-03-16T09:13:37.130791+0000","last_active":"2024-03-16T09:13:38.109132+0000","last_peered":"2024-03-16T09:13:38.109132+0000","last_clean":"2024-03-16T09:13:38.109132+0000","last_became_active":"2024-03-16T09:13:37.130428+0000","last_became_peered":"2024-03-16T09:13:37.130428+0000","last_unstale":"2024-03-16T09:13:38.109132+0000","last_undegraded":"2024-03-16T09:13:38.109132+0000","last_fullsized":"2024-03-16T09:13:38.109132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:32:33.185589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:40.139597+0000","last_change":"2024-03-16T09:13:36.119881+0000","last_active":"2024-03-16T09:13:40.139597+0000","last_peered":"2024-03-16T09:13:40.139597+0000","last_clean":"2024-03-16T09:13:40.139597+0000","last_became_active":"2024-03-16T09:13:36.119513+0000","last_became_peered":"2024-03-16T09:13:36.119513+0000","last_unstale":"2024-03-16T09:13:40.139597+0000","last_undegraded":"2024-03-16T09:13:40.139597+0000","last_fullsized":"2024-03-16T09:13:40.139597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:47:35.202982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121124+0000","last_change":"2024-03-16T09:13:37.124946+0000","last_active":"2024-03-16T09:13:38.121124+0000","last_peered":"2024-03-16T09:13:38.121124+0000","last_clean":"2024-03-16T09:13:38.121124+0000","last_became_active":"2024-03-16T09:13:37.124624+0000","last_became_peered":"2024-03-16T09:13:37.124624+0000","last_unstale":"2024-03-16T09:13:38.121124+0000","last_undegraded":"2024-03-16T09:13:38.121124+0000","last_fullsized":"2024-03-16T09:13:38.121124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:18:15.188823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121291+0000","last_change":"2024-03-16T09:13:36.116312+0000","last_active":"2024-03-16T09:13:38.121291+0000","last_peered":"2024-03-16T09:13:38.121291+0000","last_clean":"2024-03-16T09:13:38.121291+0000","last_became_active":"2024-03-16T09:13:36.115999+0000","last_became_peered":"2024-03-16T09:13:36.115999+0000","last_unstale":"2024-03-16T09:13:38.121291+0000","last_undegraded":"2024-03-16T09:13:38.121291+0000","last_fullsized":"2024-03-16T09:13:38.121291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:04:46.795291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130563+0000","last_change":"2024-03-16T09:13:37.126024+0000","last_active":"2024-03-16T09:13:39.130563+0000","last_peered":"2024-03-16T09:13:39.130563+0000","last_clean":"2024-03-16T09:13:39.130563+0000","last_became_active":"2024-03-16T09:13:37.125722+0000","last_became_peered":"2024-03-16T09:13:37.125722+0000","last_unstale":"2024-03-16T09:13:39.130563+0000","last_undegraded":"2024-03-16T09:13:39.130563+0000","last_fullsized":"2024-03-16T09:13:39.130563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:27:24.923022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121707+0000","last_change":"2024-03-16T09:13:36.119700+0000","last_active":"2024-03-16T09:13:38.121707+0000","last_peered":"2024-03-16T09:13:38.121707+0000","last_clean":"2024-03-16T09:13:38.121707+0000","last_became_active":"2024-03-16T09:13:36.119361+0000","last_became_peered":"2024-03-16T09:13:36.119361+0000","last_unstale":"2024-03-16T09:13:38.121707+0000","last_undegraded":"2024-03-16T09:13:38.121707+0000","last_fullsized":"2024-03-16T09:13:38.121707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:26:01.238914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.108897+0000","last_change":"2024-03-16T09:13:37.127593+0000","last_active":"2024-03-16T09:13:38.108897+0000","last_peered":"2024-03-16T09:13:38.108897+0000","last_clean":"2024-03-16T09:13:38.108897+0000","last_became_active":"2024-03-16T09:13:37.127255+0000","last_became_peered":"2024-03-16T09:13:37.127255+0000","last_unstale":"2024-03-16T09:13:38.108897+0000","last_undegraded":"2024-03-16T09:13:38.108897+0000","last_fullsized":"2024-03-16T09:13:38.108897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:07:51.263266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121586+0000","last_change":"2024-03-16T09:13:36.117583+0000","last_active":"2024-03-16T09:13:38.121586+0000","last_peered":"2024-03-16T09:13:38.121586+0000","last_clean":"2024-03-16T09:13:38.121586+0000","last_became_active":"2024-03-16T09:13:36.117270+0000","last_became_peered":"2024-03-16T09:13:36.117270+0000","last_unstale":"2024-03-16T09:13:38.121586+0000","last_undegraded":"2024-03-16T09:13:38.121586+0000","last_fullsized":"2024-03-16T09:13:38.121586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:10:53.541872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130671+0000","last_change":"2024-03-16T09:13:37.119019+0000","last_active":"2024-03-16T09:13:39.130671+0000","last_peered":"2024-03-16T09:13:39.130671+0000","last_clean":"2024-03-16T09:13:39.130671+0000","last_became_active":"2024-03-16T09:13:37.118148+0000","last_became_peered":"2024-03-16T09:13:37.118148+0000","last_unstale":"2024-03-16T09:13:39.130671+0000","last_undegraded":"2024-03-16T09:13:39.130671+0000","last_fullsized":"2024-03-16T09:13:39.130671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:17:43.042779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:40.326228+0000","last_change":"2024-03-16T09:13:36.114415+0000","last_active":"2024-03-16T09:13:40.326228+0000","last_peered":"2024-03-16T09:13:40.326228+0000","last_clean":"2024-03-16T09:13:40.326228+0000","last_became_active":"2024-03-16T09:13:36.114056+0000","last_became_peered":"2024-03-16T09:13:36.114056+0000","last_unstale":"2024-03-16T09:13:40.326228+0000","last_undegraded":"2024-03-16T09:13:40.326228+0000","last_fullsized":"2024-03-16T09:13:40.326228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:50:28.322627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121411+0000","last_change":"2024-03-16T09:13:37.124997+0000","last_active":"2024-03-16T09:13:38.121411+0000","last_peered":"2024-03-16T09:13:38.121411+0000","last_clean":"2024-03-16T09:13:38.121411+0000","last_became_active":"2024-03-16T09:13:37.124664+0000","last_became_peered":"2024-03-16T09:13:37.124664+0000","last_unstale":"2024-03-16T09:13:38.121411+0000","last_undegraded":"2024-03-16T09:13:38.121411+0000","last_fullsized":"2024-03-16T09:13:38.121411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:24:30.104576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:40.139397+0000","last_change":"2024-03-16T09:13:36.121980+0000","last_active":"2024-03-16T09:13:40.139397+0000","last_peered":"2024-03-16T09:13:40.139397+0000","last_clean":"2024-03-16T09:13:40.139397+0000","last_became_active":"2024-03-16T09:13:36.121703+0000","last_became_peered":"2024-03-16T09:13:36.121703+0000","last_unstale":"2024-03-16T09:13:40.139397+0000","last_undegraded":"2024-03-16T09:13:40.139397+0000","last_fullsized":"2024-03-16T09:13:40.139397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:40:26.601688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.132653+0000","last_change":"2024-03-16T09:13:37.123443+0000","last_active":"2024-03-16T09:13:39.132653+0000","last_peered":"2024-03-16T09:13:39.132653+0000","last_clean":"2024-03-16T09:13:39.132653+0000","last_became_active":"2024-03-16T09:13:37.121987+0000","last_became_peered":"2024-03-16T09:13:37.121987+0000","last_unstale":"2024-03-16T09:13:39.132653+0000","last_undegraded":"2024-03-16T09:13:39.132653+0000","last_fullsized":"2024-03-16T09:13:39.132653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:37:08.478552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.099642+0000","last_change":"2024-03-16T09:13:36.106391+0000","last_active":"2024-03-16T09:13:37.099642+0000","last_peered":"2024-03-16T09:13:37.099642+0000","last_clean":"2024-03-16T09:13:37.099642+0000","last_became_active":"2024-03-16T09:13:36.106046+0000","last_became_peered":"2024-03-16T09:13:36.106046+0000","last_unstale":"2024-03-16T09:13:37.099642+0000","last_undegraded":"2024-03-16T09:13:37.099642+0000","last_fullsized":"2024-03-16T09:13:37.099642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:03:23.255519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.120697+0000","last_change":"2024-03-16T09:13:37.114337+0000","last_active":"2024-03-16T09:13:38.120697+0000","last_peered":"2024-03-16T09:13:38.120697+0000","last_clean":"2024-03-16T09:13:38.120697+0000","last_became_active":"2024-03-16T09:13:37.113991+0000","last_became_peered":"2024-03-16T09:13:37.113991+0000","last_unstale":"2024-03-16T09:13:38.120697+0000","last_undegraded":"2024-03-16T09:13:38.120697+0000","last_fullsized":"2024-03-16T09:13:38.120697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:23:54.141479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130119+0000","last_change":"2024-03-16T09:13:36.113297+0000","last_active":"2024-03-16T09:13:39.130119+0000","last_peered":"2024-03-16T09:13:39.130119+0000","last_clean":"2024-03-16T09:13:39.130119+0000","last_became_active":"2024-03-16T09:13:36.112950+0000","last_became_peered":"2024-03-16T09:13:36.112950+0000","last_unstale":"2024-03-16T09:13:39.130119+0000","last_undegraded":"2024-03-16T09:13:39.130119+0000","last_fullsized":"2024-03-16T09:13:39.130119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:40:17.682450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.119150+0000","last_change":"2024-03-16T09:13:37.112239+0000","last_active":"2024-03-16T09:13:38.119150+0000","last_peered":"2024-03-16T09:13:38.119150+0000","last_clean":"2024-03-16T09:13:38.119150+0000","last_became_active":"2024-03-16T09:13:37.111885+0000","last_became_peered":"2024-03-16T09:13:37.111885+0000","last_unstale":"2024-03-16T09:13:38.119150+0000","last_undegraded":"2024-03-16T09:13:38.119150+0000","last_fullsized":"2024-03-16T09:13:38.119150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:49.565846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.134946+0000","last_change":"2024-03-16T09:13:36.111341+0000","last_active":"2024-03-16T09:13:39.134946+0000","last_peered":"2024-03-16T09:13:39.134946+0000","last_clean":"2024-03-16T09:13:39.134946+0000","last_became_active":"2024-03-16T09:13:36.110558+0000","last_became_peered":"2024-03-16T09:13:36.110558+0000","last_unstale":"2024-03-16T09:13:39.134946+0000","last_undegraded":"2024-03-16T09:13:39.134946+0000","last_fullsized":"2024-03-16T09:13:39.134946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:56:44.252002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.109148+0000","last_change":"2024-03-16T09:13:37.126808+0000","last_active":"2024-03-16T09:13:38.109148+0000","last_peered":"2024-03-16T09:13:38.109148+0000","last_clean":"2024-03-16T09:13:38.109148+0000","last_became_active":"2024-03-16T09:13:37.126442+0000","last_became_peered":"2024-03-16T09:13:37.126442+0000","last_unstale":"2024-03-16T09:13:38.109148+0000","last_undegraded":"2024-03-16T09:13:38.109148+0000","last_fullsized":"2024-03-16T09:13:38.109148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:04:39.216835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.134201+0000","last_change":"2024-03-16T09:13:36.113907+0000","last_active":"2024-03-16T09:13:39.134201+0000","last_peered":"2024-03-16T09:13:39.134201+0000","last_clean":"2024-03-16T09:13:39.134201+0000","last_became_active":"2024-03-16T09:13:36.113564+0000","last_became_peered":"2024-03-16T09:13:36.113564+0000","last_unstale":"2024-03-16T09:13:39.134201+0000","last_undegraded":"2024-03-16T09:13:39.134201+0000","last_fullsized":"2024-03-16T09:13:39.134201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:24:33.272919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130356+0000","last_change":"2024-03-16T09:13:37.125833+0000","last_active":"2024-03-16T09:13:39.130356+0000","last_peered":"2024-03-16T09:13:39.130356+0000","last_clean":"2024-03-16T09:13:39.130356+0000","last_became_active":"2024-03-16T09:13:37.125508+0000","last_became_peered":"2024-03-16T09:13:37.125508+0000","last_unstale":"2024-03-16T09:13:39.130356+0000","last_undegraded":"2024-03-16T09:13:39.130356+0000","last_fullsized":"2024-03-16T09:13:39.130356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:14:47.749715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.134366+0000","last_change":"2024-03-16T09:13:36.112200+0000","last_active":"2024-03-16T09:13:39.134366+0000","last_peered":"2024-03-16T09:13:39.134366+0000","last_clean":"2024-03-16T09:13:39.134366+0000","last_became_active":"2024-03-16T09:13:36.111830+0000","last_became_peered":"2024-03-16T09:13:36.111830+0000","last_unstale":"2024-03-16T09:13:39.134366+0000","last_undegraded":"2024-03-16T09:13:39.134366+0000","last_fullsized":"2024-03-16T09:13:39.134366+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:07:57.147960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.109167+0000","last_change":"2024-03-16T09:13:37.118434+0000","last_active":"2024-03-16T09:13:38.109167+0000","last_peered":"2024-03-16T09:13:38.109167+0000","last_clean":"2024-03-16T09:13:38.109167+0000","last_became_active":"2024-03-16T09:13:37.118069+0000","last_became_peered":"2024-03-16T09:13:37.118069+0000","last_unstale":"2024-03-16T09:13:38.109167+0000","last_undegraded":"2024-03-16T09:13:38.109167+0000","last_fullsized":"2024-03-16T09:13:38.109167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:02:56.559384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:40.325141+0000","last_change":"2024-03-16T09:13:36.114591+0000","last_active":"2024-03-16T09:13:40.325141+0000","last_peered":"2024-03-16T09:13:40.325141+0000","last_clean":"2024-03-16T09:13:40.325141+0000","last_became_active":"2024-03-16T09:13:36.114252+0000","last_became_peered":"2024-03-16T09:13:36.114252+0000","last_unstale":"2024-03-16T09:13:40.325141+0000","last_undegraded":"2024-03-16T09:13:40.325141+0000","last_fullsized":"2024-03-16T09:13:40.325141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:37:52.637394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121706+0000","last_change":"2024-03-16T09:13:37.133102+0000","last_active":"2024-03-16T09:13:38.121706+0000","last_peered":"2024-03-16T09:13:38.121706+0000","last_clean":"2024-03-16T09:13:38.121706+0000","last_became_active":"2024-03-16T09:13:37.132728+0000","last_became_peered":"2024-03-16T09:13:37.132728+0000","last_unstale":"2024-03-16T09:13:38.121706+0000","last_undegraded":"2024-03-16T09:13:38.121706+0000","last_fullsized":"2024-03-16T09:13:38.121706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:54:27.511997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:40.138064+0000","last_change":"2024-03-16T09:13:36.114195+0000","last_active":"2024-03-16T09:13:40.138064+0000","last_peered":"2024-03-16T09:13:40.138064+0000","last_clean":"2024-03-16T09:13:40.138064+0000","last_became_active":"2024-03-16T09:13:36.113829+0000","last_became_peered":"2024-03-16T09:13:36.113829+0000","last_unstale":"2024-03-16T09:13:40.138064+0000","last_undegraded":"2024-03-16T09:13:40.138064+0000","last_fullsized":"2024-03-16T09:13:40.138064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:28:40.509877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.133476+0000","last_change":"2024-03-16T09:13:37.132974+0000","last_active":"2024-03-16T09:13:37.133476+0000","last_peered":"2024-03-16T09:13:37.133476+0000","last_clean":"2024-03-16T09:13:37.133476+0000","last_became_active":"2024-03-16T09:13:37.132474+0000","last_became_peered":"2024-03-16T09:13:37.132474+0000","last_unstale":"2024-03-16T09:13:37.133476+0000","last_undegraded":"2024-03-16T09:13:37.133476+0000","last_fullsized":"2024-03-16T09:13:37.133476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:56:45.637577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.135533+0000","last_change":"2024-03-16T09:13:36.115242+0000","last_active":"2024-03-16T09:13:39.135533+0000","last_peered":"2024-03-16T09:13:39.135533+0000","last_clean":"2024-03-16T09:13:39.135533+0000","last_became_active":"2024-03-16T09:13:36.113566+0000","last_became_peered":"2024-03-16T09:13:36.113566+0000","last_unstale":"2024-03-16T09:13:39.135533+0000","last_undegraded":"2024-03-16T09:13:39.135533+0000","last_fullsized":"2024-03-16T09:13:39.135533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:47:50.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130410+0000","last_change":"2024-03-16T09:13:37.130801+0000","last_active":"2024-03-16T09:13:39.130410+0000","last_peered":"2024-03-16T09:13:39.130410+0000","last_clean":"2024-03-16T09:13:39.130410+0000","last_became_active":"2024-03-16T09:13:37.130491+0000","last_became_peered":"2024-03-16T09:13:37.130491+0000","last_unstale":"2024-03-16T09:13:39.130410+0000","last_undegraded":"2024-03-16T09:13:39.130410+0000","last_fullsized":"2024-03-16T09:13:39.130410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:09:23.866570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.109281+0000","last_change":"2024-03-16T09:13:36.109585+0000","last_active":"2024-03-16T09:13:37.109281+0000","last_peered":"2024-03-16T09:13:37.109281+0000","last_clean":"2024-03-16T09:13:37.109281+0000","last_became_active":"2024-03-16T09:13:36.109267+0000","last_became_peered":"2024-03-16T09:13:36.109267+0000","last_unstale":"2024-03-16T09:13:37.109281+0000","last_undegraded":"2024-03-16T09:13:37.109281+0000","last_fullsized":"2024-03-16T09:13:37.109281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:26:55.700791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.129224+0000","last_change":"2024-03-16T09:13:37.136465+0000","last_active":"2024-03-16T09:13:39.129224+0000","last_peered":"2024-03-16T09:13:39.129224+0000","last_clean":"2024-03-16T09:13:39.129224+0000","last_became_active":"2024-03-16T09:13:37.135933+0000","last_became_peered":"2024-03-16T09:13:37.135933+0000","last_unstale":"2024-03-16T09:13:39.129224+0000","last_undegraded":"2024-03-16T09:13:39.129224+0000","last_fullsized":"2024-03-16T09:13:39.129224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:08:28.490533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121206+0000","last_change":"2024-03-16T09:13:36.119428+0000","last_active":"2024-03-16T09:13:38.121206+0000","last_peered":"2024-03-16T09:13:38.121206+0000","last_clean":"2024-03-16T09:13:38.121206+0000","last_became_active":"2024-03-16T09:13:36.119083+0000","last_became_peered":"2024-03-16T09:13:36.119083+0000","last_unstale":"2024-03-16T09:13:38.121206+0000","last_undegraded":"2024-03-16T09:13:38.121206+0000","last_fullsized":"2024-03-16T09:13:38.121206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:58:56.065151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130253+0000","last_change":"2024-03-16T09:13:37.118082+0000","last_active":"2024-03-16T09:13:39.130253+0000","last_peered":"2024-03-16T09:13:39.130253+0000","last_clean":"2024-03-16T09:13:39.130253+0000","last_became_active":"2024-03-16T09:13:37.117671+0000","last_became_peered":"2024-03-16T09:13:37.117671+0000","last_unstale":"2024-03-16T09:13:39.130253+0000","last_undegraded":"2024-03-16T09:13:39.130253+0000","last_fullsized":"2024-03-16T09:13:39.130253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:14:05.352191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121630+0000","last_change":"2024-03-16T09:13:36.121077+0000","last_active":"2024-03-16T09:13:38.121630+0000","last_peered":"2024-03-16T09:13:38.121630+0000","last_clean":"2024-03-16T09:13:38.121630+0000","last_became_active":"2024-03-16T09:13:36.120723+0000","last_became_peered":"2024-03-16T09:13:36.120723+0000","last_unstale":"2024-03-16T09:13:38.121630+0000","last_undegraded":"2024-03-16T09:13:38.121630+0000","last_fullsized":"2024-03-16T09:13:38.121630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:58:14.746215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.117304+0000","last_change":"2024-03-16T09:13:37.116725+0000","last_active":"2024-03-16T09:13:37.117304+0000","last_peered":"2024-03-16T09:13:37.117304+0000","last_clean":"2024-03-16T09:13:37.117304+0000","last_became_active":"2024-03-16T09:13:37.116406+0000","last_became_peered":"2024-03-16T09:13:37.116406+0000","last_unstale":"2024-03-16T09:13:37.117304+0000","last_undegraded":"2024-03-16T09:13:37.117304+0000","last_fullsized":"2024-03-16T09:13:37.117304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:04:07.808008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.119179+0000","last_change":"2024-03-16T09:13:36.110568+0000","last_active":"2024-03-16T09:13:38.119179+0000","last_peered":"2024-03-16T09:13:38.119179+0000","last_clean":"2024-03-16T09:13:38.119179+0000","last_became_active":"2024-03-16T09:13:36.110246+0000","last_became_peered":"2024-03-16T09:13:36.110246+0000","last_unstale":"2024-03-16T09:13:38.119179+0000","last_undegraded":"2024-03-16T09:13:38.119179+0000","last_fullsized":"2024-03-16T09:13:38.119179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:36:50.011941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.247198+0000","last_change":"2024-03-16T09:13:37.122296+0000","last_active":"2024-03-16T09:13:38.247198+0000","last_peered":"2024-03-16T09:13:38.247198+0000","last_clean":"2024-03-16T09:13:38.247198+0000","last_became_active":"2024-03-16T09:13:37.121968+0000","last_became_peered":"2024-03-16T09:13:37.121968+0000","last_unstale":"2024-03-16T09:13:38.247198+0000","last_undegraded":"2024-03-16T09:13:38.247198+0000","last_fullsized":"2024-03-16T09:13:38.247198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:10:43.449397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.129808+0000","last_change":"2024-03-16T09:13:36.112262+0000","last_active":"2024-03-16T09:13:39.129808+0000","last_peered":"2024-03-16T09:13:39.129808+0000","last_clean":"2024-03-16T09:13:39.129808+0000","last_became_active":"2024-03-16T09:13:36.111916+0000","last_became_peered":"2024-03-16T09:13:36.111916+0000","last_unstale":"2024-03-16T09:13:39.129808+0000","last_undegraded":"2024-03-16T09:13:39.129808+0000","last_fullsized":"2024-03-16T09:13:39.129808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:27:41.163508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.247117+0000","last_change":"2024-03-16T09:13:37.119814+0000","last_active":"2024-03-16T09:13:38.247117+0000","last_peered":"2024-03-16T09:13:38.247117+0000","last_clean":"2024-03-16T09:13:38.247117+0000","last_became_active":"2024-03-16T09:13:37.119454+0000","last_became_peered":"2024-03-16T09:13:37.119454+0000","last_unstale":"2024-03-16T09:13:38.247117+0000","last_undegraded":"2024-03-16T09:13:38.247117+0000","last_fullsized":"2024-03-16T09:13:38.247117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:57:25.988044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130491+0000","last_change":"2024-03-16T09:13:37.116728+0000","last_active":"2024-03-16T09:13:39.130491+0000","last_peered":"2024-03-16T09:13:39.130491+0000","last_clean":"2024-03-16T09:13:39.130491+0000","last_became_active":"2024-03-16T09:13:37.116406+0000","last_became_peered":"2024-03-16T09:13:37.116406+0000","last_unstale":"2024-03-16T09:13:39.130491+0000","last_undegraded":"2024-03-16T09:13:39.130491+0000","last_fullsized":"2024-03-16T09:13:39.130491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:38.134284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.099459+0000","last_change":"2024-03-16T09:13:36.111300+0000","last_active":"2024-03-16T09:13:37.099459+0000","last_peered":"2024-03-16T09:13:37.099459+0000","last_clean":"2024-03-16T09:13:37.099459+0000","last_became_active":"2024-03-16T09:13:36.110966+0000","last_became_peered":"2024-03-16T09:13:36.110966+0000","last_unstale":"2024-03-16T09:13:37.099459+0000","last_undegraded":"2024-03-16T09:13:37.099459+0000","last_fullsized":"2024-03-16T09:13:37.099459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:18:33.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.120980+0000","last_change":"2024-03-16T09:13:37.125791+0000","last_active":"2024-03-16T09:13:38.120980+0000","last_peered":"2024-03-16T09:13:38.120980+0000","last_clean":"2024-03-16T09:13:38.120980+0000","last_became_active":"2024-03-16T09:13:37.125448+0000","last_became_peered":"2024-03-16T09:13:37.125448+0000","last_unstale":"2024-03-16T09:13:38.120980+0000","last_undegraded":"2024-03-16T09:13:38.120980+0000","last_fullsized":"2024-03-16T09:13:38.120980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:51:30.767750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.112831+0000","last_change":"2024-03-16T09:13:36.107462+0000","last_active":"2024-03-16T09:13:37.112831+0000","last_peered":"2024-03-16T09:13:37.112831+0000","last_clean":"2024-03-16T09:13:37.112831+0000","last_became_active":"2024-03-16T09:13:36.107115+0000","last_became_peered":"2024-03-16T09:13:36.107115+0000","last_unstale":"2024-03-16T09:13:37.112831+0000","last_undegraded":"2024-03-16T09:13:37.112831+0000","last_fullsized":"2024-03-16T09:13:37.112831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:37.796719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":337,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.129120+0000","last_change":"2024-03-16T09:12:24.331159+0000","last_active":"2024-03-16T09:13:39.129120+0000","last_peered":"2024-03-16T09:13:39.129120+0000","last_clean":"2024-03-16T09:13:39.129120+0000","last_became_active":"2024-03-16T09:12:24.330554+0000","last_became_peered":"2024-03-16T09:12:24.330554+0000","last_unstale":"2024-03-16T09:13:39.129120+0000","last_undegraded":"2024-03-16T09:13:39.129120+0000","last_fullsized":"2024-03-16T09:13:39.129120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:08:57.697012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:08:57.697012+0000","last_clean_scrub_stamp":"2024-03-16T09:08:57.697012+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:43:02.986865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.115533+0000","last_change":"2024-03-16T09:13:37.114299+0000","last_active":"2024-03-16T09:13:37.115533+0000","last_peered":"2024-03-16T09:13:37.115533+0000","last_clean":"2024-03-16T09:13:37.115533+0000","last_became_active":"2024-03-16T09:13:37.113981+0000","last_became_peered":"2024-03-16T09:13:37.113981+0000","last_unstale":"2024-03-16T09:13:37.115533+0000","last_undegraded":"2024-03-16T09:13:37.115533+0000","last_fullsized":"2024-03-16T09:13:37.115533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:57.533537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.135006+0000","last_change":"2024-03-16T09:13:36.113962+0000","last_active":"2024-03-16T09:13:39.135006+0000","last_peered":"2024-03-16T09:13:39.135006+0000","last_clean":"2024-03-16T09:13:39.135006+0000","last_became_active":"2024-03-16T09:13:36.113616+0000","last_became_peered":"2024-03-16T09:13:36.113616+0000","last_unstale":"2024-03-16T09:13:39.135006+0000","last_undegraded":"2024-03-16T09:13:39.135006+0000","last_fullsized":"2024-03-16T09:13:39.135006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:15:15.004360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121506+0000","last_change":"2024-03-16T09:13:36.121946+0000","last_active":"2024-03-16T09:13:38.121506+0000","last_peered":"2024-03-16T09:13:38.121506+0000","last_clean":"2024-03-16T09:13:38.121506+0000","last_became_active":"2024-03-16T09:13:36.121209+0000","last_became_peered":"2024-03-16T09:13:36.121209+0000","last_unstale":"2024-03-16T09:13:38.121506+0000","last_undegraded":"2024-03-16T09:13:38.121506+0000","last_fullsized":"2024-03-16T09:13:38.121506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:09:51.139468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.129423+0000","last_change":"2024-03-16T09:13:37.128931+0000","last_active":"2024-03-16T09:13:37.129423+0000","last_peered":"2024-03-16T09:13:37.129423+0000","last_clean":"2024-03-16T09:13:37.129423+0000","last_became_active":"2024-03-16T09:13:37.128515+0000","last_became_peered":"2024-03-16T09:13:37.128515+0000","last_unstale":"2024-03-16T09:13:37.129423+0000","last_undegraded":"2024-03-16T09:13:37.129423+0000","last_fullsized":"2024-03-16T09:13:37.129423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:39:45.696095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:40.138392+0000","last_change":"2024-03-16T09:13:36.118831+0000","last_active":"2024-03-16T09:13:40.138392+0000","last_peered":"2024-03-16T09:13:40.138392+0000","last_clean":"2024-03-16T09:13:40.138392+0000","last_became_active":"2024-03-16T09:13:36.118489+0000","last_became_peered":"2024-03-16T09:13:36.118489+0000","last_unstale":"2024-03-16T09:13:40.138392+0000","last_undegraded":"2024-03-16T09:13:40.138392+0000","last_fullsized":"2024-03-16T09:13:40.138392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:55:34.089451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121214+0000","last_change":"2024-03-16T09:13:37.120581+0000","last_active":"2024-03-16T09:13:38.121214+0000","last_peered":"2024-03-16T09:13:38.121214+0000","last_clean":"2024-03-16T09:13:38.121214+0000","last_became_active":"2024-03-16T09:13:37.120266+0000","last_became_peered":"2024-03-16T09:13:37.120266+0000","last_unstale":"2024-03-16T09:13:38.121214+0000","last_undegraded":"2024-03-16T09:13:38.121214+0000","last_fullsized":"2024-03-16T09:13:38.121214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:16:10.784229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.132387+0000","last_change":"2024-03-16T09:13:36.107125+0000","last_active":"2024-03-16T09:13:37.132387+0000","last_peered":"2024-03-16T09:13:37.132387+0000","last_clean":"2024-03-16T09:13:37.132387+0000","last_became_active":"2024-03-16T09:13:36.106795+0000","last_became_peered":"2024-03-16T09:13:36.106795+0000","last_unstale":"2024-03-16T09:13:37.132387+0000","last_undegraded":"2024-03-16T09:13:37.132387+0000","last_fullsized":"2024-03-16T09:13:37.132387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:29:10.130565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130456+0000","last_change":"2024-03-16T09:13:37.123013+0000","last_active":"2024-03-16T09:13:39.130456+0000","last_peered":"2024-03-16T09:13:39.130456+0000","last_clean":"2024-03-16T09:13:39.130456+0000","last_became_active":"2024-03-16T09:13:37.122642+0000","last_became_peered":"2024-03-16T09:13:37.122642+0000","last_unstale":"2024-03-16T09:13:39.130456+0000","last_undegraded":"2024-03-16T09:13:39.130456+0000","last_fullsized":"2024-03-16T09:13:39.130456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:35:05.297337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.108638+0000","last_change":"2024-03-16T09:13:36.110868+0000","last_active":"2024-03-16T09:13:38.108638+0000","last_peered":"2024-03-16T09:13:38.108638+0000","last_clean":"2024-03-16T09:13:38.108638+0000","last_became_active":"2024-03-16T09:13:36.110536+0000","last_became_peered":"2024-03-16T09:13:36.110536+0000","last_unstale":"2024-03-16T09:13:38.108638+0000","last_undegraded":"2024-03-16T09:13:38.108638+0000","last_fullsized":"2024-03-16T09:13:38.108638+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:07:06.829458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121389+0000","last_change":"2024-03-16T09:13:37.116579+0000","last_active":"2024-03-16T09:13:38.121389+0000","last_peered":"2024-03-16T09:13:38.121389+0000","last_clean":"2024-03-16T09:13:38.121389+0000","last_became_active":"2024-03-16T09:13:37.116226+0000","last_became_peered":"2024-03-16T09:13:37.116226+0000","last_unstale":"2024-03-16T09:13:38.121389+0000","last_undegraded":"2024-03-16T09:13:38.121389+0000","last_fullsized":"2024-03-16T09:13:38.121389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:08:39.015162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.131174+0000","last_change":"2024-03-16T09:13:36.107881+0000","last_active":"2024-03-16T09:13:39.131174+0000","last_peered":"2024-03-16T09:13:39.131174+0000","last_clean":"2024-03-16T09:13:39.131174+0000","last_became_active":"2024-03-16T09:13:36.107538+0000","last_became_peered":"2024-03-16T09:13:36.107538+0000","last_unstale":"2024-03-16T09:13:39.131174+0000","last_undegraded":"2024-03-16T09:13:39.131174+0000","last_fullsized":"2024-03-16T09:13:39.131174+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:49:41.932089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.124842+0000","last_change":"2024-03-16T09:13:37.118580+0000","last_active":"2024-03-16T09:13:37.124842+0000","last_peered":"2024-03-16T09:13:37.124842+0000","last_clean":"2024-03-16T09:13:37.124842+0000","last_became_active":"2024-03-16T09:13:37.118166+0000","last_became_peered":"2024-03-16T09:13:37.118166+0000","last_unstale":"2024-03-16T09:13:37.124842+0000","last_undegraded":"2024-03-16T09:13:37.124842+0000","last_fullsized":"2024-03-16T09:13:37.124842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:15:54.370764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.108787+0000","last_change":"2024-03-16T09:13:36.122307+0000","last_active":"2024-03-16T09:13:38.108787+0000","last_peered":"2024-03-16T09:13:38.108787+0000","last_clean":"2024-03-16T09:13:38.108787+0000","last_became_active":"2024-03-16T09:13:36.121961+0000","last_became_peered":"2024-03-16T09:13:36.121961+0000","last_unstale":"2024-03-16T09:13:38.108787+0000","last_undegraded":"2024-03-16T09:13:38.108787+0000","last_fullsized":"2024-03-16T09:13:38.108787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:50:03.482300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121121+0000","last_change":"2024-03-16T09:13:37.131918+0000","last_active":"2024-03-16T09:13:38.121121+0000","last_peered":"2024-03-16T09:13:38.121121+0000","last_clean":"2024-03-16T09:13:38.121121+0000","last_became_active":"2024-03-16T09:13:37.131560+0000","last_became_peered":"2024-03-16T09:13:37.131560+0000","last_unstale":"2024-03-16T09:13:38.121121+0000","last_undegraded":"2024-03-16T09:13:38.121121+0000","last_fullsized":"2024-03-16T09:13:38.121121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:57.315125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121288+0000","last_change":"2024-03-16T09:13:36.120192+0000","last_active":"2024-03-16T09:13:38.121288+0000","last_peered":"2024-03-16T09:13:38.121288+0000","last_clean":"2024-03-16T09:13:38.121288+0000","last_became_active":"2024-03-16T09:13:36.119063+0000","last_became_peered":"2024-03-16T09:13:36.119063+0000","last_unstale":"2024-03-16T09:13:38.121288+0000","last_undegraded":"2024-03-16T09:13:38.121288+0000","last_fullsized":"2024-03-16T09:13:38.121288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:18:56.616624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130027+0000","last_change":"2024-03-16T09:13:37.129597+0000","last_active":"2024-03-16T09:13:39.130027+0000","last_peered":"2024-03-16T09:13:39.130027+0000","last_clean":"2024-03-16T09:13:39.130027+0000","last_became_active":"2024-03-16T09:13:37.129248+0000","last_became_peered":"2024-03-16T09:13:37.129248+0000","last_unstale":"2024-03-16T09:13:39.130027+0000","last_undegraded":"2024-03-16T09:13:39.130027+0000","last_fullsized":"2024-03-16T09:13:39.130027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:41:11.118192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:40.139671+0000","last_change":"2024-03-16T09:13:36.116652+0000","last_active":"2024-03-16T09:13:40.139671+0000","last_peered":"2024-03-16T09:13:40.139671+0000","last_clean":"2024-03-16T09:13:40.139671+0000","last_became_active":"2024-03-16T09:13:36.116324+0000","last_became_peered":"2024-03-16T09:13:36.116324+0000","last_unstale":"2024-03-16T09:13:40.139671+0000","last_undegraded":"2024-03-16T09:13:40.139671+0000","last_fullsized":"2024-03-16T09:13:40.139671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:45:10.185614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130795+0000","last_change":"2024-03-16T09:13:37.120317+0000","last_active":"2024-03-16T09:13:39.130795+0000","last_peered":"2024-03-16T09:13:39.130795+0000","last_clean":"2024-03-16T09:13:39.130795+0000","last_became_active":"2024-03-16T09:13:37.119974+0000","last_became_peered":"2024-03-16T09:13:37.119974+0000","last_unstale":"2024-03-16T09:13:39.130795+0000","last_undegraded":"2024-03-16T09:13:39.130795+0000","last_fullsized":"2024-03-16T09:13:39.130795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:25:34.774377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130531+0000","last_change":"2024-03-16T09:13:36.112395+0000","last_active":"2024-03-16T09:13:39.130531+0000","last_peered":"2024-03-16T09:13:39.130531+0000","last_clean":"2024-03-16T09:13:39.130531+0000","last_became_active":"2024-03-16T09:13:36.112036+0000","last_became_peered":"2024-03-16T09:13:36.112036+0000","last_unstale":"2024-03-16T09:13:39.130531+0000","last_undegraded":"2024-03-16T09:13:39.130531+0000","last_fullsized":"2024-03-16T09:13:39.130531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:41:02.895820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.131112+0000","last_change":"2024-03-16T09:13:37.133845+0000","last_active":"2024-03-16T09:13:39.131112+0000","last_peered":"2024-03-16T09:13:39.131112+0000","last_clean":"2024-03-16T09:13:39.131112+0000","last_became_active":"2024-03-16T09:13:37.132833+0000","last_became_peered":"2024-03-16T09:13:37.132833+0000","last_unstale":"2024-03-16T09:13:39.131112+0000","last_undegraded":"2024-03-16T09:13:39.131112+0000","last_fullsized":"2024-03-16T09:13:39.131112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:12:18.049210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.128594+0000","last_change":"2024-03-16T09:13:36.114830+0000","last_active":"2024-03-16T09:13:37.128594+0000","last_peered":"2024-03-16T09:13:37.128594+0000","last_clean":"2024-03-16T09:13:37.128594+0000","last_became_active":"2024-03-16T09:13:36.113898+0000","last_became_peered":"2024-03-16T09:13:36.113898+0000","last_unstale":"2024-03-16T09:13:37.128594+0000","last_undegraded":"2024-03-16T09:13:37.128594+0000","last_fullsized":"2024-03-16T09:13:37.128594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:27:38.910819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.131273+0000","last_change":"2024-03-16T09:13:37.115991+0000","last_active":"2024-03-16T09:13:39.131273+0000","last_peered":"2024-03-16T09:13:39.131273+0000","last_clean":"2024-03-16T09:13:39.131273+0000","last_became_active":"2024-03-16T09:13:37.115671+0000","last_became_peered":"2024-03-16T09:13:37.115671+0000","last_unstale":"2024-03-16T09:13:39.131273+0000","last_undegraded":"2024-03-16T09:13:39.131273+0000","last_fullsized":"2024-03-16T09:13:39.131273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:05:59.703675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.115823+0000","last_change":"2024-03-16T09:13:37.115122+0000","last_active":"2024-03-16T09:13:37.115823+0000","last_peered":"2024-03-16T09:13:37.115823+0000","last_clean":"2024-03-16T09:13:37.115823+0000","last_became_active":"2024-03-16T09:13:37.114779+0000","last_became_peered":"2024-03-16T09:13:37.114779+0000","last_unstale":"2024-03-16T09:13:37.115823+0000","last_undegraded":"2024-03-16T09:13:37.115823+0000","last_fullsized":"2024-03-16T09:13:37.115823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:19:09.501237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.108934+0000","last_change":"2024-03-16T09:13:37.116542+0000","last_active":"2024-03-16T09:13:38.108934+0000","last_peered":"2024-03-16T09:13:38.108934+0000","last_clean":"2024-03-16T09:13:38.108934+0000","last_became_active":"2024-03-16T09:13:37.116224+0000","last_became_peered":"2024-03-16T09:13:37.116224+0000","last_unstale":"2024-03-16T09:13:38.108934+0000","last_undegraded":"2024-03-16T09:13:38.108934+0000","last_fullsized":"2024-03-16T09:13:38.108934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:32:40.106979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.108831+0000","last_change":"2024-03-16T09:13:37.128750+0000","last_active":"2024-03-16T09:13:38.108831+0000","last_peered":"2024-03-16T09:13:38.108831+0000","last_clean":"2024-03-16T09:13:38.108831+0000","last_became_active":"2024-03-16T09:13:37.128411+0000","last_became_peered":"2024-03-16T09:13:37.128411+0000","last_unstale":"2024-03-16T09:13:38.108831+0000","last_undegraded":"2024-03-16T09:13:38.108831+0000","last_fullsized":"2024-03-16T09:13:38.108831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:01:02.119578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130588+0000","last_change":"2024-03-16T09:13:37.124024+0000","last_active":"2024-03-16T09:13:39.130588+0000","last_peered":"2024-03-16T09:13:39.130588+0000","last_clean":"2024-03-16T09:13:39.130588+0000","last_became_active":"2024-03-16T09:13:37.123721+0000","last_became_peered":"2024-03-16T09:13:37.123721+0000","last_unstale":"2024-03-16T09:13:39.130588+0000","last_undegraded":"2024-03-16T09:13:39.130588+0000","last_fullsized":"2024-03-16T09:13:39.130588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:29:28.079163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.116966+0000","last_change":"2024-03-16T09:13:37.116603+0000","last_active":"2024-03-16T09:13:37.116966+0000","last_peered":"2024-03-16T09:13:37.116966+0000","last_clean":"2024-03-16T09:13:37.116966+0000","last_became_active":"2024-03-16T09:13:37.116279+0000","last_became_peered":"2024-03-16T09:13:37.116279+0000","last_unstale":"2024-03-16T09:13:37.116966+0000","last_undegraded":"2024-03-16T09:13:37.116966+0000","last_fullsized":"2024-03-16T09:13:37.116966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:47:04.061172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.117780+0000","last_change":"2024-03-16T09:13:37.116863+0000","last_active":"2024-03-16T09:13:37.117780+0000","last_peered":"2024-03-16T09:13:37.117780+0000","last_clean":"2024-03-16T09:13:37.117780+0000","last_became_active":"2024-03-16T09:13:37.116543+0000","last_became_peered":"2024-03-16T09:13:37.116543+0000","last_unstale":"2024-03-16T09:13:37.117780+0000","last_undegraded":"2024-03-16T09:13:37.117780+0000","last_fullsized":"2024-03-16T09:13:37.117780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:51:28.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.109259+0000","last_change":"2024-03-16T09:13:37.120325+0000","last_active":"2024-03-16T09:13:38.109259+0000","last_peered":"2024-03-16T09:13:38.109259+0000","last_clean":"2024-03-16T09:13:38.109259+0000","last_became_active":"2024-03-16T09:13:37.119985+0000","last_became_peered":"2024-03-16T09:13:37.119985+0000","last_unstale":"2024-03-16T09:13:38.109259+0000","last_undegraded":"2024-03-16T09:13:38.109259+0000","last_fullsized":"2024-03-16T09:13:38.109259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:35.419225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121115+0000","last_change":"2024-03-16T09:13:37.119604+0000","last_active":"2024-03-16T09:13:38.121115+0000","last_peered":"2024-03-16T09:13:38.121115+0000","last_clean":"2024-03-16T09:13:38.121115+0000","last_became_active":"2024-03-16T09:13:37.119019+0000","last_became_peered":"2024-03-16T09:13:37.119019+0000","last_unstale":"2024-03-16T09:13:38.121115+0000","last_undegraded":"2024-03-16T09:13:38.121115+0000","last_fullsized":"2024-03-16T09:13:38.121115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:42:57.902370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130290+0000","last_change":"2024-03-16T09:13:37.123845+0000","last_active":"2024-03-16T09:13:39.130290+0000","last_peered":"2024-03-16T09:13:39.130290+0000","last_clean":"2024-03-16T09:13:39.130290+0000","last_became_active":"2024-03-16T09:13:37.121753+0000","last_became_peered":"2024-03-16T09:13:37.121753+0000","last_unstale":"2024-03-16T09:13:39.130290+0000","last_undegraded":"2024-03-16T09:13:39.130290+0000","last_fullsized":"2024-03-16T09:13:39.130290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:23:01.289378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.120704+0000","last_change":"2024-03-16T09:13:37.118182+0000","last_active":"2024-03-16T09:13:38.120704+0000","last_peered":"2024-03-16T09:13:38.120704+0000","last_clean":"2024-03-16T09:13:38.120704+0000","last_became_active":"2024-03-16T09:13:37.117820+0000","last_became_peered":"2024-03-16T09:13:37.117820+0000","last_unstale":"2024-03-16T09:13:38.120704+0000","last_undegraded":"2024-03-16T09:13:38.120704+0000","last_fullsized":"2024-03-16T09:13:38.120704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:27:31.393266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.108990+0000","last_change":"2024-03-16T09:13:37.121348+0000","last_active":"2024-03-16T09:13:38.108990+0000","last_peered":"2024-03-16T09:13:38.108990+0000","last_clean":"2024-03-16T09:13:38.108990+0000","last_became_active":"2024-03-16T09:13:37.120826+0000","last_became_peered":"2024-03-16T09:13:37.120826+0000","last_unstale":"2024-03-16T09:13:38.108990+0000","last_undegraded":"2024-03-16T09:13:38.108990+0000","last_fullsized":"2024-03-16T09:13:38.108990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:36:33.456230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.133210+0000","last_change":"2024-03-16T09:13:37.131889+0000","last_active":"2024-03-16T09:13:37.133210+0000","last_peered":"2024-03-16T09:13:37.133210+0000","last_clean":"2024-03-16T09:13:37.133210+0000","last_became_active":"2024-03-16T09:13:37.131564+0000","last_became_peered":"2024-03-16T09:13:37.131564+0000","last_unstale":"2024-03-16T09:13:37.133210+0000","last_undegraded":"2024-03-16T09:13:37.133210+0000","last_fullsized":"2024-03-16T09:13:37.133210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:53:41.448241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.125095+0000","last_change":"2024-03-16T09:13:37.120041+0000","last_active":"2024-03-16T09:13:37.125095+0000","last_peered":"2024-03-16T09:13:37.125095+0000","last_clean":"2024-03-16T09:13:37.125095+0000","last_became_active":"2024-03-16T09:13:37.118989+0000","last_became_peered":"2024-03-16T09:13:37.118989+0000","last_unstale":"2024-03-16T09:13:37.125095+0000","last_undegraded":"2024-03-16T09:13:37.125095+0000","last_fullsized":"2024-03-16T09:13:37.125095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:15:14.529994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280795,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4376,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":88,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":212992,"data_stored":32149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383693,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":429951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645330,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":1031457,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":439339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070492,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":430555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332130,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":1020901,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495654,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":429951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":905216,"data_stored":430041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":441483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":438805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":429973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757292,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":1030365,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":22245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T09:13:44.014 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T09:13:44.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.149+0000 7f08242f1640 1 -- 172.21.15.112:0/2926445338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f081c100720 msgr2=0x7f081c100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:44.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.149+0000 7f08242f1640 1 --2- 172.21.15.112:0/2926445338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f081c100720 0x7f081c100b40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0818009900 tx=0x7f081802bc30 comp rx=0 tx=0).stop 2024-03-16T09:13:44.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.149+0000 7f08242f1640 1 -- 172.21.15.112:0/2926445338 shutdown_connections 2024-03-16T09:13:44.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.149+0000 7f08242f1640 1 --2- 172.21.15.112:0/2926445338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f081c1022d0 0x7f081c108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:44.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.149+0000 7f08242f1640 1 --2- 172.21.15.112:0/2926445338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f081c101910 0x7f081c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:44.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.149+0000 7f08242f1640 1 --2- 172.21.15.112:0/2926445338 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f081c100720 0x7f081c100b40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.149+0000 7f08242f1640 1 -- 172.21.15.112:0/2926445338 >> 172.21.15.112:0/2926445338 conn(0x7f081c0fbc90 msgr2=0x7f081c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.149+0000 7f08242f1640 1 -- 172.21.15.112:0/2926445338 shutdown_connections 2024-03-16T09:13:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.149+0000 7f08242f1640 1 -- 172.21.15.112:0/2926445338 wait complete. 2024-03-16T09:13:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f08242f1640 1 Processor -- start 2024-03-16T09:13:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f08242f1640 1 -- start start 2024-03-16T09:13:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f08242f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f081c100720 0x7f081c06a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f08242f1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f081c101910 0x7f081c19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f08242f1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f081c1022d0 0x7f081c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f08242f1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f081c10c480 con 0x7f081c100720 2024-03-16T09:13:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f08242f1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f081c10c300 con 0x7f081c101910 2024-03-16T09:13:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f08242f1640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f081c10c600 con 0x7f081c1022d0 2024-03-16T09:13:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f082208d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f081c100720 0x7f081c06a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f082208d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f081c100720 0x7f081c06a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59230/0 (socket says 172.21.15.112:59230) 2024-03-16T09:13:44.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f082208d640 1 -- 172.21.15.112:0/3185194201 learned_addr learned my addr 172.21.15.112:0/3185194201 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f082188c640 1 --2- 172.21.15.112:0/3185194201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f081c101910 0x7f081c19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f082288e640 1 --2- 172.21.15.112:0/3185194201 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f081c1022d0 0x7f081c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f082208d640 1 -- 172.21.15.112:0/3185194201 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f081c1022d0 msgr2=0x7f081c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f082208d640 1 --2- 172.21.15.112:0/3185194201 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f081c1022d0 0x7f081c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f082208d640 1 -- 172.21.15.112:0/3185194201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f081c101910 msgr2=0x7f081c19c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f082208d640 1 --2- 172.21.15.112:0/3185194201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f081c101910 0x7f081c19c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f082208d640 1 -- 172.21.15.112:0/3185194201 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081c19f880 con 0x7f081c100720 2024-03-16T09:13:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f082208d640 1 --2- 172.21.15.112:0/3185194201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f081c100720 0x7f081c06a980 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0818000c00 tx=0x7f0818004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.150+0000 7f08137fe640 1 -- 172.21.15.112:0/3185194201 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818043020 con 0x7f081c100720 2024-03-16T09:13:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.151+0000 7f08137fe640 1 -- 172.21.15.112:0/3185194201 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0818004250 con 0x7f081c100720 2024-03-16T09:13:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.151+0000 7f08242f1640 1 -- 172.21.15.112:0/3185194201 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081c19fab0 con 0x7f081c100720 2024-03-16T09:13:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.151+0000 7f08137fe640 1 -- 172.21.15.112:0/3185194201 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818003b90 con 0x7f081c100720 2024-03-16T09:13:44.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.151+0000 7f08242f1640 1 -- 172.21.15.112:0/3185194201 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f081c1a0000 con 0x7f081c100720 2024-03-16T09:13:44.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.152+0000 7f08242f1640 1 -- 172.21.15.112:0/3185194201 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f081c107a40 con 0x7f081c100720 2024-03-16T09:13:44.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.153+0000 7f08137fe640 1 -- 172.21.15.112:0/3185194201 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137291+0+0 (secure 0 0 0) 0x7f0818046020 con 0x7f081c100720 2024-03-16T09:13:44.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.153+0000 7f08137fe640 1 --2- 172.21.15.112:0/3185194201 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f08080a2800 0x7f08080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:44.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.153+0000 7f082188c640 1 --2- 172.21.15.112:0/3185194201 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f08080a2800 0x7f08080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:44.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.154+0000 7f08137fe640 1 -- 172.21.15.112:0/3185194201 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f08180f09e0 con 0x7f081c100720 2024-03-16T09:13:44.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.154+0000 7f082188c640 1 --2- 172.21.15.112:0/3185194201 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f08080a2800 0x7f08080a4cc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f081c06a6f0 tx=0x7f080c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:44.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.158+0000 7f08137fe640 1 -- 172.21.15.112:0/3185194201 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f081804d020 con 0x7f081c100720 2024-03-16T09:13:44.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.332+0000 7f08242f1640 1 -- 172.21.15.112:0/3185194201 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f081c061c20 con 0x7f08080a2800 2024-03-16T09:13:44.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.343+0000 7f08137fe640 1 -- 172.21.15.112:0/3185194201 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f08180b9b40 con 0x7f081c100720 2024-03-16T09:13:44.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.344+0000 7f08137fe640 1 -- 172.21.15.112:0/3185194201 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258886 (secure 0 0 0) 0x7f081c061c20 con 0x7f08080a2800 2024-03-16T09:13:44.348 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:44.350 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-03-16T09:13:44.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.347+0000 7f08242f1640 1 -- 172.21.15.112:0/3185194201 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f08080a2800 msgr2=0x7f08080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:44.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.347+0000 7f08242f1640 1 --2- 172.21.15.112:0/3185194201 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f08080a2800 0x7f08080a4cc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f081c06a6f0 tx=0x7f080c005f90 comp rx=0 tx=0).stop 2024-03-16T09:13:44.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.347+0000 7f08242f1640 1 -- 172.21.15.112:0/3185194201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f081c100720 msgr2=0x7f081c06a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:44.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.347+0000 7f08242f1640 1 --2- 172.21.15.112:0/3185194201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f081c100720 0x7f081c06a980 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0818000c00 tx=0x7f0818004a30 comp rx=0 tx=0).stop 2024-03-16T09:13:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.347+0000 7f08242f1640 1 -- 172.21.15.112:0/3185194201 shutdown_connections 2024-03-16T09:13:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.347+0000 7f08242f1640 1 --2- 172.21.15.112:0/3185194201 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f08080a2800 0x7f08080a4cc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.347+0000 7f08242f1640 1 --2- 172.21.15.112:0/3185194201 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f081c1022d0 0x7f081c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.347+0000 7f08242f1640 1 --2- 172.21.15.112:0/3185194201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f081c101910 0x7f081c19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.347+0000 7f08242f1640 1 --2- 172.21.15.112:0/3185194201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f081c100720 0x7f081c06a980 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.347+0000 7f08242f1640 1 -- 172.21.15.112:0/3185194201 >> 172.21.15.112:0/3185194201 conn(0x7f081c0fbc90 msgr2=0x7f081c104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.347+0000 7f08242f1640 1 -- 172.21.15.112:0/3185194201 shutdown_connections 2024-03-16T09:13:44.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:44.347+0000 7f08242f1640 1 -- 172.21.15.112:0/3185194201 wait complete. 2024-03-16T09:13:44.361 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-03-16T09:13:42.875107+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594744,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":140,"num_read_kb":120,"num_write":269,"num_write_kb":2958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":235,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351832,"kb_used_data":10944,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589992,"statfs":{"total":1151940427776,"available":1151580151808,"internally_reserved":0,"allocated":11206656,"data_stored":6992954,"data_compressed":107142,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4376,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":88,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.778266"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.125141+0000","last_change":"2024-03-16T09:13:37.119969+0000","last_active":"2024-03-16T09:13:37.125141+0000","last_peered":"2024-03-16T09:13:37.125141+0000","last_clean":"2024-03-16T09:13:37.125141+0000","last_became_active":"2024-03-16T09:13:37.119177+0000","last_became_peered":"2024-03-16T09:13:37.119177+0000","last_unstale":"2024-03-16T09:13:37.125141+0000","last_undegraded":"2024-03-16T09:13:37.125141+0000","last_fullsized":"2024-03-16T09:13:37.125141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:58:30.822116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130335+0000","last_change":"2024-03-16T09:13:37.127564+0000","last_active":"2024-03-16T09:13:39.130335+0000","last_peered":"2024-03-16T09:13:39.130335+0000","last_clean":"2024-03-16T09:13:39.130335+0000","last_became_active":"2024-03-16T09:13:37.127214+0000","last_became_peered":"2024-03-16T09:13:37.127214+0000","last_unstale":"2024-03-16T09:13:39.130335+0000","last_undegraded":"2024-03-16T09:13:39.130335+0000","last_fullsized":"2024-03-16T09:13:39.130335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:46:14.655707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130296+0000","last_change":"2024-03-16T09:13:37.126288+0000","last_active":"2024-03-16T09:13:39.130296+0000","last_peered":"2024-03-16T09:13:39.130296+0000","last_clean":"2024-03-16T09:13:39.130296+0000","last_became_active":"2024-03-16T09:13:37.125383+0000","last_became_peered":"2024-03-16T09:13:37.125383+0000","last_unstale":"2024-03-16T09:13:39.130296+0000","last_undegraded":"2024-03-16T09:13:39.130296+0000","last_fullsized":"2024-03-16T09:13:39.130296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:47:09.901508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.126813+0000","last_change":"2024-03-16T09:13:37.125503+0000","last_active":"2024-03-16T09:13:37.126813+0000","last_peered":"2024-03-16T09:13:37.126813+0000","last_clean":"2024-03-16T09:13:37.126813+0000","last_became_active":"2024-03-16T09:13:37.125144+0000","last_became_peered":"2024-03-16T09:13:37.125144+0000","last_unstale":"2024-03-16T09:13:37.126813+0000","last_undegraded":"2024-03-16T09:13:37.126813+0000","last_fullsized":"2024-03-16T09:13:37.126813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:23:39.839396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121380+0000","last_change":"2024-03-16T09:13:37.114237+0000","last_active":"2024-03-16T09:13:38.121380+0000","last_peered":"2024-03-16T09:13:38.121380+0000","last_clean":"2024-03-16T09:13:38.121380+0000","last_became_active":"2024-03-16T09:13:37.113595+0000","last_became_peered":"2024-03-16T09:13:37.113595+0000","last_unstale":"2024-03-16T09:13:38.121380+0000","last_undegraded":"2024-03-16T09:13:38.121380+0000","last_fullsized":"2024-03-16T09:13:38.121380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:06:17.735301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.246871+0000","last_change":"2024-03-16T09:13:37.118843+0000","last_active":"2024-03-16T09:13:38.246871+0000","last_peered":"2024-03-16T09:13:38.246871+0000","last_clean":"2024-03-16T09:13:38.246871+0000","last_became_active":"2024-03-16T09:13:37.117086+0000","last_became_peered":"2024-03-16T09:13:37.117086+0000","last_unstale":"2024-03-16T09:13:38.246871+0000","last_undegraded":"2024-03-16T09:13:38.246871+0000","last_fullsized":"2024-03-16T09:13:38.246871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:23:12.929931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.129493+0000","last_change":"2024-03-16T09:13:37.133427+0000","last_active":"2024-03-16T09:13:39.129493+0000","last_peered":"2024-03-16T09:13:39.129493+0000","last_clean":"2024-03-16T09:13:39.129493+0000","last_became_active":"2024-03-16T09:13:37.133032+0000","last_became_peered":"2024-03-16T09:13:37.133032+0000","last_unstale":"2024-03-16T09:13:39.129493+0000","last_undegraded":"2024-03-16T09:13:39.129493+0000","last_fullsized":"2024-03-16T09:13:39.129493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:49:28.179883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.120741+0000","last_change":"2024-03-16T09:13:37.132132+0000","last_active":"2024-03-16T09:13:38.120741+0000","last_peered":"2024-03-16T09:13:38.120741+0000","last_clean":"2024-03-16T09:13:38.120741+0000","last_became_active":"2024-03-16T09:13:37.131801+0000","last_became_peered":"2024-03-16T09:13:37.131801+0000","last_unstale":"2024-03-16T09:13:38.120741+0000","last_undegraded":"2024-03-16T09:13:38.120741+0000","last_fullsized":"2024-03-16T09:13:38.120741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:45:03.891725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.109314+0000","last_change":"2024-03-16T09:13:37.116378+0000","last_active":"2024-03-16T09:13:38.109314+0000","last_peered":"2024-03-16T09:13:38.109314+0000","last_clean":"2024-03-16T09:13:38.109314+0000","last_became_active":"2024-03-16T09:13:37.116027+0000","last_became_peered":"2024-03-16T09:13:37.116027+0000","last_unstale":"2024-03-16T09:13:38.109314+0000","last_undegraded":"2024-03-16T09:13:38.109314+0000","last_fullsized":"2024-03-16T09:13:38.109314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:14:38.681613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.247275+0000","last_change":"2024-03-16T09:13:37.121417+0000","last_active":"2024-03-16T09:13:38.247275+0000","last_peered":"2024-03-16T09:13:38.247275+0000","last_clean":"2024-03-16T09:13:38.247275+0000","last_became_active":"2024-03-16T09:13:37.121112+0000","last_became_peered":"2024-03-16T09:13:37.121112+0000","last_unstale":"2024-03-16T09:13:38.247275+0000","last_undegraded":"2024-03-16T09:13:38.247275+0000","last_fullsized":"2024-03-16T09:13:38.247275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:38:07.823604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130669+0000","last_change":"2024-03-16T09:13:37.121368+0000","last_active":"2024-03-16T09:13:39.130669+0000","last_peered":"2024-03-16T09:13:39.130669+0000","last_clean":"2024-03-16T09:13:39.130669+0000","last_became_active":"2024-03-16T09:13:37.121060+0000","last_became_peered":"2024-03-16T09:13:37.121060+0000","last_unstale":"2024-03-16T09:13:39.130669+0000","last_undegraded":"2024-03-16T09:13:39.130669+0000","last_fullsized":"2024-03-16T09:13:39.130669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:48:49.168285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.128968+0000","last_change":"2024-03-16T09:13:37.127004+0000","last_active":"2024-03-16T09:13:37.128968+0000","last_peered":"2024-03-16T09:13:37.128968+0000","last_clean":"2024-03-16T09:13:37.128968+0000","last_became_active":"2024-03-16T09:13:37.126579+0000","last_became_peered":"2024-03-16T09:13:37.126579+0000","last_unstale":"2024-03-16T09:13:37.128968+0000","last_undegraded":"2024-03-16T09:13:37.128968+0000","last_fullsized":"2024-03-16T09:13:37.128968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:44:40.229978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.105639+0000","last_change":"2024-03-16T09:13:37.104534+0000","last_active":"2024-03-16T09:13:37.105639+0000","last_peered":"2024-03-16T09:13:37.105639+0000","last_clean":"2024-03-16T09:13:37.105639+0000","last_became_active":"2024-03-16T09:13:37.104202+0000","last_became_peered":"2024-03-16T09:13:37.104202+0000","last_unstale":"2024-03-16T09:13:37.105639+0000","last_undegraded":"2024-03-16T09:13:37.105639+0000","last_fullsized":"2024-03-16T09:13:37.105639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:30:31.196479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130427+0000","last_change":"2024-03-16T09:13:37.117166+0000","last_active":"2024-03-16T09:13:39.130427+0000","last_peered":"2024-03-16T09:13:39.130427+0000","last_clean":"2024-03-16T09:13:39.130427+0000","last_became_active":"2024-03-16T09:13:37.116747+0000","last_became_peered":"2024-03-16T09:13:37.116747+0000","last_unstale":"2024-03-16T09:13:39.130427+0000","last_undegraded":"2024-03-16T09:13:39.130427+0000","last_fullsized":"2024-03-16T09:13:39.130427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:50:58.705683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130296+0000","last_change":"2024-03-16T09:13:37.124898+0000","last_active":"2024-03-16T09:13:39.130296+0000","last_peered":"2024-03-16T09:13:39.130296+0000","last_clean":"2024-03-16T09:13:39.130296+0000","last_became_active":"2024-03-16T09:13:37.124623+0000","last_became_peered":"2024-03-16T09:13:37.124623+0000","last_unstale":"2024-03-16T09:13:39.130296+0000","last_undegraded":"2024-03-16T09:13:39.130296+0000","last_fullsized":"2024-03-16T09:13:39.130296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:52:35.159740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.108690+0000","last_change":"2024-03-16T09:13:37.120583+0000","last_active":"2024-03-16T09:13:38.108690+0000","last_peered":"2024-03-16T09:13:38.108690+0000","last_clean":"2024-03-16T09:13:38.108690+0000","last_became_active":"2024-03-16T09:13:37.120217+0000","last_became_peered":"2024-03-16T09:13:37.120217+0000","last_unstale":"2024-03-16T09:13:38.108690+0000","last_undegraded":"2024-03-16T09:13:38.108690+0000","last_fullsized":"2024-03-16T09:13:38.108690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:00:17.653736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.125163+0000","last_change":"2024-03-16T09:13:37.121179+0000","last_active":"2024-03-16T09:13:37.125163+0000","last_peered":"2024-03-16T09:13:37.125163+0000","last_clean":"2024-03-16T09:13:37.125163+0000","last_became_active":"2024-03-16T09:13:37.120070+0000","last_became_peered":"2024-03-16T09:13:37.120070+0000","last_unstale":"2024-03-16T09:13:37.125163+0000","last_undegraded":"2024-03-16T09:13:37.125163+0000","last_fullsized":"2024-03-16T09:13:37.125163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:59:30.359969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.119335+0000","last_change":"2024-03-16T09:13:37.110074+0000","last_active":"2024-03-16T09:13:38.119335+0000","last_peered":"2024-03-16T09:13:38.119335+0000","last_clean":"2024-03-16T09:13:38.119335+0000","last_became_active":"2024-03-16T09:13:37.109767+0000","last_became_peered":"2024-03-16T09:13:37.109767+0000","last_unstale":"2024-03-16T09:13:38.119335+0000","last_undegraded":"2024-03-16T09:13:38.119335+0000","last_fullsized":"2024-03-16T09:13:38.119335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:20:37.408262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.109132+0000","last_change":"2024-03-16T09:13:37.130791+0000","last_active":"2024-03-16T09:13:38.109132+0000","last_peered":"2024-03-16T09:13:38.109132+0000","last_clean":"2024-03-16T09:13:38.109132+0000","last_became_active":"2024-03-16T09:13:37.130428+0000","last_became_peered":"2024-03-16T09:13:37.130428+0000","last_unstale":"2024-03-16T09:13:38.109132+0000","last_undegraded":"2024-03-16T09:13:38.109132+0000","last_fullsized":"2024-03-16T09:13:38.109132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:32:33.185589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:40.139597+0000","last_change":"2024-03-16T09:13:36.119881+0000","last_active":"2024-03-16T09:13:40.139597+0000","last_peered":"2024-03-16T09:13:40.139597+0000","last_clean":"2024-03-16T09:13:40.139597+0000","last_became_active":"2024-03-16T09:13:36.119513+0000","last_became_peered":"2024-03-16T09:13:36.119513+0000","last_unstale":"2024-03-16T09:13:40.139597+0000","last_undegraded":"2024-03-16T09:13:40.139597+0000","last_fullsized":"2024-03-16T09:13:40.139597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:47:35.202982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121124+0000","last_change":"2024-03-16T09:13:37.124946+0000","last_active":"2024-03-16T09:13:38.121124+0000","last_peered":"2024-03-16T09:13:38.121124+0000","last_clean":"2024-03-16T09:13:38.121124+0000","last_became_active":"2024-03-16T09:13:37.124624+0000","last_became_peered":"2024-03-16T09:13:37.124624+0000","last_unstale":"2024-03-16T09:13:38.121124+0000","last_undegraded":"2024-03-16T09:13:38.121124+0000","last_fullsized":"2024-03-16T09:13:38.121124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:18:15.188823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121291+0000","last_change":"2024-03-16T09:13:36.116312+0000","last_active":"2024-03-16T09:13:38.121291+0000","last_peered":"2024-03-16T09:13:38.121291+0000","last_clean":"2024-03-16T09:13:38.121291+0000","last_became_active":"2024-03-16T09:13:36.115999+0000","last_became_peered":"2024-03-16T09:13:36.115999+0000","last_unstale":"2024-03-16T09:13:38.121291+0000","last_undegraded":"2024-03-16T09:13:38.121291+0000","last_fullsized":"2024-03-16T09:13:38.121291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:04:46.795291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130563+0000","last_change":"2024-03-16T09:13:37.126024+0000","last_active":"2024-03-16T09:13:39.130563+0000","last_peered":"2024-03-16T09:13:39.130563+0000","last_clean":"2024-03-16T09:13:39.130563+0000","last_became_active":"2024-03-16T09:13:37.125722+0000","last_became_peered":"2024-03-16T09:13:37.125722+0000","last_unstale":"2024-03-16T09:13:39.130563+0000","last_undegraded":"2024-03-16T09:13:39.130563+0000","last_fullsized":"2024-03-16T09:13:39.130563+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:27:24.923022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121707+0000","last_change":"2024-03-16T09:13:36.119700+0000","last_active":"2024-03-16T09:13:38.121707+0000","last_peered":"2024-03-16T09:13:38.121707+0000","last_clean":"2024-03-16T09:13:38.121707+0000","last_became_active":"2024-03-16T09:13:36.119361+0000","last_became_peered":"2024-03-16T09:13:36.119361+0000","last_unstale":"2024-03-16T09:13:38.121707+0000","last_undegraded":"2024-03-16T09:13:38.121707+0000","last_fullsized":"2024-03-16T09:13:38.121707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:26:01.238914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.108897+0000","last_change":"2024-03-16T09:13:37.127593+0000","last_active":"2024-03-16T09:13:38.108897+0000","last_peered":"2024-03-16T09:13:38.108897+0000","last_clean":"2024-03-16T09:13:38.108897+0000","last_became_active":"2024-03-16T09:13:37.127255+0000","last_became_peered":"2024-03-16T09:13:37.127255+0000","last_unstale":"2024-03-16T09:13:38.108897+0000","last_undegraded":"2024-03-16T09:13:38.108897+0000","last_fullsized":"2024-03-16T09:13:38.108897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:07:51.263266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121586+0000","last_change":"2024-03-16T09:13:36.117583+0000","last_active":"2024-03-16T09:13:38.121586+0000","last_peered":"2024-03-16T09:13:38.121586+0000","last_clean":"2024-03-16T09:13:38.121586+0000","last_became_active":"2024-03-16T09:13:36.117270+0000","last_became_peered":"2024-03-16T09:13:36.117270+0000","last_unstale":"2024-03-16T09:13:38.121586+0000","last_undegraded":"2024-03-16T09:13:38.121586+0000","last_fullsized":"2024-03-16T09:13:38.121586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:10:53.541872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130671+0000","last_change":"2024-03-16T09:13:37.119019+0000","last_active":"2024-03-16T09:13:39.130671+0000","last_peered":"2024-03-16T09:13:39.130671+0000","last_clean":"2024-03-16T09:13:39.130671+0000","last_became_active":"2024-03-16T09:13:37.118148+0000","last_became_peered":"2024-03-16T09:13:37.118148+0000","last_unstale":"2024-03-16T09:13:39.130671+0000","last_undegraded":"2024-03-16T09:13:39.130671+0000","last_fullsized":"2024-03-16T09:13:39.130671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:17:43.042779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:40.326228+0000","last_change":"2024-03-16T09:13:36.114415+0000","last_active":"2024-03-16T09:13:40.326228+0000","last_peered":"2024-03-16T09:13:40.326228+0000","last_clean":"2024-03-16T09:13:40.326228+0000","last_became_active":"2024-03-16T09:13:36.114056+0000","last_became_peered":"2024-03-16T09:13:36.114056+0000","last_unstale":"2024-03-16T09:13:40.326228+0000","last_undegraded":"2024-03-16T09:13:40.326228+0000","last_fullsized":"2024-03-16T09:13:40.326228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:50:28.322627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121411+0000","last_change":"2024-03-16T09:13:37.124997+0000","last_active":"2024-03-16T09:13:38.121411+0000","last_peered":"2024-03-16T09:13:38.121411+0000","last_clean":"2024-03-16T09:13:38.121411+0000","last_became_active":"2024-03-16T09:13:37.124664+0000","last_became_peered":"2024-03-16T09:13:37.124664+0000","last_unstale":"2024-03-16T09:13:38.121411+0000","last_undegraded":"2024-03-16T09:13:38.121411+0000","last_fullsized":"2024-03-16T09:13:38.121411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:24:30.104576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:40.139397+0000","last_change":"2024-03-16T09:13:36.121980+0000","last_active":"2024-03-16T09:13:40.139397+0000","last_peered":"2024-03-16T09:13:40.139397+0000","last_clean":"2024-03-16T09:13:40.139397+0000","last_became_active":"2024-03-16T09:13:36.121703+0000","last_became_peered":"2024-03-16T09:13:36.121703+0000","last_unstale":"2024-03-16T09:13:40.139397+0000","last_undegraded":"2024-03-16T09:13:40.139397+0000","last_fullsized":"2024-03-16T09:13:40.139397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:40:26.601688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.132653+0000","last_change":"2024-03-16T09:13:37.123443+0000","last_active":"2024-03-16T09:13:39.132653+0000","last_peered":"2024-03-16T09:13:39.132653+0000","last_clean":"2024-03-16T09:13:39.132653+0000","last_became_active":"2024-03-16T09:13:37.121987+0000","last_became_peered":"2024-03-16T09:13:37.121987+0000","last_unstale":"2024-03-16T09:13:39.132653+0000","last_undegraded":"2024-03-16T09:13:39.132653+0000","last_fullsized":"2024-03-16T09:13:39.132653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:37:08.478552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.099642+0000","last_change":"2024-03-16T09:13:36.106391+0000","last_active":"2024-03-16T09:13:37.099642+0000","last_peered":"2024-03-16T09:13:37.099642+0000","last_clean":"2024-03-16T09:13:37.099642+0000","last_became_active":"2024-03-16T09:13:36.106046+0000","last_became_peered":"2024-03-16T09:13:36.106046+0000","last_unstale":"2024-03-16T09:13:37.099642+0000","last_undegraded":"2024-03-16T09:13:37.099642+0000","last_fullsized":"2024-03-16T09:13:37.099642+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:03:23.255519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.120697+0000","last_change":"2024-03-16T09:13:37.114337+0000","last_active":"2024-03-16T09:13:38.120697+0000","last_peered":"2024-03-16T09:13:38.120697+0000","last_clean":"2024-03-16T09:13:38.120697+0000","last_became_active":"2024-03-16T09:13:37.113991+0000","last_became_peered":"2024-03-16T09:13:37.113991+0000","last_unstale":"2024-03-16T09:13:38.120697+0000","last_undegraded":"2024-03-16T09:13:38.120697+0000","last_fullsized":"2024-03-16T09:13:38.120697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:23:54.141479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130119+0000","last_change":"2024-03-16T09:13:36.113297+0000","last_active":"2024-03-16T09:13:39.130119+0000","last_peered":"2024-03-16T09:13:39.130119+0000","last_clean":"2024-03-16T09:13:39.130119+0000","last_became_active":"2024-03-16T09:13:36.112950+0000","last_became_peered":"2024-03-16T09:13:36.112950+0000","last_unstale":"2024-03-16T09:13:39.130119+0000","last_undegraded":"2024-03-16T09:13:39.130119+0000","last_fullsized":"2024-03-16T09:13:39.130119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:40:17.682450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.119150+0000","last_change":"2024-03-16T09:13:37.112239+0000","last_active":"2024-03-16T09:13:38.119150+0000","last_peered":"2024-03-16T09:13:38.119150+0000","last_clean":"2024-03-16T09:13:38.119150+0000","last_became_active":"2024-03-16T09:13:37.111885+0000","last_became_peered":"2024-03-16T09:13:37.111885+0000","last_unstale":"2024-03-16T09:13:38.119150+0000","last_undegraded":"2024-03-16T09:13:38.119150+0000","last_fullsized":"2024-03-16T09:13:38.119150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:49.565846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.134946+0000","last_change":"2024-03-16T09:13:36.111341+0000","last_active":"2024-03-16T09:13:39.134946+0000","last_peered":"2024-03-16T09:13:39.134946+0000","last_clean":"2024-03-16T09:13:39.134946+0000","last_became_active":"2024-03-16T09:13:36.110558+0000","last_became_peered":"2024-03-16T09:13:36.110558+0000","last_unstale":"2024-03-16T09:13:39.134946+0000","last_undegraded":"2024-03-16T09:13:39.134946+0000","last_fullsized":"2024-03-16T09:13:39.134946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:56:44.252002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.109148+0000","last_change":"2024-03-16T09:13:37.126808+0000","last_active":"2024-03-16T09:13:38.109148+0000","last_peered":"2024-03-16T09:13:38.109148+0000","last_clean":"2024-03-16T09:13:38.109148+0000","last_became_active":"2024-03-16T09:13:37.126442+0000","last_became_peered":"2024-03-16T09:13:37.126442+0000","last_unstale":"2024-03-16T09:13:38.109148+0000","last_undegraded":"2024-03-16T09:13:38.109148+0000","last_fullsized":"2024-03-16T09:13:38.109148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:04:39.216835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.134201+0000","last_change":"2024-03-16T09:13:36.113907+0000","last_active":"2024-03-16T09:13:39.134201+0000","last_peered":"2024-03-16T09:13:39.134201+0000","last_clean":"2024-03-16T09:13:39.134201+0000","last_became_active":"2024-03-16T09:13:36.113564+0000","last_became_peered":"2024-03-16T09:13:36.113564+0000","last_unstale":"2024-03-16T09:13:39.134201+0000","last_undegraded":"2024-03-16T09:13:39.134201+0000","last_fullsized":"2024-03-16T09:13:39.134201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:24:33.272919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130356+0000","last_change":"2024-03-16T09:13:37.125833+0000","last_active":"2024-03-16T09:13:39.130356+0000","last_peered":"2024-03-16T09:13:39.130356+0000","last_clean":"2024-03-16T09:13:39.130356+0000","last_became_active":"2024-03-16T09:13:37.125508+0000","last_became_peered":"2024-03-16T09:13:37.125508+0000","last_unstale":"2024-03-16T09:13:39.130356+0000","last_undegraded":"2024-03-16T09:13:39.130356+0000","last_fullsized":"2024-03-16T09:13:39.130356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:14:47.749715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.134366+0000","last_change":"2024-03-16T09:13:36.112200+0000","last_active":"2024-03-16T09:13:39.134366+0000","last_peered":"2024-03-16T09:13:39.134366+0000","last_clean":"2024-03-16T09:13:39.134366+0000","last_became_active":"2024-03-16T09:13:36.111830+0000","last_became_peered":"2024-03-16T09:13:36.111830+0000","last_unstale":"2024-03-16T09:13:39.134366+0000","last_undegraded":"2024-03-16T09:13:39.134366+0000","last_fullsized":"2024-03-16T09:13:39.134366+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:07:57.147960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.109167+0000","last_change":"2024-03-16T09:13:37.118434+0000","last_active":"2024-03-16T09:13:38.109167+0000","last_peered":"2024-03-16T09:13:38.109167+0000","last_clean":"2024-03-16T09:13:38.109167+0000","last_became_active":"2024-03-16T09:13:37.118069+0000","last_became_peered":"2024-03-16T09:13:37.118069+0000","last_unstale":"2024-03-16T09:13:38.109167+0000","last_undegraded":"2024-03-16T09:13:38.109167+0000","last_fullsized":"2024-03-16T09:13:38.109167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:02:56.559384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:40.325141+0000","last_change":"2024-03-16T09:13:36.114591+0000","last_active":"2024-03-16T09:13:40.325141+0000","last_peered":"2024-03-16T09:13:40.325141+0000","last_clean":"2024-03-16T09:13:40.325141+0000","last_became_active":"2024-03-16T09:13:36.114252+0000","last_became_peered":"2024-03-16T09:13:36.114252+0000","last_unstale":"2024-03-16T09:13:40.325141+0000","last_undegraded":"2024-03-16T09:13:40.325141+0000","last_fullsized":"2024-03-16T09:13:40.325141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:37:52.637394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121706+0000","last_change":"2024-03-16T09:13:37.133102+0000","last_active":"2024-03-16T09:13:38.121706+0000","last_peered":"2024-03-16T09:13:38.121706+0000","last_clean":"2024-03-16T09:13:38.121706+0000","last_became_active":"2024-03-16T09:13:37.132728+0000","last_became_peered":"2024-03-16T09:13:37.132728+0000","last_unstale":"2024-03-16T09:13:38.121706+0000","last_undegraded":"2024-03-16T09:13:38.121706+0000","last_fullsized":"2024-03-16T09:13:38.121706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:54:27.511997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:40.138064+0000","last_change":"2024-03-16T09:13:36.114195+0000","last_active":"2024-03-16T09:13:40.138064+0000","last_peered":"2024-03-16T09:13:40.138064+0000","last_clean":"2024-03-16T09:13:40.138064+0000","last_became_active":"2024-03-16T09:13:36.113829+0000","last_became_peered":"2024-03-16T09:13:36.113829+0000","last_unstale":"2024-03-16T09:13:40.138064+0000","last_undegraded":"2024-03-16T09:13:40.138064+0000","last_fullsized":"2024-03-16T09:13:40.138064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:28:40.509877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.133476+0000","last_change":"2024-03-16T09:13:37.132974+0000","last_active":"2024-03-16T09:13:37.133476+0000","last_peered":"2024-03-16T09:13:37.133476+0000","last_clean":"2024-03-16T09:13:37.133476+0000","last_became_active":"2024-03-16T09:13:37.132474+0000","last_became_peered":"2024-03-16T09:13:37.132474+0000","last_unstale":"2024-03-16T09:13:37.133476+0000","last_undegraded":"2024-03-16T09:13:37.133476+0000","last_fullsized":"2024-03-16T09:13:37.133476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:56:45.637577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.135533+0000","last_change":"2024-03-16T09:13:36.115242+0000","last_active":"2024-03-16T09:13:39.135533+0000","last_peered":"2024-03-16T09:13:39.135533+0000","last_clean":"2024-03-16T09:13:39.135533+0000","last_became_active":"2024-03-16T09:13:36.113566+0000","last_became_peered":"2024-03-16T09:13:36.113566+0000","last_unstale":"2024-03-16T09:13:39.135533+0000","last_undegraded":"2024-03-16T09:13:39.135533+0000","last_fullsized":"2024-03-16T09:13:39.135533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:47:50.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130410+0000","last_change":"2024-03-16T09:13:37.130801+0000","last_active":"2024-03-16T09:13:39.130410+0000","last_peered":"2024-03-16T09:13:39.130410+0000","last_clean":"2024-03-16T09:13:39.130410+0000","last_became_active":"2024-03-16T09:13:37.130491+0000","last_became_peered":"2024-03-16T09:13:37.130491+0000","last_unstale":"2024-03-16T09:13:39.130410+0000","last_undegraded":"2024-03-16T09:13:39.130410+0000","last_fullsized":"2024-03-16T09:13:39.130410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:09:23.866570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.109281+0000","last_change":"2024-03-16T09:13:36.109585+0000","last_active":"2024-03-16T09:13:37.109281+0000","last_peered":"2024-03-16T09:13:37.109281+0000","last_clean":"2024-03-16T09:13:37.109281+0000","last_became_active":"2024-03-16T09:13:36.109267+0000","last_became_peered":"2024-03-16T09:13:36.109267+0000","last_unstale":"2024-03-16T09:13:37.109281+0000","last_undegraded":"2024-03-16T09:13:37.109281+0000","last_fullsized":"2024-03-16T09:13:37.109281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:26:55.700791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.129224+0000","last_change":"2024-03-16T09:13:37.136465+0000","last_active":"2024-03-16T09:13:39.129224+0000","last_peered":"2024-03-16T09:13:39.129224+0000","last_clean":"2024-03-16T09:13:39.129224+0000","last_became_active":"2024-03-16T09:13:37.135933+0000","last_became_peered":"2024-03-16T09:13:37.135933+0000","last_unstale":"2024-03-16T09:13:39.129224+0000","last_undegraded":"2024-03-16T09:13:39.129224+0000","last_fullsized":"2024-03-16T09:13:39.129224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:08:28.490533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121206+0000","last_change":"2024-03-16T09:13:36.119428+0000","last_active":"2024-03-16T09:13:38.121206+0000","last_peered":"2024-03-16T09:13:38.121206+0000","last_clean":"2024-03-16T09:13:38.121206+0000","last_became_active":"2024-03-16T09:13:36.119083+0000","last_became_peered":"2024-03-16T09:13:36.119083+0000","last_unstale":"2024-03-16T09:13:38.121206+0000","last_undegraded":"2024-03-16T09:13:38.121206+0000","last_fullsized":"2024-03-16T09:13:38.121206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:58:56.065151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130253+0000","last_change":"2024-03-16T09:13:37.118082+0000","last_active":"2024-03-16T09:13:39.130253+0000","last_peered":"2024-03-16T09:13:39.130253+0000","last_clean":"2024-03-16T09:13:39.130253+0000","last_became_active":"2024-03-16T09:13:37.117671+0000","last_became_peered":"2024-03-16T09:13:37.117671+0000","last_unstale":"2024-03-16T09:13:39.130253+0000","last_undegraded":"2024-03-16T09:13:39.130253+0000","last_fullsized":"2024-03-16T09:13:39.130253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:14:05.352191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121630+0000","last_change":"2024-03-16T09:13:36.121077+0000","last_active":"2024-03-16T09:13:38.121630+0000","last_peered":"2024-03-16T09:13:38.121630+0000","last_clean":"2024-03-16T09:13:38.121630+0000","last_became_active":"2024-03-16T09:13:36.120723+0000","last_became_peered":"2024-03-16T09:13:36.120723+0000","last_unstale":"2024-03-16T09:13:38.121630+0000","last_undegraded":"2024-03-16T09:13:38.121630+0000","last_fullsized":"2024-03-16T09:13:38.121630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:58:14.746215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.117304+0000","last_change":"2024-03-16T09:13:37.116725+0000","last_active":"2024-03-16T09:13:37.117304+0000","last_peered":"2024-03-16T09:13:37.117304+0000","last_clean":"2024-03-16T09:13:37.117304+0000","last_became_active":"2024-03-16T09:13:37.116406+0000","last_became_peered":"2024-03-16T09:13:37.116406+0000","last_unstale":"2024-03-16T09:13:37.117304+0000","last_undegraded":"2024-03-16T09:13:37.117304+0000","last_fullsized":"2024-03-16T09:13:37.117304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:04:07.808008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.119179+0000","last_change":"2024-03-16T09:13:36.110568+0000","last_active":"2024-03-16T09:13:38.119179+0000","last_peered":"2024-03-16T09:13:38.119179+0000","last_clean":"2024-03-16T09:13:38.119179+0000","last_became_active":"2024-03-16T09:13:36.110246+0000","last_became_peered":"2024-03-16T09:13:36.110246+0000","last_unstale":"2024-03-16T09:13:38.119179+0000","last_undegraded":"2024-03-16T09:13:38.119179+0000","last_fullsized":"2024-03-16T09:13:38.119179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:36:50.011941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.247198+0000","last_change":"2024-03-16T09:13:37.122296+0000","last_active":"2024-03-16T09:13:38.247198+0000","last_peered":"2024-03-16T09:13:38.247198+0000","last_clean":"2024-03-16T09:13:38.247198+0000","last_became_active":"2024-03-16T09:13:37.121968+0000","last_became_peered":"2024-03-16T09:13:37.121968+0000","last_unstale":"2024-03-16T09:13:38.247198+0000","last_undegraded":"2024-03-16T09:13:38.247198+0000","last_fullsized":"2024-03-16T09:13:38.247198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:10:43.449397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.129808+0000","last_change":"2024-03-16T09:13:36.112262+0000","last_active":"2024-03-16T09:13:39.129808+0000","last_peered":"2024-03-16T09:13:39.129808+0000","last_clean":"2024-03-16T09:13:39.129808+0000","last_became_active":"2024-03-16T09:13:36.111916+0000","last_became_peered":"2024-03-16T09:13:36.111916+0000","last_unstale":"2024-03-16T09:13:39.129808+0000","last_undegraded":"2024-03-16T09:13:39.129808+0000","last_fullsized":"2024-03-16T09:13:39.129808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:27:41.163508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.247117+0000","last_change":"2024-03-16T09:13:37.119814+0000","last_active":"2024-03-16T09:13:38.247117+0000","last_peered":"2024-03-16T09:13:38.247117+0000","last_clean":"2024-03-16T09:13:38.247117+0000","last_became_active":"2024-03-16T09:13:37.119454+0000","last_became_peered":"2024-03-16T09:13:37.119454+0000","last_unstale":"2024-03-16T09:13:38.247117+0000","last_undegraded":"2024-03-16T09:13:38.247117+0000","last_fullsized":"2024-03-16T09:13:38.247117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:57:25.988044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130491+0000","last_change":"2024-03-16T09:13:37.116728+0000","last_active":"2024-03-16T09:13:39.130491+0000","last_peered":"2024-03-16T09:13:39.130491+0000","last_clean":"2024-03-16T09:13:39.130491+0000","last_became_active":"2024-03-16T09:13:37.116406+0000","last_became_peered":"2024-03-16T09:13:37.116406+0000","last_unstale":"2024-03-16T09:13:39.130491+0000","last_undegraded":"2024-03-16T09:13:39.130491+0000","last_fullsized":"2024-03-16T09:13:39.130491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:38.134284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.099459+0000","last_change":"2024-03-16T09:13:36.111300+0000","last_active":"2024-03-16T09:13:37.099459+0000","last_peered":"2024-03-16T09:13:37.099459+0000","last_clean":"2024-03-16T09:13:37.099459+0000","last_became_active":"2024-03-16T09:13:36.110966+0000","last_became_peered":"2024-03-16T09:13:36.110966+0000","last_unstale":"2024-03-16T09:13:37.099459+0000","last_undegraded":"2024-03-16T09:13:37.099459+0000","last_fullsized":"2024-03-16T09:13:37.099459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:18:33.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.120980+0000","last_change":"2024-03-16T09:13:37.125791+0000","last_active":"2024-03-16T09:13:38.120980+0000","last_peered":"2024-03-16T09:13:38.120980+0000","last_clean":"2024-03-16T09:13:38.120980+0000","last_became_active":"2024-03-16T09:13:37.125448+0000","last_became_peered":"2024-03-16T09:13:37.125448+0000","last_unstale":"2024-03-16T09:13:38.120980+0000","last_undegraded":"2024-03-16T09:13:38.120980+0000","last_fullsized":"2024-03-16T09:13:38.120980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:51:30.767750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.112831+0000","last_change":"2024-03-16T09:13:36.107462+0000","last_active":"2024-03-16T09:13:37.112831+0000","last_peered":"2024-03-16T09:13:37.112831+0000","last_clean":"2024-03-16T09:13:37.112831+0000","last_became_active":"2024-03-16T09:13:36.107115+0000","last_became_peered":"2024-03-16T09:13:36.107115+0000","last_unstale":"2024-03-16T09:13:37.112831+0000","last_undegraded":"2024-03-16T09:13:37.112831+0000","last_fullsized":"2024-03-16T09:13:37.112831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:37.796719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":337,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.129120+0000","last_change":"2024-03-16T09:12:24.331159+0000","last_active":"2024-03-16T09:13:39.129120+0000","last_peered":"2024-03-16T09:13:39.129120+0000","last_clean":"2024-03-16T09:13:39.129120+0000","last_became_active":"2024-03-16T09:12:24.330554+0000","last_became_peered":"2024-03-16T09:12:24.330554+0000","last_unstale":"2024-03-16T09:13:39.129120+0000","last_undegraded":"2024-03-16T09:13:39.129120+0000","last_fullsized":"2024-03-16T09:13:39.129120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:08:57.697012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:08:57.697012+0000","last_clean_scrub_stamp":"2024-03-16T09:08:57.697012+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:43:02.986865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.115533+0000","last_change":"2024-03-16T09:13:37.114299+0000","last_active":"2024-03-16T09:13:37.115533+0000","last_peered":"2024-03-16T09:13:37.115533+0000","last_clean":"2024-03-16T09:13:37.115533+0000","last_became_active":"2024-03-16T09:13:37.113981+0000","last_became_peered":"2024-03-16T09:13:37.113981+0000","last_unstale":"2024-03-16T09:13:37.115533+0000","last_undegraded":"2024-03-16T09:13:37.115533+0000","last_fullsized":"2024-03-16T09:13:37.115533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:57.533537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.135006+0000","last_change":"2024-03-16T09:13:36.113962+0000","last_active":"2024-03-16T09:13:39.135006+0000","last_peered":"2024-03-16T09:13:39.135006+0000","last_clean":"2024-03-16T09:13:39.135006+0000","last_became_active":"2024-03-16T09:13:36.113616+0000","last_became_peered":"2024-03-16T09:13:36.113616+0000","last_unstale":"2024-03-16T09:13:39.135006+0000","last_undegraded":"2024-03-16T09:13:39.135006+0000","last_fullsized":"2024-03-16T09:13:39.135006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:15:15.004360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121506+0000","last_change":"2024-03-16T09:13:36.121946+0000","last_active":"2024-03-16T09:13:38.121506+0000","last_peered":"2024-03-16T09:13:38.121506+0000","last_clean":"2024-03-16T09:13:38.121506+0000","last_became_active":"2024-03-16T09:13:36.121209+0000","last_became_peered":"2024-03-16T09:13:36.121209+0000","last_unstale":"2024-03-16T09:13:38.121506+0000","last_undegraded":"2024-03-16T09:13:38.121506+0000","last_fullsized":"2024-03-16T09:13:38.121506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:09:51.139468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.129423+0000","last_change":"2024-03-16T09:13:37.128931+0000","last_active":"2024-03-16T09:13:37.129423+0000","last_peered":"2024-03-16T09:13:37.129423+0000","last_clean":"2024-03-16T09:13:37.129423+0000","last_became_active":"2024-03-16T09:13:37.128515+0000","last_became_peered":"2024-03-16T09:13:37.128515+0000","last_unstale":"2024-03-16T09:13:37.129423+0000","last_undegraded":"2024-03-16T09:13:37.129423+0000","last_fullsized":"2024-03-16T09:13:37.129423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:39:45.696095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:40.138392+0000","last_change":"2024-03-16T09:13:36.118831+0000","last_active":"2024-03-16T09:13:40.138392+0000","last_peered":"2024-03-16T09:13:40.138392+0000","last_clean":"2024-03-16T09:13:40.138392+0000","last_became_active":"2024-03-16T09:13:36.118489+0000","last_became_peered":"2024-03-16T09:13:36.118489+0000","last_unstale":"2024-03-16T09:13:40.138392+0000","last_undegraded":"2024-03-16T09:13:40.138392+0000","last_fullsized":"2024-03-16T09:13:40.138392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:55:34.089451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121214+0000","last_change":"2024-03-16T09:13:37.120581+0000","last_active":"2024-03-16T09:13:38.121214+0000","last_peered":"2024-03-16T09:13:38.121214+0000","last_clean":"2024-03-16T09:13:38.121214+0000","last_became_active":"2024-03-16T09:13:37.120266+0000","last_became_peered":"2024-03-16T09:13:37.120266+0000","last_unstale":"2024-03-16T09:13:38.121214+0000","last_undegraded":"2024-03-16T09:13:38.121214+0000","last_fullsized":"2024-03-16T09:13:38.121214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:16:10.784229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.132387+0000","last_change":"2024-03-16T09:13:36.107125+0000","last_active":"2024-03-16T09:13:37.132387+0000","last_peered":"2024-03-16T09:13:37.132387+0000","last_clean":"2024-03-16T09:13:37.132387+0000","last_became_active":"2024-03-16T09:13:36.106795+0000","last_became_peered":"2024-03-16T09:13:36.106795+0000","last_unstale":"2024-03-16T09:13:37.132387+0000","last_undegraded":"2024-03-16T09:13:37.132387+0000","last_fullsized":"2024-03-16T09:13:37.132387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:29:10.130565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130456+0000","last_change":"2024-03-16T09:13:37.123013+0000","last_active":"2024-03-16T09:13:39.130456+0000","last_peered":"2024-03-16T09:13:39.130456+0000","last_clean":"2024-03-16T09:13:39.130456+0000","last_became_active":"2024-03-16T09:13:37.122642+0000","last_became_peered":"2024-03-16T09:13:37.122642+0000","last_unstale":"2024-03-16T09:13:39.130456+0000","last_undegraded":"2024-03-16T09:13:39.130456+0000","last_fullsized":"2024-03-16T09:13:39.130456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:35:05.297337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.108638+0000","last_change":"2024-03-16T09:13:36.110868+0000","last_active":"2024-03-16T09:13:38.108638+0000","last_peered":"2024-03-16T09:13:38.108638+0000","last_clean":"2024-03-16T09:13:38.108638+0000","last_became_active":"2024-03-16T09:13:36.110536+0000","last_became_peered":"2024-03-16T09:13:36.110536+0000","last_unstale":"2024-03-16T09:13:38.108638+0000","last_undegraded":"2024-03-16T09:13:38.108638+0000","last_fullsized":"2024-03-16T09:13:38.108638+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:07:06.829458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121389+0000","last_change":"2024-03-16T09:13:37.116579+0000","last_active":"2024-03-16T09:13:38.121389+0000","last_peered":"2024-03-16T09:13:38.121389+0000","last_clean":"2024-03-16T09:13:38.121389+0000","last_became_active":"2024-03-16T09:13:37.116226+0000","last_became_peered":"2024-03-16T09:13:37.116226+0000","last_unstale":"2024-03-16T09:13:38.121389+0000","last_undegraded":"2024-03-16T09:13:38.121389+0000","last_fullsized":"2024-03-16T09:13:38.121389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:08:39.015162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.131174+0000","last_change":"2024-03-16T09:13:36.107881+0000","last_active":"2024-03-16T09:13:39.131174+0000","last_peered":"2024-03-16T09:13:39.131174+0000","last_clean":"2024-03-16T09:13:39.131174+0000","last_became_active":"2024-03-16T09:13:36.107538+0000","last_became_peered":"2024-03-16T09:13:36.107538+0000","last_unstale":"2024-03-16T09:13:39.131174+0000","last_undegraded":"2024-03-16T09:13:39.131174+0000","last_fullsized":"2024-03-16T09:13:39.131174+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:49:41.932089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.124842+0000","last_change":"2024-03-16T09:13:37.118580+0000","last_active":"2024-03-16T09:13:37.124842+0000","last_peered":"2024-03-16T09:13:37.124842+0000","last_clean":"2024-03-16T09:13:37.124842+0000","last_became_active":"2024-03-16T09:13:37.118166+0000","last_became_peered":"2024-03-16T09:13:37.118166+0000","last_unstale":"2024-03-16T09:13:37.124842+0000","last_undegraded":"2024-03-16T09:13:37.124842+0000","last_fullsized":"2024-03-16T09:13:37.124842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:15:54.370764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.108787+0000","last_change":"2024-03-16T09:13:36.122307+0000","last_active":"2024-03-16T09:13:38.108787+0000","last_peered":"2024-03-16T09:13:38.108787+0000","last_clean":"2024-03-16T09:13:38.108787+0000","last_became_active":"2024-03-16T09:13:36.121961+0000","last_became_peered":"2024-03-16T09:13:36.121961+0000","last_unstale":"2024-03-16T09:13:38.108787+0000","last_undegraded":"2024-03-16T09:13:38.108787+0000","last_fullsized":"2024-03-16T09:13:38.108787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:50:03.482300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121121+0000","last_change":"2024-03-16T09:13:37.131918+0000","last_active":"2024-03-16T09:13:38.121121+0000","last_peered":"2024-03-16T09:13:38.121121+0000","last_clean":"2024-03-16T09:13:38.121121+0000","last_became_active":"2024-03-16T09:13:37.131560+0000","last_became_peered":"2024-03-16T09:13:37.131560+0000","last_unstale":"2024-03-16T09:13:38.121121+0000","last_undegraded":"2024-03-16T09:13:38.121121+0000","last_fullsized":"2024-03-16T09:13:38.121121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:57.315125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121288+0000","last_change":"2024-03-16T09:13:36.120192+0000","last_active":"2024-03-16T09:13:38.121288+0000","last_peered":"2024-03-16T09:13:38.121288+0000","last_clean":"2024-03-16T09:13:38.121288+0000","last_became_active":"2024-03-16T09:13:36.119063+0000","last_became_peered":"2024-03-16T09:13:36.119063+0000","last_unstale":"2024-03-16T09:13:38.121288+0000","last_undegraded":"2024-03-16T09:13:38.121288+0000","last_fullsized":"2024-03-16T09:13:38.121288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:18:56.616624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130027+0000","last_change":"2024-03-16T09:13:37.129597+0000","last_active":"2024-03-16T09:13:39.130027+0000","last_peered":"2024-03-16T09:13:39.130027+0000","last_clean":"2024-03-16T09:13:39.130027+0000","last_became_active":"2024-03-16T09:13:37.129248+0000","last_became_peered":"2024-03-16T09:13:37.129248+0000","last_unstale":"2024-03-16T09:13:39.130027+0000","last_undegraded":"2024-03-16T09:13:39.130027+0000","last_fullsized":"2024-03-16T09:13:39.130027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:41:11.118192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:40.139671+0000","last_change":"2024-03-16T09:13:36.116652+0000","last_active":"2024-03-16T09:13:40.139671+0000","last_peered":"2024-03-16T09:13:40.139671+0000","last_clean":"2024-03-16T09:13:40.139671+0000","last_became_active":"2024-03-16T09:13:36.116324+0000","last_became_peered":"2024-03-16T09:13:36.116324+0000","last_unstale":"2024-03-16T09:13:40.139671+0000","last_undegraded":"2024-03-16T09:13:40.139671+0000","last_fullsized":"2024-03-16T09:13:40.139671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:45:10.185614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130795+0000","last_change":"2024-03-16T09:13:37.120317+0000","last_active":"2024-03-16T09:13:39.130795+0000","last_peered":"2024-03-16T09:13:39.130795+0000","last_clean":"2024-03-16T09:13:39.130795+0000","last_became_active":"2024-03-16T09:13:37.119974+0000","last_became_peered":"2024-03-16T09:13:37.119974+0000","last_unstale":"2024-03-16T09:13:39.130795+0000","last_undegraded":"2024-03-16T09:13:39.130795+0000","last_fullsized":"2024-03-16T09:13:39.130795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:25:34.774377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130531+0000","last_change":"2024-03-16T09:13:36.112395+0000","last_active":"2024-03-16T09:13:39.130531+0000","last_peered":"2024-03-16T09:13:39.130531+0000","last_clean":"2024-03-16T09:13:39.130531+0000","last_became_active":"2024-03-16T09:13:36.112036+0000","last_became_peered":"2024-03-16T09:13:36.112036+0000","last_unstale":"2024-03-16T09:13:39.130531+0000","last_undegraded":"2024-03-16T09:13:39.130531+0000","last_fullsized":"2024-03-16T09:13:39.130531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:41:02.895820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.131112+0000","last_change":"2024-03-16T09:13:37.133845+0000","last_active":"2024-03-16T09:13:39.131112+0000","last_peered":"2024-03-16T09:13:39.131112+0000","last_clean":"2024-03-16T09:13:39.131112+0000","last_became_active":"2024-03-16T09:13:37.132833+0000","last_became_peered":"2024-03-16T09:13:37.132833+0000","last_unstale":"2024-03-16T09:13:39.131112+0000","last_undegraded":"2024-03-16T09:13:39.131112+0000","last_fullsized":"2024-03-16T09:13:39.131112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:12:18.049210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.128594+0000","last_change":"2024-03-16T09:13:36.114830+0000","last_active":"2024-03-16T09:13:37.128594+0000","last_peered":"2024-03-16T09:13:37.128594+0000","last_clean":"2024-03-16T09:13:37.128594+0000","last_became_active":"2024-03-16T09:13:36.113898+0000","last_became_peered":"2024-03-16T09:13:36.113898+0000","last_unstale":"2024-03-16T09:13:37.128594+0000","last_undegraded":"2024-03-16T09:13:37.128594+0000","last_fullsized":"2024-03-16T09:13:37.128594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:27:38.910819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.131273+0000","last_change":"2024-03-16T09:13:37.115991+0000","last_active":"2024-03-16T09:13:39.131273+0000","last_peered":"2024-03-16T09:13:39.131273+0000","last_clean":"2024-03-16T09:13:39.131273+0000","last_became_active":"2024-03-16T09:13:37.115671+0000","last_became_peered":"2024-03-16T09:13:37.115671+0000","last_unstale":"2024-03-16T09:13:39.131273+0000","last_undegraded":"2024-03-16T09:13:39.131273+0000","last_fullsized":"2024-03-16T09:13:39.131273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:05:59.703675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.115823+0000","last_change":"2024-03-16T09:13:37.115122+0000","last_active":"2024-03-16T09:13:37.115823+0000","last_peered":"2024-03-16T09:13:37.115823+0000","last_clean":"2024-03-16T09:13:37.115823+0000","last_became_active":"2024-03-16T09:13:37.114779+0000","last_became_peered":"2024-03-16T09:13:37.114779+0000","last_unstale":"2024-03-16T09:13:37.115823+0000","last_undegraded":"2024-03-16T09:13:37.115823+0000","last_fullsized":"2024-03-16T09:13:37.115823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:19:09.501237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.108934+0000","last_change":"2024-03-16T09:13:37.116542+0000","last_active":"2024-03-16T09:13:38.108934+0000","last_peered":"2024-03-16T09:13:38.108934+0000","last_clean":"2024-03-16T09:13:38.108934+0000","last_became_active":"2024-03-16T09:13:37.116224+0000","last_became_peered":"2024-03-16T09:13:37.116224+0000","last_unstale":"2024-03-16T09:13:38.108934+0000","last_undegraded":"2024-03-16T09:13:38.108934+0000","last_fullsized":"2024-03-16T09:13:38.108934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:32:40.106979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.108831+0000","last_change":"2024-03-16T09:13:37.128750+0000","last_active":"2024-03-16T09:13:38.108831+0000","last_peered":"2024-03-16T09:13:38.108831+0000","last_clean":"2024-03-16T09:13:38.108831+0000","last_became_active":"2024-03-16T09:13:37.128411+0000","last_became_peered":"2024-03-16T09:13:37.128411+0000","last_unstale":"2024-03-16T09:13:38.108831+0000","last_undegraded":"2024-03-16T09:13:38.108831+0000","last_fullsized":"2024-03-16T09:13:38.108831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:01:02.119578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130588+0000","last_change":"2024-03-16T09:13:37.124024+0000","last_active":"2024-03-16T09:13:39.130588+0000","last_peered":"2024-03-16T09:13:39.130588+0000","last_clean":"2024-03-16T09:13:39.130588+0000","last_became_active":"2024-03-16T09:13:37.123721+0000","last_became_peered":"2024-03-16T09:13:37.123721+0000","last_unstale":"2024-03-16T09:13:39.130588+0000","last_undegraded":"2024-03-16T09:13:39.130588+0000","last_fullsized":"2024-03-16T09:13:39.130588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:29:28.079163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.116966+0000","last_change":"2024-03-16T09:13:37.116603+0000","last_active":"2024-03-16T09:13:37.116966+0000","last_peered":"2024-03-16T09:13:37.116966+0000","last_clean":"2024-03-16T09:13:37.116966+0000","last_became_active":"2024-03-16T09:13:37.116279+0000","last_became_peered":"2024-03-16T09:13:37.116279+0000","last_unstale":"2024-03-16T09:13:37.116966+0000","last_undegraded":"2024-03-16T09:13:37.116966+0000","last_fullsized":"2024-03-16T09:13:37.116966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:47:04.061172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.117780+0000","last_change":"2024-03-16T09:13:37.116863+0000","last_active":"2024-03-16T09:13:37.117780+0000","last_peered":"2024-03-16T09:13:37.117780+0000","last_clean":"2024-03-16T09:13:37.117780+0000","last_became_active":"2024-03-16T09:13:37.116543+0000","last_became_peered":"2024-03-16T09:13:37.116543+0000","last_unstale":"2024-03-16T09:13:37.117780+0000","last_undegraded":"2024-03-16T09:13:37.117780+0000","last_fullsized":"2024-03-16T09:13:37.117780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:51:28.615628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.109259+0000","last_change":"2024-03-16T09:13:37.120325+0000","last_active":"2024-03-16T09:13:38.109259+0000","last_peered":"2024-03-16T09:13:38.109259+0000","last_clean":"2024-03-16T09:13:38.109259+0000","last_became_active":"2024-03-16T09:13:37.119985+0000","last_became_peered":"2024-03-16T09:13:37.119985+0000","last_unstale":"2024-03-16T09:13:38.109259+0000","last_undegraded":"2024-03-16T09:13:38.109259+0000","last_fullsized":"2024-03-16T09:13:38.109259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:35.419225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.121115+0000","last_change":"2024-03-16T09:13:37.119604+0000","last_active":"2024-03-16T09:13:38.121115+0000","last_peered":"2024-03-16T09:13:38.121115+0000","last_clean":"2024-03-16T09:13:38.121115+0000","last_became_active":"2024-03-16T09:13:37.119019+0000","last_became_peered":"2024-03-16T09:13:37.119019+0000","last_unstale":"2024-03-16T09:13:38.121115+0000","last_undegraded":"2024-03-16T09:13:38.121115+0000","last_fullsized":"2024-03-16T09:13:38.121115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:42:57.902370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:39.130290+0000","last_change":"2024-03-16T09:13:37.123845+0000","last_active":"2024-03-16T09:13:39.130290+0000","last_peered":"2024-03-16T09:13:39.130290+0000","last_clean":"2024-03-16T09:13:39.130290+0000","last_became_active":"2024-03-16T09:13:37.121753+0000","last_became_peered":"2024-03-16T09:13:37.121753+0000","last_unstale":"2024-03-16T09:13:39.130290+0000","last_undegraded":"2024-03-16T09:13:39.130290+0000","last_fullsized":"2024-03-16T09:13:39.130290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:23:01.289378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.120704+0000","last_change":"2024-03-16T09:13:37.118182+0000","last_active":"2024-03-16T09:13:38.120704+0000","last_peered":"2024-03-16T09:13:38.120704+0000","last_clean":"2024-03-16T09:13:38.120704+0000","last_became_active":"2024-03-16T09:13:37.117820+0000","last_became_peered":"2024-03-16T09:13:37.117820+0000","last_unstale":"2024-03-16T09:13:38.120704+0000","last_undegraded":"2024-03-16T09:13:38.120704+0000","last_fullsized":"2024-03-16T09:13:38.120704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:27:31.393266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-16T09:13:38.108990+0000","last_change":"2024-03-16T09:13:37.121348+0000","last_active":"2024-03-16T09:13:38.108990+0000","last_peered":"2024-03-16T09:13:38.108990+0000","last_clean":"2024-03-16T09:13:38.108990+0000","last_became_active":"2024-03-16T09:13:37.120826+0000","last_became_peered":"2024-03-16T09:13:37.120826+0000","last_unstale":"2024-03-16T09:13:38.108990+0000","last_undegraded":"2024-03-16T09:13:38.108990+0000","last_fullsized":"2024-03-16T09:13:38.108990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:36:33.456230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.133210+0000","last_change":"2024-03-16T09:13:37.131889+0000","last_active":"2024-03-16T09:13:37.133210+0000","last_peered":"2024-03-16T09:13:37.133210+0000","last_clean":"2024-03-16T09:13:37.133210+0000","last_became_active":"2024-03-16T09:13:37.131564+0000","last_became_peered":"2024-03-16T09:13:37.131564+0000","last_unstale":"2024-03-16T09:13:37.133210+0000","last_undegraded":"2024-03-16T09:13:37.133210+0000","last_fullsized":"2024-03-16T09:13:37.133210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:53:41.448241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-03-16T09:13:37.125095+0000","last_change":"2024-03-16T09:13:37.120041+0000","last_active":"2024-03-16T09:13:37.125095+0000","last_peered":"2024-03-16T09:13:37.125095+0000","last_clean":"2024-03-16T09:13:37.125095+0000","last_became_active":"2024-03-16T09:13:37.118989+0000","last_became_peered":"2024-03-16T09:13:37.118989+0000","last_unstale":"2024-03-16T09:13:37.125095+0000","last_undegraded":"2024-03-16T09:13:37.125095+0000","last_fullsized":"2024-03-16T09:13:37.125095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:15:14.529994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280795,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4376,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":88,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":212992,"data_stored":32149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383693,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":429951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645330,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":1031457,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":439339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070492,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":430555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332130,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":954368,"data_stored":1020901,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495654,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":429951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":905216,"data_stored":430041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":441483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":438805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":429973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757292,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":1030365,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":22245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T09:13:44.366 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-16T09:13:44.366 INFO:tasks.cephadm:Setting up client nodes... 2024-03-16T09:13:44.366 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-16T09:13:44.637 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:13:44.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:44 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/230924235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:44.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:44 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3251299522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:44.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:44 smithi194 ceph-mon[36729]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:13:44.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:44 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/230924235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:44.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:44 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3251299522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:44.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:44 smithi159 ceph-mon[36739]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:13:44.758 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:44 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/230924235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:13:44.759 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:44 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3251299522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:44.759 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:44 smithi112 ceph-mon[32307]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:13:45.637 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:45 smithi159 ceph-mon[36739]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:13:45.637 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:45 smithi159 ceph-mon[36739]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-16T09:13:45.637 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:45 smithi159 ceph-mon[36739]: pgmap v292: 97 pgs: 97 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 4.9 KiB/s wr, 19 op/s 2024-03-16T09:13:45.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:45 smithi194 ceph-mon[36729]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:13:45.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:45 smithi194 ceph-mon[36729]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-16T09:13:45.652 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:45 smithi194 ceph-mon[36729]: pgmap v292: 97 pgs: 97 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 4.9 KiB/s wr, 19 op/s 2024-03-16T09:13:45.777 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:45 smithi112 ceph-mon[32307]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:13:45.777 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:45 smithi112 ceph-mon[32307]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-16T09:13:45.777 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:45 smithi112 ceph-mon[32307]: pgmap v292: 97 pgs: 97 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 4.9 KiB/s wr, 19 op/s 2024-03-16T09:13:46.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.372+0000 7f018ba8e700 1 -- 172.21.15.112:0/2176896376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0184102dc0 msgr2=0x7f0184109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:46.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.372+0000 7f018ba8e700 1 --2- 172.21.15.112:0/2176896376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0184102dc0 0x7f0184109670 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f017c00b0d0 tx=0x7f017c00b3e0 comp rx=0 tx=0).stop 2024-03-16T09:13:46.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.372+0000 7f018ba8e700 1 -- 172.21.15.112:0/2176896376 shutdown_connections 2024-03-16T09:13:46.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.372+0000 7f018ba8e700 1 --2- 172.21.15.112:0/2176896376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0184102dc0 0x7f0184109670 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:46.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.372+0000 7f018ba8e700 1 --2- 172.21.15.112:0/2176896376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0184101220 0x7f0184101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:46.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.372+0000 7f018ba8e700 1 --2- 172.21.15.112:0/2176896376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0184102420 0x7f0184102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:46.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.372+0000 7f018ba8e700 1 -- 172.21.15.112:0/2176896376 >> 172.21.15.112:0/2176896376 conn(0x7f01840fc7c0 msgr2=0x7f01840fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:46.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.372+0000 7f018ba8e700 1 -- 172.21.15.112:0/2176896376 shutdown_connections 2024-03-16T09:13:46.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.372+0000 7f018ba8e700 1 -- 172.21.15.112:0/2176896376 wait complete. 2024-03-16T09:13:46.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.373+0000 7f018ba8e700 1 Processor -- start 2024-03-16T09:13:46.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.373+0000 7f018ba8e700 1 -- start start 2024-03-16T09:13:46.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.373+0000 7f018ba8e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0184101220 0x7f01841051b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:46.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.373+0000 7f018ba8e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0184102420 0x7f0184103800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:46.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.373+0000 7f018ba8e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0184102dc0 0x7f0184103d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:46.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.373+0000 7f018ba8e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f018410cfa0 con 0x7f0184101220 2024-03-16T09:13:46.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.373+0000 7f018ba8e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f018410ce20 con 0x7f0184102420 2024-03-16T09:13:46.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.373+0000 7f018ba8e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f018410d120 con 0x7f0184102dc0 2024-03-16T09:13:46.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.373+0000 7f018982a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0184101220 0x7f01841051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:46.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.373+0000 7f018982a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0184101220 0x7f01841051b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59248/0 (socket says 172.21.15.112:59248) 2024-03-16T09:13:46.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.373+0000 7f018982a700 1 -- 172.21.15.112:0/821571917 learned_addr learned my addr 172.21.15.112:0/821571917 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:46.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.373+0000 7f018a02b700 1 --2- 172.21.15.112:0/821571917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0184102dc0 0x7f0184103d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:46.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.373+0000 7f0189029700 1 --2- 172.21.15.112:0/821571917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0184102420 0x7f0184103800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.374+0000 7f018982a700 1 -- 172.21.15.112:0/821571917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0184102dc0 msgr2=0x7f0184103d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.374+0000 7f018982a700 1 --2- 172.21.15.112:0/821571917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0184102dc0 0x7f0184103d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.374+0000 7f018982a700 1 -- 172.21.15.112:0/821571917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0184102420 msgr2=0x7f0184103800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.374+0000 7f018982a700 1 --2- 172.21.15.112:0/821571917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0184102420 0x7f0184103800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.374+0000 7f018982a700 1 -- 172.21.15.112:0/821571917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01841045f0 con 0x7f0184101220 2024-03-16T09:13:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.374+0000 7f018a02b700 1 --2- 172.21.15.112:0/821571917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0184102dc0 0x7f0184103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:13:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.374+0000 7f018982a700 1 --2- 172.21.15.112:0/821571917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0184101220 0x7f01841051b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f018000e950 tx=0x7f018000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.374+0000 7f017affd700 1 -- 172.21.15.112:0/821571917 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180004230 con 0x7f0184101220 2024-03-16T09:13:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.374+0000 7f018ba8e700 1 -- 172.21.15.112:0/821571917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01841a2670 con 0x7f0184101220 2024-03-16T09:13:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.374+0000 7f017affd700 1 -- 172.21.15.112:0/821571917 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f01800043d0 con 0x7f0184101220 2024-03-16T09:13:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.374+0000 7f017affd700 1 -- 172.21.15.112:0/821571917 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0180010430 con 0x7f0184101220 2024-03-16T09:13:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.374+0000 7f018ba8e700 1 -- 172.21.15.112:0/821571917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01841a2b80 con 0x7f0184101220 2024-03-16T09:13:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.375+0000 7f018ba8e700 1 -- 172.21.15.112:0/821571917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01840507e0 con 0x7f0184101220 2024-03-16T09:13:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.376+0000 7f017affd700 1 -- 172.21.15.112:0/821571917 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f018000c7e0 con 0x7f0184101220 2024-03-16T09:13:46.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.376+0000 7f017affd700 1 --2- 172.21.15.112:0/821571917 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f01700a29d0 0x7f01700a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:46.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.376+0000 7f0189029700 1 --2- 172.21.15.112:0/821571917 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f01700a29d0 0x7f01700a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:46.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.377+0000 7f017affd700 1 -- 172.21.15.112:0/821571917 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f01800cec60 con 0x7f0184101220 2024-03-16T09:13:46.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.377+0000 7f0189029700 1 --2- 172.21.15.112:0/821571917 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f01700a29d0 0x7f01700a4e90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0174005950 tx=0x7f017400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:46.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.381+0000 7f017affd700 1 -- 172.21.15.112:0/821571917 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f018009bd50 con 0x7f0184101220 2024-03-16T09:13:46.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.655+0000 7f018ba8e700 1 -- 172.21.15.112:0/821571917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f01841a2fd0 con 0x7f0184101220 2024-03-16T09:13:46.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.657+0000 7f017affd700 1 -- 172.21.15.112:0/821571917 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f018009b650 con 0x7f0184101220 2024-03-16T09:13:46.659 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-03-16T09:13:46.659 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDKYvVlGSoiJxAABZ8mFQOrYInlXRgovXP3Dw== 2024-03-16T09:13:46.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.659+0000 7f018ba8e700 1 -- 172.21.15.112:0/821571917 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f01700a29d0 msgr2=0x7f01700a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:46.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.659+0000 7f018ba8e700 1 --2- 172.21.15.112:0/821571917 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f01700a29d0 0x7f01700a4e90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0174005950 tx=0x7f017400b410 comp rx=0 tx=0).stop 2024-03-16T09:13:46.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.659+0000 7f018ba8e700 1 -- 172.21.15.112:0/821571917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0184101220 msgr2=0x7f01841051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:46.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.659+0000 7f018ba8e700 1 --2- 172.21.15.112:0/821571917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0184101220 0x7f01841051b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f018000e950 tx=0x7f018000ec60 comp rx=0 tx=0).stop 2024-03-16T09:13:46.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.659+0000 7f018ba8e700 1 -- 172.21.15.112:0/821571917 shutdown_connections 2024-03-16T09:13:46.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.659+0000 7f018ba8e700 1 --2- 172.21.15.112:0/821571917 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f01700a29d0 0x7f01700a4e90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:46.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.659+0000 7f018ba8e700 1 --2- 172.21.15.112:0/821571917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0184102dc0 0x7f0184103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:46.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.659+0000 7f018ba8e700 1 --2- 172.21.15.112:0/821571917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0184101220 0x7f01841051b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:46.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.659+0000 7f018ba8e700 1 --2- 172.21.15.112:0/821571917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0184102420 0x7f0184103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:46.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.659+0000 7f018ba8e700 1 -- 172.21.15.112:0/821571917 >> 172.21.15.112:0/821571917 conn(0x7f01840fc7c0 msgr2=0x7f01840fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:46.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.659+0000 7f018ba8e700 1 -- 172.21.15.112:0/821571917 shutdown_connections 2024-03-16T09:13:46.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:46.660+0000 7f018ba8e700 1 -- 172.21.15.112:0/821571917 wait complete. 2024-03-16T09:13:46.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:46 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/821571917' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T09:13:46.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:46 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/821571917' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-16T09:13:47.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:46 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/821571917' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T09:13:47.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:46 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/821571917' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-16T09:13:47.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:46 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/821571917' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-16T09:13:47.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:46 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/821571917' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-16T09:13:47.482 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T09:13:47.482 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-16T09:13:47.482 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-16T09:13:47.525 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-16T09:13:47.525 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-16T09:13:47.526 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph mgr dump --format=json 2024-03-16T09:13:47.806 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:13:47.986 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:47 smithi159 ceph-mon[36739]: pgmap v293: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.2 KiB/s wr, 19 op/s 2024-03-16T09:13:48.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:47 smithi112 ceph-mon[32307]: pgmap v293: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.2 KiB/s wr, 19 op/s 2024-03-16T09:13:48.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:47 smithi194 ceph-mon[36729]: pgmap v293: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.2 KiB/s wr, 19 op/s 2024-03-16T09:13:49.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.420+0000 7fc8c75a0700 1 -- 172.21.15.112:0/3363379734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8c0101e10 msgr2=0x7fc8c0105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:49.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.420+0000 7fc8c75a0700 1 --2- 172.21.15.112:0/3363379734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8c0101e10 0x7fc8c0105dd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc0099c0 tx=0x7fc8bc009cd0 comp rx=0 tx=0).stop 2024-03-16T09:13:49.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.420+0000 7fc8c75a0700 1 -- 172.21.15.112:0/3363379734 shutdown_connections 2024-03-16T09:13:49.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.420+0000 7fc8c75a0700 1 --2- 172.21.15.112:0/3363379734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0101460 0x7fc8c0101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:49.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.420+0000 7fc8c75a0700 1 --2- 172.21.15.112:0/3363379734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8c01064b0 0x7fc8c010e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:49.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.420+0000 7fc8c75a0700 1 --2- 172.21.15.112:0/3363379734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8c0101e10 0x7fc8c0105dd0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:49.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.420+0000 7fc8c75a0700 1 -- 172.21.15.112:0/3363379734 >> 172.21.15.112:0/3363379734 conn(0x7fc8c0077220 msgr2=0x7fc8c0077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:49.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.420+0000 7fc8c75a0700 1 -- 172.21.15.112:0/3363379734 shutdown_connections 2024-03-16T09:13:49.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.421+0000 7fc8c75a0700 1 -- 172.21.15.112:0/3363379734 wait complete. 2024-03-16T09:13:49.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.421+0000 7fc8c75a0700 1 Processor -- start 2024-03-16T09:13:49.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.421+0000 7fc8c75a0700 1 -- start start 2024-03-16T09:13:49.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.421+0000 7fc8c75a0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8c0101460 0x7fc8c0199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:49.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.421+0000 7fc8c75a0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8c0101e10 0x7fc8c0199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:49.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.421+0000 7fc8c75a0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c01064b0 0x7fc8c01a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:49.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.421+0000 7fc8c75a0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c0111560 con 0x7fc8c0101460 2024-03-16T09:13:49.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.421+0000 7fc8c75a0700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01113e0 con 0x7fc8c01064b0 2024-03-16T09:13:49.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.421+0000 7fc8c75a0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c01116e0 con 0x7fc8c0101e10 2024-03-16T09:13:49.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.421+0000 7fc8c533c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8c0101460 0x7fc8c0199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:49.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.421+0000 7fc8c533c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8c0101460 0x7fc8c0199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59270/0 (socket says 172.21.15.112:59270) 2024-03-16T09:13:49.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.421+0000 7fc8c533c700 1 -- 172.21.15.112:0/3549693524 learned_addr learned my addr 172.21.15.112:0/3549693524 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:49.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.421+0000 7fc8c4b3b700 1 --2- 172.21.15.112:0/3549693524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8c0101e10 0x7fc8c0199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:49.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.421+0000 7fc8c5b3d700 1 --2- 172.21.15.112:0/3549693524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c01064b0 0x7fc8c01a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:49.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.422+0000 7fc8c533c700 1 -- 172.21.15.112:0/3549693524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8c0101e10 msgr2=0x7fc8c0199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:49.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.422+0000 7fc8c533c700 1 --2- 172.21.15.112:0/3549693524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8c0101e10 0x7fc8c0199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:49.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.422+0000 7fc8c533c700 1 -- 172.21.15.112:0/3549693524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c01064b0 msgr2=0x7fc8c01a49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:49.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.422+0000 7fc8c533c700 1 --2- 172.21.15.112:0/3549693524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c01064b0 0x7fc8c01a49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:49.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.422+0000 7fc8c533c700 1 -- 172.21.15.112:0/3549693524 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c01a4f00 con 0x7fc8c0101460 2024-03-16T09:13:49.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.422+0000 7fc8c533c700 1 --2- 172.21.15.112:0/3549693524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8c0101460 0x7fc8c0199940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000e9a0 tx=0x7fc8b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:49.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.422+0000 7fc8b67fc700 1 -- 172.21.15.112:0/3549693524 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000ca10 con 0x7fc8c0101460 2024-03-16T09:13:49.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.422+0000 7fc8b67fc700 1 -- 172.21.15.112:0/3549693524 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc8b000cbb0 con 0x7fc8c0101460 2024-03-16T09:13:49.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.422+0000 7fc8c75a0700 1 -- 172.21.15.112:0/3549693524 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c01a5130 con 0x7fc8c0101460 2024-03-16T09:13:49.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.422+0000 7fc8b67fc700 1 -- 172.21.15.112:0/3549693524 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0018710 con 0x7fc8c0101460 2024-03-16T09:13:49.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.422+0000 7fc8c75a0700 1 -- 172.21.15.112:0/3549693524 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c01a5640 con 0x7fc8c0101460 2024-03-16T09:13:49.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.423+0000 7fc8c75a0700 1 -- 172.21.15.112:0/3549693524 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c01028b0 con 0x7fc8c0101460 2024-03-16T09:13:49.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.424+0000 7fc8b67fc700 1 -- 172.21.15.112:0/3549693524 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fc8b0010430 con 0x7fc8c0101460 2024-03-16T09:13:49.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.425+0000 7fc8b67fc700 1 --2- 172.21.15.112:0/3549693524 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc8ac0a2920 0x7fc8ac0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:49.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.425+0000 7fc8c4b3b700 1 --2- 172.21.15.112:0/3549693524 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc8ac0a2920 0x7fc8ac0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:49.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.425+0000 7fc8b67fc700 1 -- 172.21.15.112:0/3549693524 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fc8b00cf050 con 0x7fc8c0101460 2024-03-16T09:13:49.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.425+0000 7fc8c4b3b700 1 --2- 172.21.15.112:0/3549693524 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc8ac0a2920 0x7fc8ac0a4de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc000c00 tx=0x7fc8bc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:49.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.428+0000 7fc8b67fc700 1 -- 172.21.15.112:0/3549693524 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc8b009c140 con 0x7fc8c0101460 2024-03-16T09:13:49.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.680+0000 7fc8c75a0700 1 -- 172.21.15.112:0/3549693524 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fc8c002db70 con 0x7fc8c0101460 2024-03-16T09:13:49.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.685+0000 7fc8b67fc700 1 -- 172.21.15.112:0/3549693524 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+262669 (secure 0 0 0) 0x7fc8b009ba40 con 0x7fc8c0101460 2024-03-16T09:13:49.690 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:49.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.687+0000 7fc8c75a0700 1 -- 172.21.15.112:0/3549693524 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc8ac0a2920 msgr2=0x7fc8ac0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:49.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.687+0000 7fc8c75a0700 1 --2- 172.21.15.112:0/3549693524 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc8ac0a2920 0x7fc8ac0a4de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc000c00 tx=0x7fc8bc005fb0 comp rx=0 tx=0).stop 2024-03-16T09:13:49.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.687+0000 7fc8c75a0700 1 -- 172.21.15.112:0/3549693524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8c0101460 msgr2=0x7fc8c0199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:49.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.687+0000 7fc8c75a0700 1 --2- 172.21.15.112:0/3549693524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8c0101460 0x7fc8c0199940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000e9a0 tx=0x7fc8b000ecb0 comp rx=0 tx=0).stop 2024-03-16T09:13:49.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.687+0000 7fc8c75a0700 1 -- 172.21.15.112:0/3549693524 shutdown_connections 2024-03-16T09:13:49.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.687+0000 7fc8c75a0700 1 --2- 172.21.15.112:0/3549693524 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc8ac0a2920 0x7fc8ac0a4de0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:49.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.687+0000 7fc8c75a0700 1 --2- 172.21.15.112:0/3549693524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c01064b0 0x7fc8c01a49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:49.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.687+0000 7fc8c75a0700 1 --2- 172.21.15.112:0/3549693524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8c0101460 0x7fc8c0199940 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:49.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.687+0000 7fc8c75a0700 1 --2- 172.21.15.112:0/3549693524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8c0101e10 0x7fc8c0199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:49.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.687+0000 7fc8c75a0700 1 -- 172.21.15.112:0/3549693524 >> 172.21.15.112:0/3549693524 conn(0x7fc8c0077220 msgr2=0x7fc8c00ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:49.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.688+0000 7fc8c75a0700 1 -- 172.21.15.112:0/3549693524 shutdown_connections 2024-03-16T09:13:49.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:49.688+0000 7fc8c75a0700 1 -- 172.21.15.112:0/3549693524 wait complete. 2024-03-16T09:13:50.049 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:49 smithi112 ceph-mon[32307]: pgmap v294: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.8 KiB/s wr, 18 op/s 2024-03-16T09:13:50.049 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:49 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3549693524' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-16T09:13:50.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:49 smithi159 ceph-mon[36739]: pgmap v294: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.8 KiB/s wr, 18 op/s 2024-03-16T09:13:50.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:49 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3549693524' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-16T09:13:50.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:49 smithi194 ceph-mon[36729]: pgmap v294: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.8 KiB/s wr, 18 op/s 2024-03-16T09:13:50.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:49 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3549693524' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-16T09:13:50.404 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2628870229},{"type":"v1","addr":"172.21.15.112:6801","nonce":2628870229}]},"active_addr":"172.21.15.112:6801/2628870229","active_change":"2024-03-16T09:06:30.793104+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.112:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":2870055140}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":2554500034}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":1364363659}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":679680737}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":3325623408}]}]} 2024-03-16T09:13:50.409 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-16T09:13:50.410 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-16T09:13:50.410 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-16T09:13:50.648 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:13:52.069 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:51 smithi112 ceph-mon[32307]: pgmap v295: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.1 KiB/s wr, 16 op/s 2024-03-16T09:13:52.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:51 smithi159 ceph-mon[36739]: pgmap v295: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.1 KiB/s wr, 16 op/s 2024-03-16T09:13:52.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.300+0000 7fcfe4039700 1 -- 172.21.15.112:0/1577442103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfdc06b3e0 msgr2=0x7fcfdc10f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:52.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.300+0000 7fcfe4039700 1 --2- 172.21.15.112:0/1577442103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfdc06b3e0 0x7fcfdc10f120 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400b0d0 tx=0x7fcfd400b3e0 comp rx=0 tx=0).stop 2024-03-16T09:13:52.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.300+0000 7fcfe4039700 1 -- 172.21.15.112:0/1577442103 shutdown_connections 2024-03-16T09:13:52.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.300+0000 7fcfe4039700 1 --2- 172.21.15.112:0/1577442103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfdc108300 0x7fcfdc1086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:52.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.300+0000 7fcfe4039700 1 --2- 172.21.15.112:0/1577442103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfdc06b3e0 0x7fcfdc10f120 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:52.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.300+0000 7fcfe4039700 1 --2- 172.21.15.112:0/1577442103 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfdc06aa20 0x7fcfdc06aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:52.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.300+0000 7fcfe4039700 1 -- 172.21.15.112:0/1577442103 >> 172.21.15.112:0/1577442103 conn(0x7fcfdc0781e0 msgr2=0x7fcfdc0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:52.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.300+0000 7fcfe4039700 1 -- 172.21.15.112:0/1577442103 shutdown_connections 2024-03-16T09:13:52.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.300+0000 7fcfe4039700 1 -- 172.21.15.112:0/1577442103 wait complete. 2024-03-16T09:13:52.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe4039700 1 Processor -- start 2024-03-16T09:13:52.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe4039700 1 -- start start 2024-03-16T09:13:52.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe4039700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfdc06aa20 0x7fcfdc19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:52.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe4039700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfdc06b3e0 0x7fcfdc19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:52.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe4039700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfdc108300 0x7fcfdc198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:52.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe4039700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1126f0 con 0x7fcfdc06b3e0 2024-03-16T09:13:52.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe4039700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc112570 con 0x7fcfdc06aa20 2024-03-16T09:13:52.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe4039700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc112870 con 0x7fcfdc108300 2024-03-16T09:13:52.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe15d4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfdc06b3e0 0x7fcfdc19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:52.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe15d4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfdc06b3e0 0x7fcfdc19ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48646/0 (socket says 172.21.15.112:48646) 2024-03-16T09:13:52.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe1dd5700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfdc06aa20 0x7fcfdc19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:52.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe15d4700 1 -- 172.21.15.112:0/1938858373 learned_addr learned my addr 172.21.15.112:0/1938858373 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:52.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe25d6700 1 --2- 172.21.15.112:0/1938858373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfdc108300 0x7fcfdc198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:52.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe1dd5700 1 -- 172.21.15.112:0/1938858373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfdc108300 msgr2=0x7fcfdc198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:52.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe1dd5700 1 --2- 172.21.15.112:0/1938858373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfdc108300 0x7fcfdc198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:52.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe1dd5700 1 -- 172.21.15.112:0/1938858373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfdc06b3e0 msgr2=0x7fcfdc19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:52.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe1dd5700 1 --2- 172.21.15.112:0/1938858373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfdc06b3e0 0x7fcfdc19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:52.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe1dd5700 1 -- 172.21.15.112:0/1938858373 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc199030 con 0x7fcfdc06aa20 2024-03-16T09:13:52.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe25d6700 1 --2- 172.21.15.112:0/1938858373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfdc108300 0x7fcfdc198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:13:52.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.301+0000 7fcfe15d4700 1 --2- 172.21.15.112:0/1938858373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfdc06b3e0 0x7fcfdc19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:13:52.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.302+0000 7fcfe1dd5700 1 --2- 172.21.15.112:0/1938858373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfdc06aa20 0x7fcfdc19e6d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800e9f0 tx=0x7fcfd800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:52.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.302+0000 7fcfd2ffd700 1 -- 172.21.15.112:0/1938858373 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd800c9c0 con 0x7fcfdc06aa20 2024-03-16T09:13:52.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.302+0000 7fcfe4039700 1 -- 172.21.15.112:0/1938858373 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfdc199320 con 0x7fcfdc06aa20 2024-03-16T09:13:52.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.302+0000 7fcfd2ffd700 1 -- 172.21.15.112:0/1938858373 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcfd800cb60 con 0x7fcfdc06aa20 2024-03-16T09:13:52.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.302+0000 7fcfd2ffd700 1 -- 172.21.15.112:0/1938858373 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8018660 con 0x7fcfdc06aa20 2024-03-16T09:13:52.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.302+0000 7fcfe4039700 1 -- 172.21.15.112:0/1938858373 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfdc1a5530 con 0x7fcfdc06aa20 2024-03-16T09:13:52.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.303+0000 7fcfe4039700 1 -- 172.21.15.112:0/1938858373 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfdc1039b0 con 0x7fcfdc06aa20 2024-03-16T09:13:52.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.304+0000 7fcfd2ffd700 1 -- 172.21.15.112:0/1938858373 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fcfd8047a90 con 0x7fcfdc06aa20 2024-03-16T09:13:52.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.304+0000 7fcfd2ffd700 1 --2- 172.21.15.112:0/1938858373 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcfc80a2920 0x7fcfc80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:52.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.304+0000 7fcfe15d4700 1 --2- 172.21.15.112:0/1938858373 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcfc80a2920 0x7fcfc80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:52.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.304+0000 7fcfd2ffd700 1 -- 172.21.15.112:0/1938858373 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fcfd80cecd0 con 0x7fcfdc06aa20 2024-03-16T09:13:52.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.305+0000 7fcfe15d4700 1 --2- 172.21.15.112:0/1938858373 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcfc80a2920 0x7fcfc80a4de0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc078670 tx=0x7fcfcc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:52.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.308+0000 7fcfd2ffd700 1 -- 172.21.15.112:0/1938858373 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcfd809bcc0 con 0x7fcfdc06aa20 2024-03-16T09:13:52.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:51 smithi194 ceph-mon[36729]: pgmap v295: 97 pgs: 97 active+clean; 618 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.1 KiB/s wr, 16 op/s 2024-03-16T09:13:52.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.506+0000 7fcfe4039700 1 -- 172.21.15.112:0/1938858373 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcfdc1a5790 con 0x7fcfdc06aa20 2024-03-16T09:13:52.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.508+0000 7fcfd2ffd700 1 -- 172.21.15.112:0/1938858373 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22291 (secure 0 0 0) 0x7fcfd809b5c0 con 0x7fcfdc06aa20 2024-03-16T09:13:52.510 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:52.510 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":85,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:13:38.104757+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T09:13:35.040628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T09:13:35.689419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:13:52.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.510+0000 7fcfe4039700 1 -- 172.21.15.112:0/1938858373 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcfc80a2920 msgr2=0x7fcfc80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:52.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.510+0000 7fcfe4039700 1 --2- 172.21.15.112:0/1938858373 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcfc80a2920 0x7fcfc80a4de0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc078670 tx=0x7fcfcc009260 comp rx=0 tx=0).stop 2024-03-16T09:13:52.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.510+0000 7fcfe4039700 1 -- 172.21.15.112:0/1938858373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfdc06aa20 msgr2=0x7fcfdc19e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:52.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.510+0000 7fcfe4039700 1 --2- 172.21.15.112:0/1938858373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfdc06aa20 0x7fcfdc19e6d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800e9f0 tx=0x7fcfd800ed00 comp rx=0 tx=0).stop 2024-03-16T09:13:52.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.510+0000 7fcfe4039700 1 -- 172.21.15.112:0/1938858373 shutdown_connections 2024-03-16T09:13:52.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.510+0000 7fcfe4039700 1 --2- 172.21.15.112:0/1938858373 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcfc80a2920 0x7fcfc80a4de0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:52.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.510+0000 7fcfe4039700 1 --2- 172.21.15.112:0/1938858373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfdc06aa20 0x7fcfdc19e6d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:52.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.510+0000 7fcfe4039700 1 --2- 172.21.15.112:0/1938858373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcfdc108300 0x7fcfdc198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:52.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.510+0000 7fcfe4039700 1 --2- 172.21.15.112:0/1938858373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfdc06b3e0 0x7fcfdc19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:52.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.510+0000 7fcfe4039700 1 -- 172.21.15.112:0/1938858373 >> 172.21.15.112:0/1938858373 conn(0x7fcfdc0781e0 msgr2=0x7fcfdc0ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:52.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.510+0000 7fcfe4039700 1 -- 172.21.15.112:0/1938858373 shutdown_connections 2024-03-16T09:13:52.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:52.510+0000 7fcfe4039700 1 -- 172.21.15.112:0/1938858373 wait complete. 2024-03-16T09:13:53.170 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:52 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1938858373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:53.199 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-16T09:13:53.199 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-16T09:13:53.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:52 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1938858373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:53.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:52 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1938858373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:53.444 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:13:54.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:53 smithi112 ceph-mon[32307]: pgmap v296: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2024-03-16T09:13:54.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:53 smithi159 ceph-mon[36739]: pgmap v296: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2024-03-16T09:13:54.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:53 smithi194 ceph-mon[36729]: pgmap v296: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2024-03-16T09:13:55.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.411+0000 7fcead9c6700 1 -- 172.21.15.112:0/2543660304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcea8101520 msgr2=0x7fcea81019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:55.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.411+0000 7fcead9c6700 1 --2- 172.21.15.112:0/2543660304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcea8101520 0x7fcea81019a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fce98009990 tx=0x7fce98009ca0 comp rx=0 tx=0).stop 2024-03-16T09:13:55.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.411+0000 7fcead9c6700 1 -- 172.21.15.112:0/2543660304 shutdown_connections 2024-03-16T09:13:55.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.411+0000 7fcead9c6700 1 --2- 172.21.15.112:0/2543660304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcea8106160 0x7fcea8106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:55.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.412+0000 7fcead9c6700 1 --2- 172.21.15.112:0/2543660304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcea8101ee0 0x7fcea810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:55.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.412+0000 7fcead9c6700 1 --2- 172.21.15.112:0/2543660304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcea8101520 0x7fcea81019a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:55.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.412+0000 7fcead9c6700 1 -- 172.21.15.112:0/2543660304 >> 172.21.15.112:0/2543660304 conn(0x7fcea8077f80 msgr2=0x7fcea8078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:55.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.412+0000 7fcead9c6700 1 -- 172.21.15.112:0/2543660304 shutdown_connections 2024-03-16T09:13:55.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.412+0000 7fcead9c6700 1 -- 172.21.15.112:0/2543660304 wait complete. 2024-03-16T09:13:55.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.412+0000 7fcead9c6700 1 Processor -- start 2024-03-16T09:13:55.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.412+0000 7fcead9c6700 1 -- start start 2024-03-16T09:13:55.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.412+0000 7fcead9c6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcea8101520 0x7fcea81999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:55.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.412+0000 7fcead9c6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcea8101ee0 0x7fcea8199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:55.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.412+0000 7fcead9c6700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcea8106160 0x7fcea81a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:55.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.412+0000 7fcead9c6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcea81116a0 con 0x7fcea8101ee0 2024-03-16T09:13:55.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.412+0000 7fcead9c6700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcea8111520 con 0x7fcea8106160 2024-03-16T09:13:55.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.412+0000 7fcead9c6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcea8111820 con 0x7fcea8101520 2024-03-16T09:13:55.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.413+0000 7fcea67fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcea8101ee0 0x7fcea8199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:55.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.413+0000 7fcea67fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcea8101ee0 0x7fcea8199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48660/0 (socket says 172.21.15.112:48660) 2024-03-16T09:13:55.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.413+0000 7fcea67fc700 1 -- 172.21.15.112:0/578529761 learned_addr learned my addr 172.21.15.112:0/578529761 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:13:55.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.413+0000 7fcea6ffd700 1 --2- 172.21.15.112:0/578529761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcea8101520 0x7fcea81999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:55.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.413+0000 7fcea77fe700 1 --2- 172.21.15.112:0/578529761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcea8106160 0x7fcea81a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:55.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.413+0000 7fcea67fc700 1 -- 172.21.15.112:0/578529761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcea8101520 msgr2=0x7fcea81999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:55.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.413+0000 7fcea67fc700 1 --2- 172.21.15.112:0/578529761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcea8101520 0x7fcea81999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:55.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.413+0000 7fcea67fc700 1 -- 172.21.15.112:0/578529761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcea8106160 msgr2=0x7fcea81a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:55.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.413+0000 7fcea67fc700 1 --2- 172.21.15.112:0/578529761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcea8106160 0x7fcea81a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:55.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.413+0000 7fcea67fc700 1 -- 172.21.15.112:0/578529761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea81a4f60 con 0x7fcea8101ee0 2024-03-16T09:13:55.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.413+0000 7fcea6ffd700 1 --2- 172.21.15.112:0/578529761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcea8101520 0x7fcea81999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:13:55.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.413+0000 7fcea67fc700 1 --2- 172.21.15.112:0/578529761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcea8101ee0 0x7fcea8199ee0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fce98000c00 tx=0x7fce9800b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:55.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.413+0000 7fceac9c4700 1 -- 172.21.15.112:0/578529761 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce98027070 con 0x7fcea8101ee0 2024-03-16T09:13:55.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.413+0000 7fcead9c6700 1 -- 172.21.15.112:0/578529761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea81a5130 con 0x7fcea8101ee0 2024-03-16T09:13:55.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.413+0000 7fceac9c4700 1 -- 172.21.15.112:0/578529761 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fce98022470 con 0x7fcea8101ee0 2024-03-16T09:13:55.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.413+0000 7fceac9c4700 1 -- 172.21.15.112:0/578529761 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce98017630 con 0x7fcea8101ee0 2024-03-16T09:13:55.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.413+0000 7fcead9c6700 1 -- 172.21.15.112:0/578529761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea81a5580 con 0x7fcea8101ee0 2024-03-16T09:13:55.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.414+0000 7fcead9c6700 1 -- 172.21.15.112:0/578529761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea8068f90 con 0x7fcea8101ee0 2024-03-16T09:13:55.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.415+0000 7fceac9c4700 1 -- 172.21.15.112:0/578529761 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fce9802a070 con 0x7fcea8101ee0 2024-03-16T09:13:55.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.416+0000 7fceac9c4700 1 --2- 172.21.15.112:0/578529761 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fce900a2920 0x7fce900a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:13:55.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.416+0000 7fcea6ffd700 1 --2- 172.21.15.112:0/578529761 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fce900a2920 0x7fce900a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:13:55.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.416+0000 7fceac9c4700 1 -- 172.21.15.112:0/578529761 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fce980cfd40 con 0x7fcea8101ee0 2024-03-16T09:13:55.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.417+0000 7fcea6ffd700 1 --2- 172.21.15.112:0/578529761 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fce900a2920 0x7fce900a4de0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009fd0 tx=0x7fce9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:13:55.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.420+0000 7fceac9c4700 1 -- 172.21.15.112:0/578529761 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fce98031070 con 0x7fcea8101ee0 2024-03-16T09:13:55.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.614+0000 7fcead9c6700 1 -- 172.21.15.112:0/578529761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcea81a58a0 con 0x7fcea8101ee0 2024-03-16T09:13:55.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.616+0000 7fceac9c4700 1 -- 172.21.15.112:0/578529761 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22291 (secure 0 0 0) 0x7fce98013070 con 0x7fcea8101ee0 2024-03-16T09:13:55.618 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:13:55.619 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":85,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:13:38.104757+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T09:13:35.040628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T09:13:35.689419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:13:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.618+0000 7fcead9c6700 1 -- 172.21.15.112:0/578529761 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fce900a2920 msgr2=0x7fce900a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.618+0000 7fcead9c6700 1 --2- 172.21.15.112:0/578529761 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fce900a2920 0x7fce900a4de0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009fd0 tx=0x7fce9c009260 comp rx=0 tx=0).stop 2024-03-16T09:13:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.618+0000 7fcead9c6700 1 -- 172.21.15.112:0/578529761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcea8101ee0 msgr2=0x7fcea8199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:13:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.618+0000 7fcead9c6700 1 --2- 172.21.15.112:0/578529761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcea8101ee0 0x7fcea8199ee0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fce98000c00 tx=0x7fce9800b610 comp rx=0 tx=0).stop 2024-03-16T09:13:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.618+0000 7fcead9c6700 1 -- 172.21.15.112:0/578529761 shutdown_connections 2024-03-16T09:13:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.618+0000 7fcead9c6700 1 --2- 172.21.15.112:0/578529761 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fce900a2920 0x7fce900a4de0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.618+0000 7fcead9c6700 1 --2- 172.21.15.112:0/578529761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcea8106160 0x7fcea81a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.618+0000 7fcead9c6700 1 --2- 172.21.15.112:0/578529761 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcea8101520 0x7fcea81999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.618+0000 7fcead9c6700 1 --2- 172.21.15.112:0/578529761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcea8101ee0 0x7fcea8199ee0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:13:55.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.618+0000 7fcead9c6700 1 -- 172.21.15.112:0/578529761 >> 172.21.15.112:0/578529761 conn(0x7fcea8077f80 msgr2=0x7fcea80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:13:55.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.618+0000 7fcead9c6700 1 -- 172.21.15.112:0/578529761 shutdown_connections 2024-03-16T09:13:55.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:13:55.618+0000 7fcead9c6700 1 -- 172.21.15.112:0/578529761 wait complete. 2024-03-16T09:13:56.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:55 smithi194 ceph-mon[36729]: pgmap v297: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-03-16T09:13:56.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:55 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/578529761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:56.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:55 smithi112 ceph-mon[32307]: pgmap v297: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-03-16T09:13:56.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:55 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/578529761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:56.216 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-16T09:13:56.216 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-16T09:13:56.217 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-16T09:13:56.217 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-16T09:13:56.217 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-16T09:13:56.218 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-16T09:13:56.218 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-16T09:13:56.219 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-16T09:13:56.219 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-16T09:13:56.219 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-16T09:13:56.220 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-16T09:13:56.220 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-16T09:13:56.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:55 smithi159 ceph-mon[36739]: pgmap v297: 97 pgs: 97 active+clean; 623 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.6 KiB/s wr, 11 op/s 2024-03-16T09:13:56.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:55 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/578529761' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:13:56.865 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:13:56.866 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:13:56.878 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:13:56.885 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:13:57.660 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:13:57.661 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:13:57.661 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:13:57.662 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:13:57.662 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:13:58.021 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:57 smithi112 ceph-mon[32307]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-03-16T09:13:58.021 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:13:58.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:57 smithi159 ceph-mon[36739]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-03-16T09:13:58.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:57 smithi194 ceph-mon[36729]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-03-16T09:14:00.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:13:59 smithi159 ceph-mon[36739]: pgmap v299: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-03-16T09:14:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:13:59 smithi112 ceph-mon[32307]: pgmap v299: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-03-16T09:14:00.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:13:59 smithi194 ceph-mon[36729]: pgmap v299: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-03-16T09:14:02.088 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:01 smithi112 ceph-mon[32307]: pgmap v300: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-03-16T09:14:02.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:01 smithi159 ceph-mon[36739]: pgmap v300: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-03-16T09:14:02.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:01 smithi194 ceph-mon[36729]: pgmap v300: 97 pgs: 97 active+clean; 637 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-03-16T09:14:02.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.719+0000 7f6901e5d700 1 -- 172.21.15.112:0/3224262697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68fc074500 msgr2=0x7f68fc0748e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:02.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.719+0000 7f6901e5d700 1 --2- 172.21.15.112:0/3224262697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68fc074500 0x7f68fc0748e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009990 tx=0x7f68ec009ca0 comp rx=0 tx=0).stop 2024-03-16T09:14:02.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.719+0000 7f6901e5d700 1 -- 172.21.15.112:0/3224262697 shutdown_connections 2024-03-16T09:14:02.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.719+0000 7f6901e5d700 1 --2- 172.21.15.112:0/3224262697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68fc10ecd0 0x7f68fc1110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:02.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.719+0000 7f6901e5d700 1 --2- 172.21.15.112:0/3224262697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68fc074500 0x7f68fc0748e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:02.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.719+0000 7f6901e5d700 1 --2- 172.21.15.112:0/3224262697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68fc074e20 0x7f68fc10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:02.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.719+0000 7f6901e5d700 1 -- 172.21.15.112:0/3224262697 >> 172.21.15.112:0/3224262697 conn(0x7f68fc079400 msgr2=0x7f68fc079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:02.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.719+0000 7f6901e5d700 1 -- 172.21.15.112:0/3224262697 shutdown_connections 2024-03-16T09:14:02.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.719+0000 7f6901e5d700 1 -- 172.21.15.112:0/3224262697 wait complete. 2024-03-16T09:14:02.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.719+0000 7f6901e5d700 1 Processor -- start 2024-03-16T09:14:02.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.719+0000 7f6901e5d700 1 -- start start 2024-03-16T09:14:02.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f6901e5d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68fc074500 0x7f68fc19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:02.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f6901e5d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68fc074e20 0x7f68fc19b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:02.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f6901e5d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68fc10ecd0 0x7f68fc19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:02.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f6901e5d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc114920 con 0x7f68fc074e20 2024-03-16T09:14:02.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f6901e5d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc1147a0 con 0x7f68fc074500 2024-03-16T09:14:02.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f6901e5d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc114aa0 con 0x7f68fc10ecd0 2024-03-16T09:14:02.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f68faffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68fc074e20 0x7f68fc19b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:02.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f68faffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68fc074e20 0x7f68fc19b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44964/0 (socket says 172.21.15.112:44964) 2024-03-16T09:14:02.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f68faffd700 1 -- 172.21.15.112:0/4167939191 learned_addr learned my addr 172.21.15.112:0/4167939191 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:02.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f68fb7fe700 1 --2- 172.21.15.112:0/4167939191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68fc074500 0x7f68fc19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:02.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f68fbfff700 1 --2- 172.21.15.112:0/4167939191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68fc10ecd0 0x7f68fc19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:02.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f68faffd700 1 -- 172.21.15.112:0/4167939191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68fc10ecd0 msgr2=0x7f68fc19f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:02.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f68faffd700 1 --2- 172.21.15.112:0/4167939191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68fc10ecd0 0x7f68fc19f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:02.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f68faffd700 1 -- 172.21.15.112:0/4167939191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68fc074500 msgr2=0x7f68fc19ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:02.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f68faffd700 1 --2- 172.21.15.112:0/4167939191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68fc074500 0x7f68fc19ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:02.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f68faffd700 1 -- 172.21.15.112:0/4167939191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68fc19fda0 con 0x7f68fc074e20 2024-03-16T09:14:02.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f68faffd700 1 --2- 172.21.15.112:0/4167939191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68fc074e20 0x7f68fc19b2d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f68f000e9c0 tx=0x7f68f000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:02.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f68f8ff9700 1 -- 172.21.15.112:0/4167939191 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f000cbd0 con 0x7f68fc074e20 2024-03-16T09:14:02.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f6901e5d700 1 -- 172.21.15.112:0/4167939191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68fc1a0090 con 0x7f68fc074e20 2024-03-16T09:14:02.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f68f8ff9700 1 -- 172.21.15.112:0/4167939191 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f68f000ee60 con 0x7f68fc074e20 2024-03-16T09:14:02.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f68f8ff9700 1 -- 172.21.15.112:0/4167939191 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0010430 con 0x7f68fc074e20 2024-03-16T09:14:02.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.720+0000 7f6901e5d700 1 -- 172.21.15.112:0/4167939191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68fc1a7910 con 0x7f68fc074e20 2024-03-16T09:14:02.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.721+0000 7f6901e5d700 1 -- 172.21.15.112:0/4167939191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f68fc062a60 con 0x7f68fc074e20 2024-03-16T09:14:02.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.722+0000 7f68f8ff9700 1 -- 172.21.15.112:0/4167939191 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f68f0004750 con 0x7f68fc074e20 2024-03-16T09:14:02.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.723+0000 7f68f8ff9700 1 --2- 172.21.15.112:0/4167939191 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f68e40a2920 0x7f68e40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:02.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.723+0000 7f68fb7fe700 1 --2- 172.21.15.112:0/4167939191 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f68e40a2920 0x7f68e40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:02.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.723+0000 7f68f8ff9700 1 -- 172.21.15.112:0/4167939191 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f68f00cf0f0 con 0x7f68fc074e20 2024-03-16T09:14:02.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.723+0000 7f68f8ff9700 1 --2- 172.21.15.112:0/4167939191 >> [v2:172.21.15.159:6824/1906671120,v1:172.21.15.159:6825/1906671120] conn(0x7f68e40ac100 0x7f68e40ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:02.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.723+0000 7f68f8ff9700 1 -- 172.21.15.112:0/4167939191 --> [v2:172.21.15.159:6824/1906671120,v1:172.21.15.159:6825/1906671120] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f68f0012ec0 con 0x7f68e40ac100 2024-03-16T09:14:02.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.723+0000 7f68f8ff9700 1 -- 172.21.15.112:0/4167939191 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f68f00cf5d0 con 0x7f68fc074e20 2024-03-16T09:14:02.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.723+0000 7f68fb7fe700 1 --2- 172.21.15.112:0/4167939191 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f68e40a2920 0x7f68e40a4de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00b2e0 tx=0x7f68ec005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:02.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.723+0000 7f68fbfff700 1 --2- 172.21.15.112:0/4167939191 >> [v2:172.21.15.159:6824/1906671120,v1:172.21.15.159:6825/1906671120] conn(0x7f68e40ac100 0x7f68e40ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:02.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.724+0000 7f68fbfff700 1 --2- 172.21.15.112:0/4167939191 >> [v2:172.21.15.159:6824/1906671120,v1:172.21.15.159:6825/1906671120] conn(0x7f68e40ac100 0x7f68e40ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:02.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.725+0000 7f68f8ff9700 1 -- 172.21.15.112:0/4167939191 <== osd.10 v2:172.21.15.159:6824/1906671120 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f68f0012ec0 con 0x7f68e40ac100 2024-03-16T09:14:02.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.747+0000 7f6901e5d700 1 -- 172.21.15.112:0/4167939191 --> [v2:172.21.15.159:6824/1906671120,v1:172.21.15.159:6825/1906671120] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f68fc062ee0 con 0x7f68e40ac100 2024-03-16T09:14:02.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.747+0000 7f68f8ff9700 1 -- 172.21.15.112:0/4167939191 <== osd.10 v2:172.21.15.159:6824/1906671120 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f68fc062ee0 con 0x7f68e40ac100 2024-03-16T09:14:02.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.747+0000 7f6901e5d700 1 -- 172.21.15.112:0/4167939191 >> [v2:172.21.15.159:6824/1906671120,v1:172.21.15.159:6825/1906671120] conn(0x7f68e40ac100 msgr2=0x7f68e40ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:02.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.747+0000 7f6901e5d700 1 --2- 172.21.15.112:0/4167939191 >> [v2:172.21.15.159:6824/1906671120,v1:172.21.15.159:6825/1906671120] conn(0x7f68e40ac100 0x7f68e40ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:02.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.747+0000 7f6901e5d700 1 -- 172.21.15.112:0/4167939191 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f68e40a2920 msgr2=0x7f68e40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:02.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.747+0000 7f6901e5d700 1 --2- 172.21.15.112:0/4167939191 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f68e40a2920 0x7f68e40a4de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00b2e0 tx=0x7f68ec005960 comp rx=0 tx=0).stop 2024-03-16T09:14:02.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.747+0000 7f6901e5d700 1 -- 172.21.15.112:0/4167939191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68fc074e20 msgr2=0x7f68fc19b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:02.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.747+0000 7f6901e5d700 1 --2- 172.21.15.112:0/4167939191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68fc074e20 0x7f68fc19b2d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f68f000e9c0 tx=0x7f68f000ecd0 comp rx=0 tx=0).stop 2024-03-16T09:14:02.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.747+0000 7f68fbfff700 1 -- 172.21.15.112:0/4167939191 reap_dead start 2024-03-16T09:14:02.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.748+0000 7f6901e5d700 1 -- 172.21.15.112:0/4167939191 shutdown_connections 2024-03-16T09:14:02.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.748+0000 7f6901e5d700 1 -- 172.21.15.112:0/4167939191 >> 172.21.15.112:0/4167939191 conn(0x7f68fc079400 msgr2=0x7f68fc0ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:02.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.748+0000 7f6901e5d700 1 -- 172.21.15.112:0/4167939191 shutdown_connections 2024-03-16T09:14:02.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:02.748+0000 7f6901e5d700 1 -- 172.21.15.112:0/4167939191 wait complete. 2024-03-16T09:14:04.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:03 smithi159 ceph-mon[36739]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-03-16T09:14:04.307 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:03 smithi112 ceph-mon[32307]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-03-16T09:14:04.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:03 smithi194 ceph-mon[36729]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-03-16T09:14:06.215 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:05 smithi112 ceph-mon[32307]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-16T09:14:06.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:05 smithi159 ceph-mon[36739]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-16T09:14:06.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:05 smithi194 ceph-mon[36729]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-16T09:14:06.970 INFO:teuthology.orchestra.run.smithi112.stdout:309237645335 2024-03-16T09:14:06.970 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-16T09:14:07.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.574+0000 7f4aa269d700 1 -- 172.21.15.112:0/4095545366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a9c1083a0 msgr2=0x7f4a9c108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:07.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.574+0000 7f4aa269d700 1 --2- 172.21.15.112:0/4095545366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a9c1083a0 0x7f4a9c108780 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c009990 tx=0x7f4a8c009ca0 comp rx=0 tx=0).stop 2024-03-16T09:14:07.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.574+0000 7f4aa269d700 1 -- 172.21.15.112:0/4095545366 shutdown_connections 2024-03-16T09:14:07.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.574+0000 7f4aa269d700 1 --2- 172.21.15.112:0/4095545366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a9c102d10 0x7f4a9c10f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:07.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.575+0000 7f4aa269d700 1 --2- 172.21.15.112:0/4095545366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a9c1083a0 0x7f4a9c108780 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:07.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.575+0000 7f4aa269d700 1 --2- 172.21.15.112:0/4095545366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a9c102350 0x7f4a9c1027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:07.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.575+0000 7f4aa269d700 1 -- 172.21.15.112:0/4095545366 >> 172.21.15.112:0/4095545366 conn(0x7f4a9c079420 msgr2=0x7f4a9c079830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:07.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.575+0000 7f4aa269d700 1 -- 172.21.15.112:0/4095545366 shutdown_connections 2024-03-16T09:14:07.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.575+0000 7f4aa269d700 1 -- 172.21.15.112:0/4095545366 wait complete. 2024-03-16T09:14:07.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.575+0000 7f4aa269d700 1 Processor -- start 2024-03-16T09:14:07.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.575+0000 7f4aa269d700 1 -- start start 2024-03-16T09:14:07.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4aa269d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a9c102350 0x7f4a9c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:07.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4aa269d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a9c102d10 0x7f4a9c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:07.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4aa269d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a9c1083a0 0x7f4a9c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:07.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4aa269d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c112740 con 0x7f4a9c102350 2024-03-16T09:14:07.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4aa269d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c1125c0 con 0x7f4a9c102d10 2024-03-16T09:14:07.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4aa269d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c1128c0 con 0x7f4a9c1083a0 2024-03-16T09:14:07.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4a9bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a9c102350 0x7f4a9c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:07.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4a9bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a9c102350 0x7f4a9c19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44970/0 (socket says 172.21.15.112:44970) 2024-03-16T09:14:07.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4a9bfff700 1 -- 172.21.15.112:0/3760501847 learned_addr learned my addr 172.21.15.112:0/3760501847 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:07.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4aa0c3a700 1 --2- 172.21.15.112:0/3760501847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a9c1083a0 0x7f4a9c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:07.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4a9b7fe700 1 --2- 172.21.15.112:0/3760501847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a9c102d10 0x7f4a9c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:07.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4a9bfff700 1 -- 172.21.15.112:0/3760501847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a9c1083a0 msgr2=0x7f4a9c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:07.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4a9bfff700 1 --2- 172.21.15.112:0/3760501847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a9c1083a0 0x7f4a9c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:07.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4a9bfff700 1 -- 172.21.15.112:0/3760501847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a9c102d10 msgr2=0x7f4a9c19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:07.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4a9bfff700 1 --2- 172.21.15.112:0/3760501847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a9c102d10 0x7f4a9c19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:07.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4a9bfff700 1 -- 172.21.15.112:0/3760501847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c1990a0 con 0x7f4a9c102350 2024-03-16T09:14:07.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4a9bfff700 1 --2- 172.21.15.112:0/3760501847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a9c102350 0x7f4a9c19e780 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c000c00 tx=0x7f4a8c00bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:07.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4a997fa700 1 -- 172.21.15.112:0/3760501847 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c027070 con 0x7f4a9c102350 2024-03-16T09:14:07.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4aa269d700 1 -- 172.21.15.112:0/3760501847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9c199330 con 0x7f4a9c102350 2024-03-16T09:14:07.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4aa269d700 1 -- 172.21.15.112:0/3760501847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9c1a5650 con 0x7f4a9c102350 2024-03-16T09:14:07.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4a997fa700 1 -- 172.21.15.112:0/3760501847 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4a8c003d90 con 0x7f4a9c102350 2024-03-16T09:14:07.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.576+0000 7f4a997fa700 1 -- 172.21.15.112:0/3760501847 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c00f460 con 0x7f4a9c102350 2024-03-16T09:14:07.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.577+0000 7f4aa269d700 1 -- 172.21.15.112:0/3760501847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a9c062a60 con 0x7f4a9c102350 2024-03-16T09:14:07.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.578+0000 7f4a997fa700 1 -- 172.21.15.112:0/3760501847 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f4a8c02a070 con 0x7f4a9c102350 2024-03-16T09:14:07.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.579+0000 7f4a997fa700 1 --2- 172.21.15.112:0/3760501847 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f4a840a2920 0x7f4a840a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:07.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.579+0000 7f4a9b7fe700 1 --2- 172.21.15.112:0/3760501847 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f4a840a2920 0x7f4a840a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:07.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.579+0000 7f4a997fa700 1 -- 172.21.15.112:0/3760501847 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f4a8c0cff30 con 0x7f4a9c102350 2024-03-16T09:14:07.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.579+0000 7f4a997fa700 1 --2- 172.21.15.112:0/3760501847 >> [v2:172.21.15.159:6808/4036563014,v1:172.21.15.159:6809/4036563014] conn(0x7f4a840ac100 0x7f4a840ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:07.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.579+0000 7f4a997fa700 1 -- 172.21.15.112:0/3760501847 --> [v2:172.21.15.159:6808/4036563014,v1:172.21.15.159:6809/4036563014] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a8c0043d0 con 0x7f4a840ac100 2024-03-16T09:14:07.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.579+0000 7f4a997fa700 1 -- 172.21.15.112:0/3760501847 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f4a8c0d0410 con 0x7f4a9c102350 2024-03-16T09:14:07.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.579+0000 7f4a9b7fe700 1 --2- 172.21.15.112:0/3760501847 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f4a840a2920 0x7f4a840a4de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c079990 tx=0x7f4a90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:07.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.579+0000 7f4aa0c3a700 1 --2- 172.21.15.112:0/3760501847 >> [v2:172.21.15.159:6808/4036563014,v1:172.21.15.159:6809/4036563014] conn(0x7f4a840ac100 0x7f4a840ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:07.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.580+0000 7f4aa0c3a700 1 --2- 172.21.15.112:0/3760501847 >> [v2:172.21.15.159:6808/4036563014,v1:172.21.15.159:6809/4036563014] conn(0x7f4a840ac100 0x7f4a840ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:07.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.581+0000 7f4a997fa700 1 -- 172.21.15.112:0/3760501847 <== osd.4 v2:172.21.15.159:6808/4036563014 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a8c0043d0 con 0x7f4a840ac100 2024-03-16T09:14:07.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.597+0000 7f4aa269d700 1 -- 172.21.15.112:0/3760501847 --> [v2:172.21.15.159:6808/4036563014,v1:172.21.15.159:6809/4036563014] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a9c062e60 con 0x7f4a840ac100 2024-03-16T09:14:07.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.597+0000 7f4a997fa700 1 -- 172.21.15.112:0/3760501847 <== osd.4 v2:172.21.15.159:6808/4036563014 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4a9c062e60 con 0x7f4a840ac100 2024-03-16T09:14:07.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.598+0000 7f4aa269d700 1 -- 172.21.15.112:0/3760501847 >> [v2:172.21.15.159:6808/4036563014,v1:172.21.15.159:6809/4036563014] conn(0x7f4a840ac100 msgr2=0x7f4a840ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:07.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.598+0000 7f4aa269d700 1 --2- 172.21.15.112:0/3760501847 >> [v2:172.21.15.159:6808/4036563014,v1:172.21.15.159:6809/4036563014] conn(0x7f4a840ac100 0x7f4a840ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:07.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.598+0000 7f4aa269d700 1 -- 172.21.15.112:0/3760501847 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f4a840a2920 msgr2=0x7f4a840a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:07.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.598+0000 7f4aa269d700 1 --2- 172.21.15.112:0/3760501847 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f4a840a2920 0x7f4a840a4de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c079990 tx=0x7f4a90009260 comp rx=0 tx=0).stop 2024-03-16T09:14:07.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.598+0000 7f4aa269d700 1 -- 172.21.15.112:0/3760501847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a9c102350 msgr2=0x7f4a9c19e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:07.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.598+0000 7f4aa269d700 1 --2- 172.21.15.112:0/3760501847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a9c102350 0x7f4a9c19e780 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c000c00 tx=0x7f4a8c00bb20 comp rx=0 tx=0).stop 2024-03-16T09:14:07.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.598+0000 7f4aa0c3a700 1 -- 172.21.15.112:0/3760501847 reap_dead start 2024-03-16T09:14:07.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.598+0000 7f4aa269d700 1 -- 172.21.15.112:0/3760501847 shutdown_connections 2024-03-16T09:14:07.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.598+0000 7f4aa269d700 1 -- 172.21.15.112:0/3760501847 >> 172.21.15.112:0/3760501847 conn(0x7f4a9c079420 msgr2=0x7f4a9c100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:07.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.598+0000 7f4aa269d700 1 -- 172.21.15.112:0/3760501847 shutdown_connections 2024-03-16T09:14:07.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:07.598+0000 7f4aa269d700 1 -- 172.21.15.112:0/3760501847 wait complete. 2024-03-16T09:14:08.088 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:14:08.207 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:07 smithi112 ceph-mon[32307]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-16T09:14:08.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:07 smithi159 ceph-mon[36739]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-16T09:14:08.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.306+0000 7fb04e1fc700 1 -- 172.21.15.112:0/1022006127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb048101e10 msgr2=0x7fb048105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:08.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.306+0000 7fb04e1fc700 1 --2- 172.21.15.112:0/1022006127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb048101e10 0x7fb048105dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0099c0 tx=0x7fb03c009cd0 comp rx=0 tx=0).stop 2024-03-16T09:14:08.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.306+0000 7fb04e1fc700 1 -- 172.21.15.112:0/1022006127 shutdown_connections 2024-03-16T09:14:08.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.306+0000 7fb04e1fc700 1 --2- 172.21.15.112:0/1022006127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0481064b0 0x7fb04810e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:08.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.306+0000 7fb04e1fc700 1 --2- 172.21.15.112:0/1022006127 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb048101460 0x7fb048101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:08.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.306+0000 7fb04e1fc700 1 --2- 172.21.15.112:0/1022006127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb048101e10 0x7fb048105dd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:08.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.306+0000 7fb04e1fc700 1 -- 172.21.15.112:0/1022006127 >> 172.21.15.112:0/1022006127 conn(0x7fb048077220 msgr2=0x7fb048077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:08.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.306+0000 7fb04e1fc700 1 -- 172.21.15.112:0/1022006127 shutdown_connections 2024-03-16T09:14:08.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.306+0000 7fb04e1fc700 1 -- 172.21.15.112:0/1022006127 wait complete. 2024-03-16T09:14:08.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.307+0000 7fb04e1fc700 1 Processor -- start 2024-03-16T09:14:08.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.307+0000 7fb04e1fc700 1 -- start start 2024-03-16T09:14:08.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.307+0000 7fb04e1fc700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb048101460 0x7fb04819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:08.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.307+0000 7fb04e1fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb048101e10 0x7fb04819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:08.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.307+0000 7fb04e1fc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0481064b0 0x7fb0481988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:08.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.307+0000 7fb04e1fc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb048111560 con 0x7fb048101e10 2024-03-16T09:14:08.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.307+0000 7fb04e1fc700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb0481113e0 con 0x7fb048101460 2024-03-16T09:14:08.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.307+0000 7fb04e1fc700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb0481116e0 con 0x7fb0481064b0 2024-03-16T09:14:08.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.307+0000 7fb046ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb048101e10 0x7fb04819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:08.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.307+0000 7fb046ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb048101e10 0x7fb04819ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44990/0 (socket says 172.21.15.112:44990) 2024-03-16T09:14:08.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.307+0000 7fb046ffd700 1 -- 172.21.15.112:0/2360313109 learned_addr learned my addr 172.21.15.112:0/2360313109 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:08.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.308+0000 7fb0477fe700 1 --2- 172.21.15.112:0/2360313109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb048101460 0x7fb04819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:08.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.307+0000 7fb047fff700 1 --2- 172.21.15.112:0/2360313109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0481064b0 0x7fb0481988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:08.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.308+0000 7fb046ffd700 1 -- 172.21.15.112:0/2360313109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0481064b0 msgr2=0x7fb0481988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:08.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.308+0000 7fb046ffd700 1 --2- 172.21.15.112:0/2360313109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0481064b0 0x7fb0481988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:08.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.308+0000 7fb046ffd700 1 -- 172.21.15.112:0/2360313109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb048101460 msgr2=0x7fb04819e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:08.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.308+0000 7fb046ffd700 1 --2- 172.21.15.112:0/2360313109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb048101460 0x7fb04819e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:08.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.308+0000 7fb046ffd700 1 -- 172.21.15.112:0/2360313109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb048199040 con 0x7fb048101e10 2024-03-16T09:14:08.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.308+0000 7fb046ffd700 1 --2- 172.21.15.112:0/2360313109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb048101e10 0x7fb04819ec60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb03c006010 tx=0x7fb03c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:08.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.308+0000 7fb044ff9700 1 -- 172.21.15.112:0/2360313109 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c009e60 con 0x7fb048101e10 2024-03-16T09:14:08.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.308+0000 7fb04e1fc700 1 -- 172.21.15.112:0/2360313109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0481992d0 con 0x7fb048101e10 2024-03-16T09:14:08.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.308+0000 7fb044ff9700 1 -- 172.21.15.112:0/2360313109 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb03c00be40 con 0x7fb048101e10 2024-03-16T09:14:08.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.308+0000 7fb044ff9700 1 -- 172.21.15.112:0/2360313109 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0173f0 con 0x7fb048101e10 2024-03-16T09:14:08.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.308+0000 7fb04e1fc700 1 -- 172.21.15.112:0/2360313109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0481a55f0 con 0x7fb048101e10 2024-03-16T09:14:08.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.308+0000 7fb0477fe700 1 --2- 172.21.15.112:0/2360313109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb048101460 0x7fb04819e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:14:08.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.309+0000 7fb04e1fc700 1 -- 172.21.15.112:0/2360313109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb0480629e0 con 0x7fb048101e10 2024-03-16T09:14:08.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.310+0000 7fb044ff9700 1 -- 172.21.15.112:0/2360313109 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fb03c00f460 con 0x7fb048101e10 2024-03-16T09:14:08.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.311+0000 7fb044ff9700 1 --2- 172.21.15.112:0/2360313109 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb0300a2920 0x7fb0300a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:08.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.311+0000 7fb0477fe700 1 --2- 172.21.15.112:0/2360313109 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb0300a2920 0x7fb0300a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:08.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.311+0000 7fb044ff9700 1 -- 172.21.15.112:0/2360313109 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb03c0cf930 con 0x7fb048101e10 2024-03-16T09:14:08.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.311+0000 7fb044ff9700 1 --2- 172.21.15.112:0/2360313109 >> [v2:172.21.15.194:6800/4037530979,v1:172.21.15.194:6801/4037530979] conn(0x7fb0300ac100 0x7fb0300ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:08.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.311+0000 7fb044ff9700 1 -- 172.21.15.112:0/2360313109 --> [v2:172.21.15.194:6800/4037530979,v1:172.21.15.194:6801/4037530979] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb03c001d30 con 0x7fb0300ac100 2024-03-16T09:14:08.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.311+0000 7fb044ff9700 1 -- 172.21.15.112:0/2360313109 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fb03c0cfe10 con 0x7fb048101e10 2024-03-16T09:14:08.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.311+0000 7fb047fff700 1 --2- 172.21.15.112:0/2360313109 >> [v2:172.21.15.194:6800/4037530979,v1:172.21.15.194:6801/4037530979] conn(0x7fb0300ac100 0x7fb0300ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:08.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.312+0000 7fb0477fe700 1 --2- 172.21.15.112:0/2360313109 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb0300a2920 0x7fb0300a4de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb048199a20 tx=0x7fb038009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:08.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.312+0000 7fb047fff700 1 --2- 172.21.15.112:0/2360313109 >> [v2:172.21.15.194:6800/4037530979,v1:172.21.15.194:6801/4037530979] conn(0x7fb0300ac100 0x7fb0300ae520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:08.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.313+0000 7fb044ff9700 1 -- 172.21.15.112:0/2360313109 <== osd.2 v2:172.21.15.194:6800/4037530979 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb03c001d30 con 0x7fb0300ac100 2024-03-16T09:14:08.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.329+0000 7fb04e1fc700 1 -- 172.21.15.112:0/2360313109 --> [v2:172.21.15.194:6800/4037530979,v1:172.21.15.194:6801/4037530979] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb048062e60 con 0x7fb0300ac100 2024-03-16T09:14:08.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.329+0000 7fb044ff9700 1 -- 172.21.15.112:0/2360313109 <== osd.2 v2:172.21.15.194:6800/4037530979 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb048062e60 con 0x7fb0300ac100 2024-03-16T09:14:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.329+0000 7fb04e1fc700 1 -- 172.21.15.112:0/2360313109 >> [v2:172.21.15.194:6800/4037530979,v1:172.21.15.194:6801/4037530979] conn(0x7fb0300ac100 msgr2=0x7fb0300ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.329+0000 7fb04e1fc700 1 --2- 172.21.15.112:0/2360313109 >> [v2:172.21.15.194:6800/4037530979,v1:172.21.15.194:6801/4037530979] conn(0x7fb0300ac100 0x7fb0300ae520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.329+0000 7fb04e1fc700 1 -- 172.21.15.112:0/2360313109 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb0300a2920 msgr2=0x7fb0300a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.329+0000 7fb04e1fc700 1 --2- 172.21.15.112:0/2360313109 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb0300a2920 0x7fb0300a4de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb048199a20 tx=0x7fb038009260 comp rx=0 tx=0).stop 2024-03-16T09:14:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.329+0000 7fb04e1fc700 1 -- 172.21.15.112:0/2360313109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb048101e10 msgr2=0x7fb04819ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.329+0000 7fb04e1fc700 1 --2- 172.21.15.112:0/2360313109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb048101e10 0x7fb04819ec60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb03c006010 tx=0x7fb03c003ce0 comp rx=0 tx=0).stop 2024-03-16T09:14:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.329+0000 7fb047fff700 1 -- 172.21.15.112:0/2360313109 reap_dead start 2024-03-16T09:14:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.329+0000 7fb04e1fc700 1 -- 172.21.15.112:0/2360313109 shutdown_connections 2024-03-16T09:14:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.329+0000 7fb04e1fc700 1 -- 172.21.15.112:0/2360313109 >> 172.21.15.112:0/2360313109 conn(0x7fb048077220 msgr2=0x7fb04806aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:08.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.330+0000 7fb04e1fc700 1 -- 172.21.15.112:0/2360313109 shutdown_connections 2024-03-16T09:14:08.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:08.330+0000 7fb04e1fc700 1 -- 172.21.15.112:0/2360313109 wait complete. 2024-03-16T09:14:08.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:07 smithi194 ceph-mon[36729]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-16T09:14:10.021 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:09 smithi112 ceph-mon[32307]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:14:10.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:09 smithi159 ceph-mon[36739]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:14:10.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:09 smithi194 ceph-mon[36729]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:14:11.639 INFO:teuthology.orchestra.run.smithi112.stdout:77309411394 2024-03-16T09:14:11.639 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-16T09:14:12.133 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:11 smithi112 ceph-mon[32307]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T09:14:12.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:11 smithi159 ceph-mon[36739]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T09:14:12.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.342+0000 7f41e400e700 1 -- 172.21.15.112:0/266727459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41dc101690 msgr2=0x7f41dc10f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:12.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.342+0000 7f41e400e700 1 --2- 172.21.15.112:0/266727459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41dc101690 0x7f41dc10f220 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f41d400b0d0 tx=0x7f41d400b3e0 comp rx=0 tx=0).stop 2024-03-16T09:14:12.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.342+0000 7f41e400e700 1 -- 172.21.15.112:0/266727459 shutdown_connections 2024-03-16T09:14:12.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.342+0000 7f41e400e700 1 --2- 172.21.15.112:0/266727459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41dc101690 0x7f41dc10f220 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:12.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.342+0000 7f41e400e700 1 --2- 172.21.15.112:0/266727459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41dc1073a0 0x7f41dc107780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:12.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.342+0000 7f41e400e700 1 --2- 172.21.15.112:0/266727459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41dc100cd0 0x7f41dc101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:12.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.342+0000 7f41e400e700 1 -- 172.21.15.112:0/266727459 >> 172.21.15.112:0/266727459 conn(0x7f41dc076520 msgr2=0x7f41dc076930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:12.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.343+0000 7f41e400e700 1 -- 172.21.15.112:0/266727459 shutdown_connections 2024-03-16T09:14:12.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.343+0000 7f41e400e700 1 -- 172.21.15.112:0/266727459 wait complete. 2024-03-16T09:14:12.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.343+0000 7f41e400e700 1 Processor -- start 2024-03-16T09:14:12.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.343+0000 7f41e400e700 1 -- start start 2024-03-16T09:14:12.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.343+0000 7f41e400e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41dc100cd0 0x7f41dc19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:12.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.343+0000 7f41e400e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41dc101690 0x7f41dc19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:12.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.343+0000 7f41e400e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41dc1073a0 0x7f41dc198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:12.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.343+0000 7f41e400e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc1127a0 con 0x7f41dc101690 2024-03-16T09:14:12.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.343+0000 7f41e400e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc112620 con 0x7f41dc1073a0 2024-03-16T09:14:12.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.343+0000 7f41e400e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc112920 con 0x7f41dc100cd0 2024-03-16T09:14:12.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.344+0000 7f41e15a9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41dc101690 0x7f41dc19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:12.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.344+0000 7f41e15a9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41dc101690 0x7f41dc19ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50018/0 (socket says 172.21.15.112:50018) 2024-03-16T09:14:12.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.344+0000 7f41e15a9700 1 -- 172.21.15.112:0/3952126251 learned_addr learned my addr 172.21.15.112:0/3952126251 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:12.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.344+0000 7f41e25ab700 1 --2- 172.21.15.112:0/3952126251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41dc1073a0 0x7f41dc198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:12.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.344+0000 7f41e1daa700 1 --2- 172.21.15.112:0/3952126251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41dc100cd0 0x7f41dc19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:12.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.344+0000 7f41e15a9700 1 -- 172.21.15.112:0/3952126251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41dc100cd0 msgr2=0x7f41dc19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:12.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.344+0000 7f41e15a9700 1 --2- 172.21.15.112:0/3952126251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f41dc100cd0 0x7f41dc19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:12.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.344+0000 7f41e15a9700 1 -- 172.21.15.112:0/3952126251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41dc1073a0 msgr2=0x7f41dc198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:12.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.344+0000 7f41e15a9700 1 --2- 172.21.15.112:0/3952126251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41dc1073a0 0x7f41dc198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:12.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.344+0000 7f41e15a9700 1 -- 172.21.15.112:0/3952126251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41dc1990e0 con 0x7f41dc101690 2024-03-16T09:14:12.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.344+0000 7f41e15a9700 1 --2- 172.21.15.112:0/3952126251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41dc101690 0x7f41dc19ed00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00e9a0 tx=0x7f41cc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:12.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.344+0000 7f41d2ffd700 1 -- 172.21.15.112:0/3952126251 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc00ca10 con 0x7f41dc101690 2024-03-16T09:14:12.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.344+0000 7f41d2ffd700 1 -- 172.21.15.112:0/3952126251 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f41cc00cbb0 con 0x7f41dc101690 2024-03-16T09:14:12.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.344+0000 7f41e400e700 1 -- 172.21.15.112:0/3952126251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41dc1993d0 con 0x7f41dc101690 2024-03-16T09:14:12.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.344+0000 7f41d2ffd700 1 -- 172.21.15.112:0/3952126251 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc018710 con 0x7f41dc101690 2024-03-16T09:14:12.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.344+0000 7f41e400e700 1 -- 172.21.15.112:0/3952126251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41dc1a5690 con 0x7f41dc101690 2024-03-16T09:14:12.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.345+0000 7f41e400e700 1 -- 172.21.15.112:0/3952126251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f41dc0ff5a0 con 0x7f41dc101690 2024-03-16T09:14:12.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.346+0000 7f41d2ffd700 1 -- 172.21.15.112:0/3952126251 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f41cc010430 con 0x7f41dc101690 2024-03-16T09:14:12.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.347+0000 7f41d2ffd700 1 --2- 172.21.15.112:0/3952126251 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f41c80a29a0 0x7f41c80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:12.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.347+0000 7f41e1daa700 1 --2- 172.21.15.112:0/3952126251 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f41c80a29a0 0x7f41c80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:12.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.347+0000 7f41d2ffd700 1 -- 172.21.15.112:0/3952126251 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f41cc0cf360 con 0x7f41dc101690 2024-03-16T09:14:12.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.347+0000 7f41d2ffd700 1 --2- 172.21.15.112:0/3952126251 >> [v2:172.21.15.194:6808/1459524918,v1:172.21.15.194:6809/1459524918] conn(0x7f41c80ac180 0x7f41c80ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:12.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.347+0000 7f41d2ffd700 1 -- 172.21.15.112:0/3952126251 --> [v2:172.21.15.194:6808/1459524918,v1:172.21.15.194:6809/1459524918] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f41cc00ae50 con 0x7f41c80ac180 2024-03-16T09:14:12.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.347+0000 7f41d2ffd700 1 -- 172.21.15.112:0/3952126251 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f41cc0cf840 con 0x7f41dc101690 2024-03-16T09:14:12.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.347+0000 7f41e1daa700 1 --2- 172.21.15.112:0/3952126251 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f41c80a29a0 0x7f41c80a4e60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f41dc199ac0 tx=0x7f41d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:12.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.347+0000 7f41e25ab700 1 --2- 172.21.15.112:0/3952126251 >> [v2:172.21.15.194:6808/1459524918,v1:172.21.15.194:6809/1459524918] conn(0x7f41c80ac180 0x7f41c80ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:12.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.348+0000 7f41e25ab700 1 --2- 172.21.15.112:0/3952126251 >> [v2:172.21.15.194:6808/1459524918,v1:172.21.15.194:6809/1459524918] conn(0x7f41c80ac180 0x7f41c80ae5a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:12.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.348+0000 7f41d2ffd700 1 -- 172.21.15.112:0/3952126251 <== osd.5 v2:172.21.15.194:6808/1459524918 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f41cc00ae50 con 0x7f41c80ac180 2024-03-16T09:14:12.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.368+0000 7f41e400e700 1 -- 172.21.15.112:0/3952126251 --> [v2:172.21.15.194:6808/1459524918,v1:172.21.15.194:6809/1459524918] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f41dc06a450 con 0x7f41c80ac180 2024-03-16T09:14:12.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.368+0000 7f41d2ffd700 1 -- 172.21.15.112:0/3952126251 <== osd.5 v2:172.21.15.194:6808/1459524918 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f41dc06a450 con 0x7f41c80ac180 2024-03-16T09:14:12.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.369+0000 7f41e400e700 1 -- 172.21.15.112:0/3952126251 >> [v2:172.21.15.194:6808/1459524918,v1:172.21.15.194:6809/1459524918] conn(0x7f41c80ac180 msgr2=0x7f41c80ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:12.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.369+0000 7f41e400e700 1 --2- 172.21.15.112:0/3952126251 >> [v2:172.21.15.194:6808/1459524918,v1:172.21.15.194:6809/1459524918] conn(0x7f41c80ac180 0x7f41c80ae5a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:12.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.369+0000 7f41e400e700 1 -- 172.21.15.112:0/3952126251 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f41c80a29a0 msgr2=0x7f41c80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:12.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.369+0000 7f41e400e700 1 --2- 172.21.15.112:0/3952126251 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f41c80a29a0 0x7f41c80a4e60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f41dc199ac0 tx=0x7f41d8009260 comp rx=0 tx=0).stop 2024-03-16T09:14:12.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.369+0000 7f41e400e700 1 -- 172.21.15.112:0/3952126251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41dc101690 msgr2=0x7f41dc19ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:12.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.369+0000 7f41e400e700 1 --2- 172.21.15.112:0/3952126251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41dc101690 0x7f41dc19ed00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00e9a0 tx=0x7f41cc00ecb0 comp rx=0 tx=0).stop 2024-03-16T09:14:12.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.369+0000 7f41e25ab700 1 -- 172.21.15.112:0/3952126251 reap_dead start 2024-03-16T09:14:12.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.369+0000 7f41e400e700 1 -- 172.21.15.112:0/3952126251 shutdown_connections 2024-03-16T09:14:12.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.369+0000 7f41e400e700 1 -- 172.21.15.112:0/3952126251 >> 172.21.15.112:0/3952126251 conn(0x7f41dc076520 msgr2=0x7f41dc0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:12.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.369+0000 7f41e400e700 1 -- 172.21.15.112:0/3952126251 shutdown_connections 2024-03-16T09:14:12.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.369+0000 7f41e400e700 1 -- 172.21.15.112:0/3952126251 wait complete. 2024-03-16T09:14:12.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:11 smithi194 ceph-mon[36729]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T09:14:12.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.787+0000 7f6b5a019700 1 -- 172.21.15.112:0/1277876789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b54100e70 msgr2=0x7f6b54101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:12.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.787+0000 7f6b5a019700 1 --2- 172.21.15.112:0/1277876789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b54100e70 0x7f6b54101250 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6b440099c0 tx=0x7f6b44009cd0 comp rx=0 tx=0).stop 2024-03-16T09:14:12.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.787+0000 7f6b5a019700 1 -- 172.21.15.112:0/1277876789 shutdown_connections 2024-03-16T09:14:12.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.787+0000 7f6b5a019700 1 --2- 172.21.15.112:0/1277876789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b54107690 0x7f6b5410f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:12.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.787+0000 7f6b5a019700 1 --2- 172.21.15.112:0/1277876789 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b54100e70 0x7f6b54101250 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:12.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.787+0000 7f6b5a019700 1 --2- 172.21.15.112:0/1277876789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b54101820 0x7f6b54106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:12.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.787+0000 7f6b5a019700 1 -- 172.21.15.112:0/1277876789 >> 172.21.15.112:0/1277876789 conn(0x7f6b54076510 msgr2=0x7f6b54076920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:12.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.787+0000 7f6b5a019700 1 -- 172.21.15.112:0/1277876789 shutdown_connections 2024-03-16T09:14:12.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.787+0000 7f6b5a019700 1 -- 172.21.15.112:0/1277876789 wait complete. 2024-03-16T09:14:12.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.787+0000 7f6b5a019700 1 Processor -- start 2024-03-16T09:14:12.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.787+0000 7f6b5a019700 1 -- start start 2024-03-16T09:14:12.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b5a019700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b54100e70 0x7f6b54101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b5a019700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b54101820 0x7f6b54102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b5a019700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b54107690 0x7f6b54102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b5a019700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6b54112710 con 0x7f6b54101820 2024-03-16T09:14:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b5a019700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6b54112590 con 0x7f6b54107690 2024-03-16T09:14:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b5a019700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6b54112890 con 0x7f6b54100e70 2024-03-16T09:14:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b4affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b54101820 0x7f6b54102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b4affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b54101820 0x7f6b54102330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50038/0 (socket says 172.21.15.112:50038) 2024-03-16T09:14:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b4affd700 1 -- 172.21.15.112:0/3459775540 learned_addr learned my addr 172.21.15.112:0/3459775540 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b537fe700 1 --2- 172.21.15.112:0/3459775540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b54100e70 0x7f6b54101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b53fff700 1 --2- 172.21.15.112:0/3459775540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b54107690 0x7f6b54102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b4affd700 1 -- 172.21.15.112:0/3459775540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b54100e70 msgr2=0x7f6b54101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b4affd700 1 --2- 172.21.15.112:0/3459775540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b54100e70 0x7f6b54101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:12.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b4affd700 1 -- 172.21.15.112:0/3459775540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b54107690 msgr2=0x7f6b54102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b4affd700 1 --2- 172.21.15.112:0/3459775540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b54107690 0x7f6b54102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b4affd700 1 -- 172.21.15.112:0/3459775540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b541a70b0 con 0x7f6b54101820 2024-03-16T09:14:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b537fe700 1 --2- 172.21.15.112:0/3459775540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b54100e70 0x7f6b54101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b4affd700 1 --2- 172.21.15.112:0/3459775540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b54101820 0x7f6b54102330 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009fd0 tx=0x7f6b3c00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b517fa700 1 -- 172.21.15.112:0/3459775540 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c00cbd0 con 0x7f6b54101820 2024-03-16T09:14:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b5a019700 1 -- 172.21.15.112:0/3459775540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b541a73a0 con 0x7f6b54101820 2024-03-16T09:14:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b517fa700 1 -- 172.21.15.112:0/3459775540 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6b3c00ee90 con 0x7f6b54101820 2024-03-16T09:14:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.788+0000 7f6b517fa700 1 -- 172.21.15.112:0/3459775540 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c010430 con 0x7f6b54101820 2024-03-16T09:14:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.789+0000 7f6b5a019700 1 -- 172.21.15.112:0/3459775540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b541a7930 con 0x7f6b54101820 2024-03-16T09:14:12.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.789+0000 7f6b5a019700 1 -- 172.21.15.112:0/3459775540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6b540629e0 con 0x7f6b54101820 2024-03-16T09:14:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.790+0000 7f6b517fa700 1 -- 172.21.15.112:0/3459775540 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f6b3c003eb0 con 0x7f6b54101820 2024-03-16T09:14:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.791+0000 7f6b517fa700 1 --2- 172.21.15.112:0/3459775540 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6b400a2920 0x7f6b400a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:12.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.791+0000 7f6b537fe700 1 --2- 172.21.15.112:0/3459775540 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6b400a2920 0x7f6b400a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:12.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.791+0000 7f6b517fa700 1 -- 172.21.15.112:0/3459775540 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f6b3c0cedf0 con 0x7f6b54101820 2024-03-16T09:14:12.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.791+0000 7f6b517fa700 1 --2- 172.21.15.112:0/3459775540 >> [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464] conn(0x7f6b400ac100 0x7f6b400ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:12.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.791+0000 7f6b517fa700 1 -- 172.21.15.112:0/3459775540 --> [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b3c011e50 con 0x7f6b400ac100 2024-03-16T09:14:12.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.791+0000 7f6b517fa700 1 -- 172.21.15.112:0/3459775540 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f6b3c0cf2d0 con 0x7f6b54101820 2024-03-16T09:14:12.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.791+0000 7f6b53fff700 1 --2- 172.21.15.112:0/3459775540 >> [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464] conn(0x7f6b400ac100 0x7f6b400ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:12.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.791+0000 7f6b537fe700 1 --2- 172.21.15.112:0/3459775540 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6b400a2920 0x7f6b400a4de0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6b44000c00 tx=0x7f6b44005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:12.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.792+0000 7f6b53fff700 1 --2- 172.21.15.112:0/3459775540 >> [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464] conn(0x7f6b400ac100 0x7f6b400ae520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:12.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.792+0000 7f6b517fa700 1 -- 172.21.15.112:0/3459775540 <== osd.3 v2:172.21.15.112:6810/3090522464 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b3c011e50 con 0x7f6b400ac100 2024-03-16T09:14:12.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.809+0000 7f6b5a019700 1 -- 172.21.15.112:0/3459775540 --> [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6b54062de0 con 0x7f6b400ac100 2024-03-16T09:14:12.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.810+0000 7f6b517fa700 1 -- 172.21.15.112:0/3459775540 <== osd.3 v2:172.21.15.112:6810/3090522464 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6b54062de0 con 0x7f6b400ac100 2024-03-16T09:14:12.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.810+0000 7f6b5a019700 1 -- 172.21.15.112:0/3459775540 >> [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464] conn(0x7f6b400ac100 msgr2=0x7f6b400ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:12.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.810+0000 7f6b5a019700 1 --2- 172.21.15.112:0/3459775540 >> [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464] conn(0x7f6b400ac100 0x7f6b400ae520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:12.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.810+0000 7f6b5a019700 1 -- 172.21.15.112:0/3459775540 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6b400a2920 msgr2=0x7f6b400a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:12.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.810+0000 7f6b5a019700 1 --2- 172.21.15.112:0/3459775540 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6b400a2920 0x7f6b400a4de0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6b44000c00 tx=0x7f6b44005fb0 comp rx=0 tx=0).stop 2024-03-16T09:14:12.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.810+0000 7f6b5a019700 1 -- 172.21.15.112:0/3459775540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b54101820 msgr2=0x7f6b54102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:12.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.810+0000 7f6b5a019700 1 --2- 172.21.15.112:0/3459775540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6b54101820 0x7f6b54102330 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009fd0 tx=0x7f6b3c00ed10 comp rx=0 tx=0).stop 2024-03-16T09:14:12.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.810+0000 7f6b53fff700 1 -- 172.21.15.112:0/3459775540 reap_dead start 2024-03-16T09:14:12.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.810+0000 7f6b5a019700 1 -- 172.21.15.112:0/3459775540 shutdown_connections 2024-03-16T09:14:12.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.810+0000 7f6b5a019700 1 -- 172.21.15.112:0/3459775540 >> 172.21.15.112:0/3459775540 conn(0x7f6b54076510 msgr2=0x7f6b540fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:12.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.810+0000 7f6b5a019700 1 -- 172.21.15.112:0/3459775540 shutdown_connections 2024-03-16T09:14:12.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:12.810+0000 7f6b5a019700 1 -- 172.21.15.112:0/3459775540 wait complete. 2024-03-16T09:14:13.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.865+0000 7fcc5e329700 1 -- 172.21.15.112:0/377958690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc58108320 msgr2=0x7fcc58108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:13.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.865+0000 7fcc5e329700 1 --2- 172.21.15.112:0/377958690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc58108320 0x7fcc58108700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc480099c0 tx=0x7fcc48009cd0 comp rx=0 tx=0).stop 2024-03-16T09:14:13.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.865+0000 7fcc5e329700 1 -- 172.21.15.112:0/377958690 shutdown_connections 2024-03-16T09:14:13.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.865+0000 7fcc5e329700 1 --2- 172.21.15.112:0/377958690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc5806a920 0x7fcc5810f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:13.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.865+0000 7fcc5e329700 1 --2- 172.21.15.112:0/377958690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc58108320 0x7fcc58108700 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:13.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.865+0000 7fcc5e329700 1 --2- 172.21.15.112:0/377958690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc58069f80 0x7fcc5806a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:13.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.865+0000 7fcc5e329700 1 -- 172.21.15.112:0/377958690 >> 172.21.15.112:0/377958690 conn(0x7fcc58078100 msgr2=0x7fcc58078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:13.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.865+0000 7fcc5e329700 1 -- 172.21.15.112:0/377958690 shutdown_connections 2024-03-16T09:14:13.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.865+0000 7fcc5e329700 1 -- 172.21.15.112:0/377958690 wait complete. 2024-03-16T09:14:13.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.866+0000 7fcc5e329700 1 Processor -- start 2024-03-16T09:14:13.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.866+0000 7fcc5e329700 1 -- start start 2024-03-16T09:14:13.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.866+0000 7fcc5e329700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc58069f80 0x7fcc5819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:13.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.866+0000 7fcc5e329700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc5806a920 0x7fcc5819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:13.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.866+0000 7fcc5e329700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc58108320 0x7fcc581988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:13.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.866+0000 7fcc5e329700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcc581126c0 con 0x7fcc5806a920 2024-03-16T09:14:13.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.866+0000 7fcc5e329700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcc58112540 con 0x7fcc58069f80 2024-03-16T09:14:13.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.866+0000 7fcc5e329700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcc58112840 con 0x7fcc58108320 2024-03-16T09:14:13.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.866+0000 7fcc577fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc5806a920 0x7fcc5819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:13.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.866+0000 7fcc57fff700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc58069f80 0x7fcc5819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:13.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.866+0000 7fcc577fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc5806a920 0x7fcc5819ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50056/0 (socket says 172.21.15.112:50056) 2024-03-16T09:14:13.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.866+0000 7fcc577fe700 1 -- 172.21.15.112:0/4151976645 learned_addr learned my addr 172.21.15.112:0/4151976645 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:13.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.866+0000 7fcc5c8c6700 1 --2- 172.21.15.112:0/4151976645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc58108320 0x7fcc581988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:13.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.867+0000 7fcc577fe700 1 -- 172.21.15.112:0/4151976645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc58108320 msgr2=0x7fcc581988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:13.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.867+0000 7fcc577fe700 1 --2- 172.21.15.112:0/4151976645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc58108320 0x7fcc581988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:13.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.867+0000 7fcc577fe700 1 -- 172.21.15.112:0/4151976645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc58069f80 msgr2=0x7fcc5819e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:13.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.867+0000 7fcc577fe700 1 --2- 172.21.15.112:0/4151976645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc58069f80 0x7fcc5819e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:13.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.867+0000 7fcc577fe700 1 -- 172.21.15.112:0/4151976645 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc58199020 con 0x7fcc5806a920 2024-03-16T09:14:13.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.867+0000 7fcc57fff700 1 --2- 172.21.15.112:0/4151976645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc58069f80 0x7fcc5819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:13.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.867+0000 7fcc577fe700 1 --2- 172.21.15.112:0/4151976645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc5806a920 0x7fcc5819ec40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00e9c0 tx=0x7fcc4c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:13.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.867+0000 7fcc557fa700 1 -- 172.21.15.112:0/4151976645 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c00cbd0 con 0x7fcc5806a920 2024-03-16T09:14:13.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.867+0000 7fcc557fa700 1 -- 172.21.15.112:0/4151976645 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc4c00ee60 con 0x7fcc5806a920 2024-03-16T09:14:13.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.867+0000 7fcc5e329700 1 -- 172.21.15.112:0/4151976645 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc58199310 con 0x7fcc5806a920 2024-03-16T09:14:13.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.867+0000 7fcc557fa700 1 -- 172.21.15.112:0/4151976645 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c010430 con 0x7fcc5806a920 2024-03-16T09:14:13.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.867+0000 7fcc5e329700 1 -- 172.21.15.112:0/4151976645 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc581a55d0 con 0x7fcc5806a920 2024-03-16T09:14:13.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.868+0000 7fcc5e329700 1 -- 172.21.15.112:0/4151976645 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcc58062a60 con 0x7fcc5806a920 2024-03-16T09:14:13.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.869+0000 7fcc557fa700 1 -- 172.21.15.112:0/4151976645 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fcc4c004750 con 0x7fcc5806a920 2024-03-16T09:14:13.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.870+0000 7fcc557fa700 1 --2- 172.21.15.112:0/4151976645 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcc400a29a0 0x7fcc400a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:13.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.870+0000 7fcc57fff700 1 --2- 172.21.15.112:0/4151976645 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcc400a29a0 0x7fcc400a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:13.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.870+0000 7fcc557fa700 1 -- 172.21.15.112:0/4151976645 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fcc4c0ced40 con 0x7fcc5806a920 2024-03-16T09:14:13.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.870+0000 7fcc557fa700 1 --2- 172.21.15.112:0/4151976645 >> [v2:172.21.15.159:6816/2484095468,v1:172.21.15.159:6817/2484095468] conn(0x7fcc400ac180 0x7fcc400ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:13.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.870+0000 7fcc557fa700 1 -- 172.21.15.112:0/4151976645 --> [v2:172.21.15.159:6816/2484095468,v1:172.21.15.159:6817/2484095468] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcc4c00ae50 con 0x7fcc400ac180 2024-03-16T09:14:13.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.870+0000 7fcc557fa700 1 -- 172.21.15.112:0/4151976645 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fcc4c0d1360 con 0x7fcc5806a920 2024-03-16T09:14:13.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.870+0000 7fcc5c8c6700 1 --2- 172.21.15.112:0/4151976645 >> [v2:172.21.15.159:6816/2484095468,v1:172.21.15.159:6817/2484095468] conn(0x7fcc400ac180 0x7fcc400ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:13.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.870+0000 7fcc57fff700 1 --2- 172.21.15.112:0/4151976645 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcc400a29a0 0x7fcc400a4e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcc58199a00 tx=0x7fcc48005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:13.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.870+0000 7fcc5c8c6700 1 --2- 172.21.15.112:0/4151976645 >> [v2:172.21.15.159:6816/2484095468,v1:172.21.15.159:6817/2484095468] conn(0x7fcc400ac180 0x7fcc400ae5a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:13.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.871+0000 7fcc557fa700 1 -- 172.21.15.112:0/4151976645 <== osd.7 v2:172.21.15.159:6816/2484095468 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcc4c00ae50 con 0x7fcc400ac180 2024-03-16T09:14:13.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.887+0000 7fcc5e329700 1 -- 172.21.15.112:0/4151976645 --> [v2:172.21.15.159:6816/2484095468,v1:172.21.15.159:6817/2484095468] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcc58062ee0 con 0x7fcc400ac180 2024-03-16T09:14:13.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.888+0000 7fcc557fa700 1 -- 172.21.15.112:0/4151976645 <== osd.7 v2:172.21.15.159:6816/2484095468 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcc58062ee0 con 0x7fcc400ac180 2024-03-16T09:14:13.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.888+0000 7fcc5e329700 1 -- 172.21.15.112:0/4151976645 >> [v2:172.21.15.159:6816/2484095468,v1:172.21.15.159:6817/2484095468] conn(0x7fcc400ac180 msgr2=0x7fcc400ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:13.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.888+0000 7fcc5e329700 1 --2- 172.21.15.112:0/4151976645 >> [v2:172.21.15.159:6816/2484095468,v1:172.21.15.159:6817/2484095468] conn(0x7fcc400ac180 0x7fcc400ae5a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:13.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.888+0000 7fcc5e329700 1 -- 172.21.15.112:0/4151976645 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcc400a29a0 msgr2=0x7fcc400a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:13.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.888+0000 7fcc5e329700 1 --2- 172.21.15.112:0/4151976645 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcc400a29a0 0x7fcc400a4e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcc58199a00 tx=0x7fcc48005a90 comp rx=0 tx=0).stop 2024-03-16T09:14:13.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.888+0000 7fcc5e329700 1 -- 172.21.15.112:0/4151976645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc5806a920 msgr2=0x7fcc5819ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:13.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.888+0000 7fcc5e329700 1 --2- 172.21.15.112:0/4151976645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc5806a920 0x7fcc5819ec40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00e9c0 tx=0x7fcc4c00ecd0 comp rx=0 tx=0).stop 2024-03-16T09:14:13.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.888+0000 7fcc5c8c6700 1 -- 172.21.15.112:0/4151976645 reap_dead start 2024-03-16T09:14:13.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.888+0000 7fcc5e329700 1 -- 172.21.15.112:0/4151976645 shutdown_connections 2024-03-16T09:14:13.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.888+0000 7fcc5e329700 1 -- 172.21.15.112:0/4151976645 >> 172.21.15.112:0/4151976645 conn(0x7fcc58078100 msgr2=0x7fcc580ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:13.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.888+0000 7fcc5e329700 1 -- 172.21.15.112:0/4151976645 shutdown_connections 2024-03-16T09:14:13.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:13.888+0000 7fcc5e329700 1 -- 172.21.15.112:0/4151976645 wait complete. 2024-03-16T09:14:14.212 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:13 smithi112 ceph-mon[32307]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:14:14.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:13 smithi159 ceph-mon[36739]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:14:14.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:13 smithi194 ceph-mon[36729]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:14:14.705 INFO:teuthology.orchestra.run.smithi112.stdout:141733920823 2024-03-16T09:14:14.706 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-16T09:14:15.361 INFO:teuthology.orchestra.run.smithi112.stdout:163208757299 2024-03-16T09:14:15.361 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-16T09:14:15.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.632+0000 7f465ca33700 1 -- 172.21.15.112:0/3915951808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4658108300 msgr2=0x7f46581086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:15.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.632+0000 7f465ca33700 1 --2- 172.21.15.112:0/3915951808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4658108300 0x7f46581086e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f46480099c0 tx=0x7f4648009cd0 comp rx=0 tx=0).stop 2024-03-16T09:14:15.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.632+0000 7f465ca33700 1 -- 172.21.15.112:0/3915951808 shutdown_connections 2024-03-16T09:14:15.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.632+0000 7f465ca33700 1 --2- 172.21.15.112:0/3915951808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4658102ce0 0x7f465810f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:15.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.632+0000 7f465ca33700 1 --2- 172.21.15.112:0/3915951808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4658108300 0x7f46581086e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:15.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.632+0000 7f465ca33700 1 --2- 172.21.15.112:0/3915951808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4658102320 0x7f46581027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:15.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.632+0000 7f465ca33700 1 -- 172.21.15.112:0/3915951808 >> 172.21.15.112:0/3915951808 conn(0x7f4658079400 msgr2=0x7f4658079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:15.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.632+0000 7f465ca33700 1 -- 172.21.15.112:0/3915951808 shutdown_connections 2024-03-16T09:14:15.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.632+0000 7f465ca33700 1 -- 172.21.15.112:0/3915951808 wait complete. 2024-03-16T09:14:15.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.633+0000 7f465ca33700 1 Processor -- start 2024-03-16T09:14:15.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.633+0000 7f465ca33700 1 -- start start 2024-03-16T09:14:15.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.633+0000 7f465ca33700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4658102320 0x7f465819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:15.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.633+0000 7f465ca33700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4658102ce0 0x7f465819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:15.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.633+0000 7f465ca33700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4658108300 0x7f4658198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:15.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.633+0000 7f465ca33700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46581126c0 con 0x7f4658108300 2024-03-16T09:14:15.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.633+0000 7f465ca33700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4658112540 con 0x7f4658102ce0 2024-03-16T09:14:15.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.633+0000 7f465ca33700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4658112840 con 0x7f4658102320 2024-03-16T09:14:15.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.634+0000 7f4656d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4658108300 0x7f4658198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:15.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.634+0000 7f4656d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4658108300 0x7f4658198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50082/0 (socket says 172.21.15.112:50082) 2024-03-16T09:14:15.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.634+0000 7f465659c700 1 --2- 172.21.15.112:0/416917746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4658102320 0x7f465819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:15.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.634+0000 7f4656d9d700 1 -- 172.21.15.112:0/416917746 learned_addr learned my addr 172.21.15.112:0/416917746 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:15.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.634+0000 7f4655d9b700 1 --2- 172.21.15.112:0/416917746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4658102ce0 0x7f465819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:15.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.634+0000 7f4656d9d700 1 -- 172.21.15.112:0/416917746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4658102320 msgr2=0x7f465819e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:15.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.634+0000 7f4656d9d700 1 --2- 172.21.15.112:0/416917746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4658102320 0x7f465819e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:15.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.634+0000 7f4656d9d700 1 -- 172.21.15.112:0/416917746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4658102ce0 msgr2=0x7f465819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:15.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.634+0000 7f4656d9d700 1 --2- 172.21.15.112:0/416917746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4658102ce0 0x7f465819eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:15.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.634+0000 7f4656d9d700 1 -- 172.21.15.112:0/416917746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4658199080 con 0x7f4658108300 2024-03-16T09:14:15.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.634+0000 7f465659c700 1 --2- 172.21.15.112:0/416917746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4658102320 0x7f465819e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:15.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.634+0000 7f4655d9b700 1 --2- 172.21.15.112:0/416917746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4658102ce0 0x7f465819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:15.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.634+0000 7f4656d9d700 1 --2- 172.21.15.112:0/416917746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4658108300 0x7f4658198920 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f465000ef30 tx=0x7f465000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:15.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.634+0000 7f46477fe700 1 -- 172.21.15.112:0/416917746 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46500092e0 con 0x7f4658108300 2024-03-16T09:14:15.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.634+0000 7f465ca33700 1 -- 172.21.15.112:0/416917746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4658199310 con 0x7f4658108300 2024-03-16T09:14:15.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.634+0000 7f46477fe700 1 -- 172.21.15.112:0/416917746 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4650004bb0 con 0x7f4658108300 2024-03-16T09:14:15.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.634+0000 7f46477fe700 1 -- 172.21.15.112:0/416917746 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650004280 con 0x7f4658108300 2024-03-16T09:14:15.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.634+0000 7f465ca33700 1 -- 172.21.15.112:0/416917746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46581a5630 con 0x7f4658108300 2024-03-16T09:14:15.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.635+0000 7f465ca33700 1 -- 172.21.15.112:0/416917746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4658062a60 con 0x7f4658108300 2024-03-16T09:14:15.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.636+0000 7f46477fe700 1 -- 172.21.15.112:0/416917746 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f465001b030 con 0x7f4658108300 2024-03-16T09:14:15.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.637+0000 7f46477fe700 1 --2- 172.21.15.112:0/416917746 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f46400a2920 0x7f46400a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:15.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.637+0000 7f465659c700 1 --2- 172.21.15.112:0/416917746 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f46400a2920 0x7f46400a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:15.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.637+0000 7f46477fe700 1 -- 172.21.15.112:0/416917746 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f46500cee10 con 0x7f4658108300 2024-03-16T09:14:15.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.637+0000 7f46477fe700 1 --2- 172.21.15.112:0/416917746 >> [v2:172.21.15.159:6800/140175088,v1:172.21.15.159:6801/140175088] conn(0x7f46400ac100 0x7f46400ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:15.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.637+0000 7f46477fe700 1 -- 172.21.15.112:0/416917746 --> [v2:172.21.15.159:6800/140175088,v1:172.21.15.159:6801/140175088] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f465001fe50 con 0x7f46400ac100 2024-03-16T09:14:15.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.637+0000 7f46477fe700 1 -- 172.21.15.112:0/416917746 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f46500cf2f0 con 0x7f4658108300 2024-03-16T09:14:15.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.637+0000 7f4655d9b700 1 --2- 172.21.15.112:0/416917746 >> [v2:172.21.15.159:6800/140175088,v1:172.21.15.159:6801/140175088] conn(0x7f46400ac100 0x7f46400ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:15.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.637+0000 7f465659c700 1 --2- 172.21.15.112:0/416917746 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f46400a2920 0x7f46400a4de0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4658199a60 tx=0x7f4648005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:15.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.638+0000 7f4655d9b700 1 --2- 172.21.15.112:0/416917746 >> [v2:172.21.15.159:6800/140175088,v1:172.21.15.159:6801/140175088] conn(0x7f46400ac100 0x7f46400ae520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:15.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.638+0000 7f46477fe700 1 -- 172.21.15.112:0/416917746 <== osd.1 v2:172.21.15.159:6800/140175088 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f465001fe50 con 0x7f46400ac100 2024-03-16T09:14:15.656 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:14:15.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.657+0000 7f465ca33700 1 -- 172.21.15.112:0/416917746 --> [v2:172.21.15.159:6800/140175088,v1:172.21.15.159:6801/140175088] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4658062ee0 con 0x7f46400ac100 2024-03-16T09:14:15.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.657+0000 7f46477fe700 1 -- 172.21.15.112:0/416917746 <== osd.1 v2:172.21.15.159:6800/140175088 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4658062ee0 con 0x7f46400ac100 2024-03-16T09:14:15.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.658+0000 7f465ca33700 1 -- 172.21.15.112:0/416917746 >> [v2:172.21.15.159:6800/140175088,v1:172.21.15.159:6801/140175088] conn(0x7f46400ac100 msgr2=0x7f46400ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:15.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.658+0000 7f465ca33700 1 --2- 172.21.15.112:0/416917746 >> [v2:172.21.15.159:6800/140175088,v1:172.21.15.159:6801/140175088] conn(0x7f46400ac100 0x7f46400ae520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:15.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.658+0000 7f465ca33700 1 -- 172.21.15.112:0/416917746 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f46400a2920 msgr2=0x7f46400a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:15.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.658+0000 7f465ca33700 1 --2- 172.21.15.112:0/416917746 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f46400a2920 0x7f46400a4de0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4658199a60 tx=0x7f4648005a90 comp rx=0 tx=0).stop 2024-03-16T09:14:15.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.658+0000 7f465ca33700 1 -- 172.21.15.112:0/416917746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4658108300 msgr2=0x7f4658198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:15.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.658+0000 7f465ca33700 1 --2- 172.21.15.112:0/416917746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4658108300 0x7f4658198920 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f465000ef30 tx=0x7f465000c3a0 comp rx=0 tx=0).stop 2024-03-16T09:14:15.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.658+0000 7f4656d9d700 1 -- 172.21.15.112:0/416917746 reap_dead start 2024-03-16T09:14:15.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.658+0000 7f465ca33700 1 -- 172.21.15.112:0/416917746 shutdown_connections 2024-03-16T09:14:15.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.658+0000 7f465ca33700 1 -- 172.21.15.112:0/416917746 >> 172.21.15.112:0/416917746 conn(0x7f4658079400 msgr2=0x7f4658100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:15.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.658+0000 7f465ca33700 1 -- 172.21.15.112:0/416917746 shutdown_connections 2024-03-16T09:14:15.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.658+0000 7f465ca33700 1 -- 172.21.15.112:0/416917746 wait complete. 2024-03-16T09:14:16.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:15 smithi159 ceph-mon[36739]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-16T09:14:16.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:15 smithi112 ceph-mon[32307]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-16T09:14:16.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:15 smithi194 ceph-mon[36729]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-16T09:14:16.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.516+0000 7f483d1c5700 1 -- 172.21.15.112:0/3844075795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4838102ce0 msgr2=0x7f483810f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:16.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.516+0000 7f483d1c5700 1 --2- 172.21.15.112:0/3844075795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4838102ce0 0x7f483810f120 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f483000b0d0 tx=0x7f483000b3e0 comp rx=0 tx=0).stop 2024-03-16T09:14:16.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.516+0000 7f483d1c5700 1 -- 172.21.15.112:0/3844075795 shutdown_connections 2024-03-16T09:14:16.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.516+0000 7f483d1c5700 1 --2- 172.21.15.112:0/3844075795 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4838102ce0 0x7f483810f120 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:16.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.516+0000 7f483d1c5700 1 --2- 172.21.15.112:0/3844075795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4838108300 0x7f48381086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:16.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.516+0000 7f483d1c5700 1 --2- 172.21.15.112:0/3844075795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4838102320 0x7f48381027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:16.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.516+0000 7f483d1c5700 1 -- 172.21.15.112:0/3844075795 >> 172.21.15.112:0/3844075795 conn(0x7f4838079400 msgr2=0x7f4838079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:16.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.516+0000 7f483d1c5700 1 -- 172.21.15.112:0/3844075795 shutdown_connections 2024-03-16T09:14:16.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.516+0000 7f483d1c5700 1 -- 172.21.15.112:0/3844075795 wait complete. 2024-03-16T09:14:16.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.517+0000 7f483d1c5700 1 Processor -- start 2024-03-16T09:14:16.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.518+0000 7f483d1c5700 1 -- start start 2024-03-16T09:14:16.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.518+0000 7f483d1c5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4838102320 0x7f48380745d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:16.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.518+0000 7f483d1c5700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4838102ce0 0x7f4838074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:16.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.518+0000 7f483d1c5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4838108300 0x7f4838075050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:16.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.518+0000 7f483d1c5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48381126c0 con 0x7f4838108300 2024-03-16T09:14:16.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.518+0000 7f483d1c5700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4838112540 con 0x7f4838102ce0 2024-03-16T09:14:16.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.518+0000 7f483d1c5700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4838112840 con 0x7f4838102320 2024-03-16T09:14:16.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.518+0000 7f483759e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4838108300 0x7f4838075050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:16.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.518+0000 7f483759e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4838108300 0x7f4838075050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50104/0 (socket says 172.21.15.112:50104) 2024-03-16T09:14:16.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.518+0000 7f483759e700 1 -- 172.21.15.112:0/3341140176 learned_addr learned my addr 172.21.15.112:0/3341140176 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:16.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.518+0000 7f4836d9d700 1 --2- 172.21.15.112:0/3341140176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4838102320 0x7f48380745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:16.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.518+0000 7f483659c700 1 --2- 172.21.15.112:0/3341140176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4838102ce0 0x7f4838074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:16.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.518+0000 7f483759e700 1 -- 172.21.15.112:0/3341140176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4838102320 msgr2=0x7f48380745d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:16.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.518+0000 7f483759e700 1 --2- 172.21.15.112:0/3341140176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4838102320 0x7f48380745d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:16.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.518+0000 7f483759e700 1 -- 172.21.15.112:0/3341140176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4838102ce0 msgr2=0x7f4838074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:16.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.518+0000 7f483759e700 1 --2- 172.21.15.112:0/3341140176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4838102ce0 0x7f4838074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:16.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.518+0000 7f483759e700 1 -- 172.21.15.112:0/3341140176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48381a6f60 con 0x7f4838108300 2024-03-16T09:14:16.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.518+0000 7f483759e700 1 --2- 172.21.15.112:0/3341140176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4838108300 0x7f4838075050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4830007a50 tx=0x7f4830007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:16.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.518+0000 7f481ffff700 1 -- 172.21.15.112:0/3341140176 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483001f070 con 0x7f4838108300 2024-03-16T09:14:16.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.519+0000 7f483d1c5700 1 -- 172.21.15.112:0/3341140176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48381a71f0 con 0x7f4838108300 2024-03-16T09:14:16.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.519+0000 7f481ffff700 1 -- 172.21.15.112:0/3341140176 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4830003c00 con 0x7f4838108300 2024-03-16T09:14:16.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.519+0000 7f481ffff700 1 -- 172.21.15.112:0/3341140176 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483001b6a0 con 0x7f4838108300 2024-03-16T09:14:16.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.519+0000 7f483d1c5700 1 -- 172.21.15.112:0/3341140176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48381a7640 con 0x7f4838108300 2024-03-16T09:14:16.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.519+0000 7f483d1c5700 1 -- 172.21.15.112:0/3341140176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4838062a60 con 0x7f4838108300 2024-03-16T09:14:16.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.521+0000 7f481ffff700 1 -- 172.21.15.112:0/3341140176 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f4830022070 con 0x7f4838108300 2024-03-16T09:14:16.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.521+0000 7f481ffff700 1 --2- 172.21.15.112:0/3341140176 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f48200a2920 0x7f48200a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:16.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.521+0000 7f4836d9d700 1 --2- 172.21.15.112:0/3341140176 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f48200a2920 0x7f48200a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:16.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.521+0000 7f481ffff700 1 -- 172.21.15.112:0/3341140176 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f483000f020 con 0x7f4838108300 2024-03-16T09:14:16.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.521+0000 7f481ffff700 1 --2- 172.21.15.112:0/3341140176 >> [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115] conn(0x7f48200ac100 0x7f48200ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:16.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.522+0000 7f481ffff700 1 -- 172.21.15.112:0/3341140176 --> [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48300020b0 con 0x7f48200ac100 2024-03-16T09:14:16.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.522+0000 7f481ffff700 1 -- 172.21.15.112:0/3341140176 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f48300d01e0 con 0x7f4838108300 2024-03-16T09:14:16.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.522+0000 7f483659c700 1 --2- 172.21.15.112:0/3341140176 >> [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115] conn(0x7f48200ac100 0x7f48200ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:16.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.522+0000 7f4836d9d700 1 --2- 172.21.15.112:0/3341140176 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f48200a2920 0x7f48200a4de0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4828007b60 tx=0x7f48280073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:16.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.522+0000 7f483659c700 1 --2- 172.21.15.112:0/3341140176 >> [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115] conn(0x7f48200ac100 0x7f48200ae520 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:16.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.523+0000 7f481ffff700 1 -- 172.21.15.112:0/3341140176 <== osd.0 v2:172.21.15.112:6802/1848821115 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f48300020b0 con 0x7f48200ac100 2024-03-16T09:14:16.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.540+0000 7f483d1c5700 1 -- 172.21.15.112:0/3341140176 --> [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4838062ee0 con 0x7f48200ac100 2024-03-16T09:14:16.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.540+0000 7f481ffff700 1 -- 172.21.15.112:0/3341140176 <== osd.0 v2:172.21.15.112:6802/1848821115 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4838062ee0 con 0x7f48200ac100 2024-03-16T09:14:16.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.541+0000 7f483d1c5700 1 -- 172.21.15.112:0/3341140176 >> [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115] conn(0x7f48200ac100 msgr2=0x7f48200ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:16.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.541+0000 7f483d1c5700 1 --2- 172.21.15.112:0/3341140176 >> [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115] conn(0x7f48200ac100 0x7f48200ae520 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:16.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.541+0000 7f483d1c5700 1 -- 172.21.15.112:0/3341140176 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f48200a2920 msgr2=0x7f48200a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:16.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.541+0000 7f483d1c5700 1 --2- 172.21.15.112:0/3341140176 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f48200a2920 0x7f48200a4de0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4828007b60 tx=0x7f48280073d0 comp rx=0 tx=0).stop 2024-03-16T09:14:16.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.541+0000 7f483d1c5700 1 -- 172.21.15.112:0/3341140176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4838108300 msgr2=0x7f4838075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:16.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.541+0000 7f483d1c5700 1 --2- 172.21.15.112:0/3341140176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4838108300 0x7f4838075050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4830007a50 tx=0x7f4830007a80 comp rx=0 tx=0).stop 2024-03-16T09:14:16.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.541+0000 7f483759e700 1 -- 172.21.15.112:0/3341140176 reap_dead start 2024-03-16T09:14:16.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.541+0000 7f483d1c5700 1 -- 172.21.15.112:0/3341140176 shutdown_connections 2024-03-16T09:14:16.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.541+0000 7f483d1c5700 1 -- 172.21.15.112:0/3341140176 >> 172.21.15.112:0/3341140176 conn(0x7f4838079400 msgr2=0x7f4838100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:16.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.541+0000 7f483d1c5700 1 -- 172.21.15.112:0/3341140176 shutdown_connections 2024-03-16T09:14:16.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:16.541+0000 7f483d1c5700 1 -- 172.21.15.112:0/3341140176 wait complete. 2024-03-16T09:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.534+0000 7f9c33ae6700 1 -- 172.21.15.112:0/695932457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c2c104ec0 msgr2=0x7f9c2c111380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.534+0000 7f9c33ae6700 1 --2- 172.21.15.112:0/695932457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c2c104ec0 0x7f9c2c111380 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9c2400b0d0 tx=0x7f9c2400b3e0 comp rx=0 tx=0).stop 2024-03-16T09:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.534+0000 7f9c33ae6700 1 -- 172.21.15.112:0/695932457 shutdown_connections 2024-03-16T09:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.534+0000 7f9c33ae6700 1 --2- 172.21.15.112:0/695932457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c2c10a5b0 0x7f9c2c10a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.534+0000 7f9c33ae6700 1 --2- 172.21.15.112:0/695932457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c2c104ec0 0x7f9c2c111380 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.534+0000 7f9c33ae6700 1 --2- 172.21.15.112:0/695932457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c2c104500 0x7f9c2c104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.534+0000 7f9c33ae6700 1 -- 172.21.15.112:0/695932457 >> 172.21.15.112:0/695932457 conn(0x7f9c2c0fffd0 msgr2=0x7f9c2c1023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.535+0000 7f9c33ae6700 1 -- 172.21.15.112:0/695932457 shutdown_connections 2024-03-16T09:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.535+0000 7f9c33ae6700 1 -- 172.21.15.112:0/695932457 wait complete. 2024-03-16T09:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.536+0000 7f9c33ae6700 1 Processor -- start 2024-03-16T09:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.536+0000 7f9c33ae6700 1 -- start start 2024-03-16T09:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.537+0000 7f9c33ae6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c2c104500 0x7f9c2c1a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.537+0000 7f9c33ae6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c2c104ec0 0x7f9c2c1a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.537+0000 7f9c31882700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c2c104500 0x7f9c2c1a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.537+0000 7f9c31882700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c2c104500 0x7f9c2c1a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:44936/0 (socket says 172.21.15.112:44936) 2024-03-16T09:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.537+0000 7f9c31882700 1 -- 172.21.15.112:0/1884453544 learned_addr learned my addr 172.21.15.112:0/1884453544 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.537+0000 7f9c33ae6700 1 --2- 172.21.15.112:0/1884453544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c2c10a5b0 0x7f9c2c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.537+0000 7f9c33ae6700 1 -- 172.21.15.112:0/1884453544 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2c114940 con 0x7f9c2c104ec0 2024-03-16T09:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.537+0000 7f9c33ae6700 1 -- 172.21.15.112:0/1884453544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2c1147c0 con 0x7f9c2c10a5b0 2024-03-16T09:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.537+0000 7f9c33ae6700 1 -- 172.21.15.112:0/1884453544 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2c114ac0 con 0x7f9c2c104500 2024-03-16T09:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.537+0000 7f9c32083700 1 --2- 172.21.15.112:0/1884453544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c2c10a5b0 0x7f9c2c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.538+0000 7f9c31081700 1 --2- 172.21.15.112:0/1884453544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c2c104ec0 0x7f9c2c1a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.538+0000 7f9c32083700 1 -- 172.21.15.112:0/1884453544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c2c104500 msgr2=0x7f9c2c1a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.538+0000 7f9c32083700 1 --2- 172.21.15.112:0/1884453544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c2c104500 0x7f9c2c1a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.538+0000 7f9c32083700 1 -- 172.21.15.112:0/1884453544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c2c104ec0 msgr2=0x7f9c2c1a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.538+0000 7f9c32083700 1 --2- 172.21.15.112:0/1884453544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c2c104ec0 0x7f9c2c1a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.538+0000 7f9c32083700 1 -- 172.21.15.112:0/1884453544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2c19b1e0 con 0x7f9c2c10a5b0 2024-03-16T09:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.538+0000 7f9c32083700 1 --2- 172.21.15.112:0/1884453544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c2c10a5b0 0x7f9c2c19aa80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009cd0 tx=0x7f9c24007a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.539+0000 7f9c22ffd700 1 -- 172.21.15.112:0/1884453544 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c24003c00 con 0x7f9c2c10a5b0 2024-03-16T09:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.539+0000 7f9c22ffd700 1 -- 172.21.15.112:0/1884453544 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9c24022070 con 0x7f9c2c10a5b0 2024-03-16T09:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.539+0000 7f9c22ffd700 1 -- 172.21.15.112:0/1884453544 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2401b6c0 con 0x7f9c2c10a5b0 2024-03-16T09:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.539+0000 7f9c33ae6700 1 -- 172.21.15.112:0/1884453544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c2c19b470 con 0x7f9c2c10a5b0 2024-03-16T09:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.539+0000 7f9c33ae6700 1 -- 172.21.15.112:0/1884453544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c2c1a7740 con 0x7f9c2c10a5b0 2024-03-16T09:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.540+0000 7f9c33ae6700 1 -- 172.21.15.112:0/1884453544 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9c2c101eb0 con 0x7f9c2c10a5b0 2024-03-16T09:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.542+0000 7f9c22ffd700 1 -- 172.21.15.112:0/1884453544 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f9c2401d470 con 0x7f9c2c10a5b0 2024-03-16T09:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.542+0000 7f9c22ffd700 1 --2- 172.21.15.112:0/1884453544 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9c180a29a0 0x7f9c180a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.542+0000 7f9c22ffd700 1 -- 172.21.15.112:0/1884453544 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f9c2400f020 con 0x7f9c2c10a5b0 2024-03-16T09:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.542+0000 7f9c22ffd700 1 --2- 172.21.15.112:0/1884453544 >> [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860] conn(0x7f9c180ac180 0x7f9c180ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:17.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.542+0000 7f9c22ffd700 1 -- 172.21.15.112:0/1884453544 --> [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9c180aec50 con 0x7f9c180ac180 2024-03-16T09:14:17.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.542+0000 7f9c22ffd700 1 -- 172.21.15.112:0/1884453544 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f9c240cffc0 con 0x7f9c2c10a5b0 2024-03-16T09:14:17.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.543+0000 7f9c31882700 1 --2- 172.21.15.112:0/1884453544 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9c180a29a0 0x7f9c180a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:17.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.543+0000 7f9c31081700 1 --2- 172.21.15.112:0/1884453544 >> [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860] conn(0x7f9c180ac180 0x7f9c180ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:17.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.543+0000 7f9c31081700 1 --2- 172.21.15.112:0/1884453544 >> [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860] conn(0x7f9c180ac180 0x7f9c180ae5a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:17.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.543+0000 7f9c31882700 1 --2- 172.21.15.112:0/1884453544 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9c180a29a0 0x7f9c180a4e60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c1094e0 tx=0x7f9c28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:17.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.544+0000 7f9c22ffd700 1 -- 172.21.15.112:0/1884453544 <== osd.9 v2:172.21.15.112:6826/952596860 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9c180aec50 con 0x7f9c180ac180 2024-03-16T09:14:17.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.560+0000 7f9c33ae6700 1 -- 172.21.15.112:0/1884453544 --> [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9c2c105c20 con 0x7f9c180ac180 2024-03-16T09:14:17.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.560+0000 7f9c22ffd700 1 -- 172.21.15.112:0/1884453544 <== osd.9 v2:172.21.15.112:6826/952596860 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9c2c105c20 con 0x7f9c180ac180 2024-03-16T09:14:17.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.561+0000 7f9c33ae6700 1 -- 172.21.15.112:0/1884453544 >> [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860] conn(0x7f9c180ac180 msgr2=0x7f9c180ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:17.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.561+0000 7f9c33ae6700 1 --2- 172.21.15.112:0/1884453544 >> [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860] conn(0x7f9c180ac180 0x7f9c180ae5a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:17.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.561+0000 7f9c33ae6700 1 -- 172.21.15.112:0/1884453544 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9c180a29a0 msgr2=0x7f9c180a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:17.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.561+0000 7f9c33ae6700 1 --2- 172.21.15.112:0/1884453544 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9c180a29a0 0x7f9c180a4e60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c1094e0 tx=0x7f9c28009260 comp rx=0 tx=0).stop 2024-03-16T09:14:17.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.561+0000 7f9c33ae6700 1 -- 172.21.15.112:0/1884453544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c2c10a5b0 msgr2=0x7f9c2c19aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:17.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.561+0000 7f9c33ae6700 1 --2- 172.21.15.112:0/1884453544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c2c10a5b0 0x7f9c2c19aa80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009cd0 tx=0x7f9c24007a20 comp rx=0 tx=0).stop 2024-03-16T09:14:17.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.561+0000 7f9c32083700 1 -- 172.21.15.112:0/1884453544 reap_dead start 2024-03-16T09:14:17.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.561+0000 7f9c33ae6700 1 -- 172.21.15.112:0/1884453544 shutdown_connections 2024-03-16T09:14:17.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.561+0000 7f9c33ae6700 1 -- 172.21.15.112:0/1884453544 >> 172.21.15.112:0/1884453544 conn(0x7f9c2c0fffd0 msgr2=0x7f9c2c101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:17.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.561+0000 7f9c33ae6700 1 -- 172.21.15.112:0/1884453544 shutdown_connections 2024-03-16T09:14:17.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:15.561+0000 7f9c33ae6700 1 -- 172.21.15.112:0/1884453544 wait complete. 2024-03-16T09:14:18.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:17 smithi159 ceph-mon[36739]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-16T09:14:18.290 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:17 smithi112 ceph-mon[32307]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-16T09:14:18.325 INFO:teuthology.orchestra.run.smithi112.stdout:287762808863 2024-03-16T09:14:18.326 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-16T09:14:18.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:17 smithi194 ceph-mon[36729]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-16T09:14:18.912 INFO:teuthology.orchestra.run.smithi112.stdout:55834574919 2024-03-16T09:14:18.913 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-16T09:14:19.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.140+0000 7fdb0bbc4700 1 -- 172.21.15.112:0/185644609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb041011b0 msgr2=0x7fdb04101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:19.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.140+0000 7fdb0bbc4700 1 --2- 172.21.15.112:0/185644609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb041011b0 0x7fdb04101630 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009990 tx=0x7fdaf4009ca0 comp rx=0 tx=0).stop 2024-03-16T09:14:19.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.140+0000 7fdb0bbc4700 1 -- 172.21.15.112:0/185644609 shutdown_connections 2024-03-16T09:14:19.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.140+0000 7fdb0bbc4700 1 --2- 172.21.15.112:0/185644609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb04107260 0x7fdb04107640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:19.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.140+0000 7fdb0bbc4700 1 --2- 172.21.15.112:0/185644609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb04101b70 0x7fdb0410e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:19.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.140+0000 7fdb0bbc4700 1 --2- 172.21.15.112:0/185644609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb041011b0 0x7fdb04101630 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:19.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.140+0000 7fdb0bbc4700 1 -- 172.21.15.112:0/185644609 >> 172.21.15.112:0/185644609 conn(0x7fdb040781e0 msgr2=0x7fdb040785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:19.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.140+0000 7fdb0bbc4700 1 -- 172.21.15.112:0/185644609 shutdown_connections 2024-03-16T09:14:19.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.140+0000 7fdb0bbc4700 1 -- 172.21.15.112:0/185644609 wait complete. 2024-03-16T09:14:19.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.140+0000 7fdb0bbc4700 1 Processor -- start 2024-03-16T09:14:19.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.140+0000 7fdb0bbc4700 1 -- start start 2024-03-16T09:14:19.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdb0bbc4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb041011b0 0x7fdb0419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:19.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdb0bbc4700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb04101b70 0x7fdb0419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:19.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdb0bbc4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb04107260 0x7fdb04198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:19.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdb0bbc4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdb041115e0 con 0x7fdb04107260 2024-03-16T09:14:19.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdb0bbc4700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdb04111460 con 0x7fdb04101b70 2024-03-16T09:14:19.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdb0bbc4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdb04111760 con 0x7fdb041011b0 2024-03-16T09:14:19.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdb09960700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb041011b0 0x7fdb0419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:19.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdb0a161700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb04107260 0x7fdb04198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:19.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdb09960700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb041011b0 0x7fdb0419e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:45008/0 (socket says 172.21.15.112:45008) 2024-03-16T09:14:19.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdb09960700 1 -- 172.21.15.112:0/1711046406 learned_addr learned my addr 172.21.15.112:0/1711046406 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:19.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdb0915f700 1 --2- 172.21.15.112:0/1711046406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb04101b70 0x7fdb0419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:19.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdb09960700 1 -- 172.21.15.112:0/1711046406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb04101b70 msgr2=0x7fdb0419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdb09960700 1 --2- 172.21.15.112:0/1711046406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb04101b70 0x7fdb0419ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdb09960700 1 -- 172.21.15.112:0/1711046406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb04107260 msgr2=0x7fdb04198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdb09960700 1 --2- 172.21.15.112:0/1711046406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb04107260 0x7fdb04198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdb09960700 1 -- 172.21.15.112:0/1711046406 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb041990a0 con 0x7fdb041011b0 2024-03-16T09:14:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdb09960700 1 --2- 172.21.15.112:0/1711046406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb041011b0 0x7fdb0419e730 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000e9a0 tx=0x7fdb0000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdafaffd700 1 -- 172.21.15.112:0/1711046406 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0000ca10 con 0x7fdb041011b0 2024-03-16T09:14:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdb0bbc4700 1 -- 172.21.15.112:0/1711046406 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb04199390 con 0x7fdb041011b0 2024-03-16T09:14:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdafaffd700 1 -- 172.21.15.112:0/1711046406 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdb0000cbb0 con 0x7fdb041011b0 2024-03-16T09:14:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdafaffd700 1 -- 172.21.15.112:0/1711046406 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00018710 con 0x7fdb041011b0 2024-03-16T09:14:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.141+0000 7fdb0bbc4700 1 -- 172.21.15.112:0/1711046406 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb041a5600 con 0x7fdb041011b0 2024-03-16T09:14:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.142+0000 7fdb0bbc4700 1 -- 172.21.15.112:0/1711046406 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb040fffc0 con 0x7fdb041011b0 2024-03-16T09:14:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.143+0000 7fdafaffd700 1 -- 172.21.15.112:0/1711046406 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fdb00010430 con 0x7fdb041011b0 2024-03-16T09:14:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.144+0000 7fdafaffd700 1 --2- 172.21.15.112:0/1711046406 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdaf00a2920 0x7fdaf00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.144+0000 7fdb0915f700 1 --2- 172.21.15.112:0/1711046406 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdaf00a2920 0x7fdaf00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.144+0000 7fdafaffd700 1 -- 172.21.15.112:0/1711046406 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fdb000cf450 con 0x7fdb041011b0 2024-03-16T09:14:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.144+0000 7fdafaffd700 1 --2- 172.21.15.112:0/1711046406 >> [v2:172.21.15.194:6824/3230458513,v1:172.21.15.194:6825/3230458513] conn(0x7fdaf00ac100 0x7fdaf00ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.144+0000 7fdafaffd700 1 -- 172.21.15.112:0/1711046406 --> [v2:172.21.15.194:6824/3230458513,v1:172.21.15.194:6825/3230458513] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb0001bec0 con 0x7fdaf00ac100 2024-03-16T09:14:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.144+0000 7fdafaffd700 1 -- 172.21.15.112:0/1711046406 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fdb000cf800 con 0x7fdb041011b0 2024-03-16T09:14:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.144+0000 7fdb0915f700 1 --2- 172.21.15.112:0/1711046406 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdaf00a2920 0x7fdaf00a4de0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400b2e0 tx=0x7fdaf4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.144+0000 7fdb0a161700 1 --2- 172.21.15.112:0/1711046406 >> [v2:172.21.15.194:6824/3230458513,v1:172.21.15.194:6825/3230458513] conn(0x7fdaf00ac100 0x7fdaf00ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.145+0000 7fdb0a161700 1 --2- 172.21.15.112:0/1711046406 >> [v2:172.21.15.194:6824/3230458513,v1:172.21.15.194:6825/3230458513] conn(0x7fdaf00ac100 0x7fdaf00ae520 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.145+0000 7fdafaffd700 1 -- 172.21.15.112:0/1711046406 <== osd.11 v2:172.21.15.194:6824/3230458513 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb0001bec0 con 0x7fdaf00ac100 2024-03-16T09:14:19.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.163+0000 7fdb0bbc4700 1 -- 172.21.15.112:0/1711046406 --> [v2:172.21.15.194:6824/3230458513,v1:172.21.15.194:6825/3230458513] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb041028a0 con 0x7fdaf00ac100 2024-03-16T09:14:19.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.163+0000 7fdafaffd700 1 -- 172.21.15.112:0/1711046406 <== osd.11 v2:172.21.15.194:6824/3230458513 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdb041028a0 con 0x7fdaf00ac100 2024-03-16T09:14:19.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.163+0000 7fdb0bbc4700 1 -- 172.21.15.112:0/1711046406 >> [v2:172.21.15.194:6824/3230458513,v1:172.21.15.194:6825/3230458513] conn(0x7fdaf00ac100 msgr2=0x7fdaf00ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:19.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.163+0000 7fdb0bbc4700 1 --2- 172.21.15.112:0/1711046406 >> [v2:172.21.15.194:6824/3230458513,v1:172.21.15.194:6825/3230458513] conn(0x7fdaf00ac100 0x7fdaf00ae520 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:19.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.163+0000 7fdb0bbc4700 1 -- 172.21.15.112:0/1711046406 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdaf00a2920 msgr2=0x7fdaf00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:19.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.163+0000 7fdb0bbc4700 1 --2- 172.21.15.112:0/1711046406 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdaf00a2920 0x7fdaf00a4de0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400b2e0 tx=0x7fdaf4005960 comp rx=0 tx=0).stop 2024-03-16T09:14:19.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.163+0000 7fdb0bbc4700 1 -- 172.21.15.112:0/1711046406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb041011b0 msgr2=0x7fdb0419e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:19.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.163+0000 7fdb0bbc4700 1 --2- 172.21.15.112:0/1711046406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb041011b0 0x7fdb0419e730 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000e9a0 tx=0x7fdb0000ecb0 comp rx=0 tx=0).stop 2024-03-16T09:14:19.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.163+0000 7fdb0a161700 1 -- 172.21.15.112:0/1711046406 reap_dead start 2024-03-16T09:14:19.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.163+0000 7fdb0bbc4700 1 -- 172.21.15.112:0/1711046406 shutdown_connections 2024-03-16T09:14:19.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.164+0000 7fdb0bbc4700 1 -- 172.21.15.112:0/1711046406 >> 172.21.15.112:0/1711046406 conn(0x7fdb040781e0 msgr2=0x7fdb04105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:19.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.164+0000 7fdb0bbc4700 1 -- 172.21.15.112:0/1711046406 shutdown_connections 2024-03-16T09:14:19.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:19.164+0000 7fdb0bbc4700 1 -- 172.21.15.112:0/1711046406 wait complete. 2024-03-16T09:14:20.064 INFO:teuthology.orchestra.run.smithi112.stdout:107374182461 2024-03-16T09:14:20.064 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-16T09:14:20.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:19 smithi159 ceph-mon[36739]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T09:14:20.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:19 smithi112 ceph-mon[32307]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T09:14:20.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:19 smithi194 ceph-mon[36729]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T09:14:20.710 INFO:teuthology.orchestra.run.smithi112.stdout:219043332137 2024-03-16T09:14:20.710 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-16T09:14:21.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.119+0000 7f23f4df4700 1 -- 172.21.15.112:0/1330545528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f0102350 msgr2=0x7f23f01027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:21.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.119+0000 7f23f4df4700 1 --2- 172.21.15.112:0/1330545528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f0102350 0x7f23f01027d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009990 tx=0x7f23e4009ca0 comp rx=0 tx=0).stop 2024-03-16T09:14:21.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.119+0000 7f23f4df4700 1 -- 172.21.15.112:0/1330545528 shutdown_connections 2024-03-16T09:14:21.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.119+0000 7f23f4df4700 1 --2- 172.21.15.112:0/1330545528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23f0102d10 0x7f23f010f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:21.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.119+0000 7f23f4df4700 1 --2- 172.21.15.112:0/1330545528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23f01083a0 0x7f23f0108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:21.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.119+0000 7f23f4df4700 1 --2- 172.21.15.112:0/1330545528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f0102350 0x7f23f01027d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:21.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.119+0000 7f23f4df4700 1 -- 172.21.15.112:0/1330545528 >> 172.21.15.112:0/1330545528 conn(0x7f23f0079390 msgr2=0x7f23f00797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:21.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.119+0000 7f23f4df4700 1 -- 172.21.15.112:0/1330545528 shutdown_connections 2024-03-16T09:14:21.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.119+0000 7f23f4df4700 1 -- 172.21.15.112:0/1330545528 wait complete. 2024-03-16T09:14:21.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.120+0000 7f23f4df4700 1 Processor -- start 2024-03-16T09:14:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.120+0000 7f23f4df4700 1 -- start start 2024-03-16T09:14:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23f4df4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23f0102350 0x7f23f019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23f4df4700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23f0102d10 0x7f23f019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23f4df4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f01083a0 0x7f23f0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23f4df4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23f0112720 con 0x7f23f01083a0 2024-03-16T09:14:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23f4df4700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f23f01125a0 con 0x7f23f0102d10 2024-03-16T09:14:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23f4df4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f23f01128a0 con 0x7f23f0102350 2024-03-16T09:14:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23eed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f01083a0 0x7f23f0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23edd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23f0102d10 0x7f23f019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23ee59c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23f0102350 0x7f23f019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23edd9b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23f0102d10 0x7f23f019ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:42076/0 (socket says 172.21.15.112:42076) 2024-03-16T09:14:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23edd9b700 1 -- 172.21.15.112:0/3439353968 learned_addr learned my addr 172.21.15.112:0/3439353968 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23eed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f01083a0 0x7f23f0198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50150/0 (socket says 172.21.15.112:50150) 2024-03-16T09:14:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23edd9b700 1 -- 172.21.15.112:0/3439353968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23f0102350 msgr2=0x7f23f019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23edd9b700 1 --2- 172.21.15.112:0/3439353968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23f0102350 0x7f23f019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23edd9b700 1 -- 172.21.15.112:0/3439353968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f01083a0 msgr2=0x7f23f0198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23edd9b700 1 --2- 172.21.15.112:0/3439353968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f01083a0 0x7f23f0198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23edd9b700 1 -- 172.21.15.112:0/3439353968 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f01990e0 con 0x7f23f0102d10 2024-03-16T09:14:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23eed9d700 1 --2- 172.21.15.112:0/3439353968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f01083a0 0x7f23f0198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23ee59c700 1 --2- 172.21.15.112:0/3439353968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23f0102350 0x7f23f019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.121+0000 7f23edd9b700 1 --2- 172.21.15.112:0/3439353968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23f0102d10 0x7f23f019ed00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f23e4000c00 tx=0x7f23e400f460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.122+0000 7f23df7fe700 1 -- 172.21.15.112:0/3439353968 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4027070 con 0x7f23f0102d10 2024-03-16T09:14:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.122+0000 7f23f4df4700 1 -- 172.21.15.112:0/3439353968 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f0199370 con 0x7f23f0102d10 2024-03-16T09:14:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.122+0000 7f23df7fe700 1 -- 172.21.15.112:0/3439353968 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f23e400fce0 con 0x7f23f0102d10 2024-03-16T09:14:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.122+0000 7f23df7fe700 1 -- 172.21.15.112:0/3439353968 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e40173f0 con 0x7f23f0102d10 2024-03-16T09:14:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.122+0000 7f23f4df4700 1 -- 172.21.15.112:0/3439353968 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f01a5690 con 0x7f23f0102d10 2024-03-16T09:14:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.122+0000 7f23f4df4700 1 -- 172.21.15.112:0/3439353968 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f23f00629e0 con 0x7f23f0102d10 2024-03-16T09:14:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.123+0000 7f23df7fe700 1 -- 172.21.15.112:0/3439353968 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f23e402a070 con 0x7f23f0102d10 2024-03-16T09:14:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.124+0000 7f23df7fe700 1 --2- 172.21.15.112:0/3439353968 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f23d80a2920 0x7f23d80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.124+0000 7f23ee59c700 1 --2- 172.21.15.112:0/3439353968 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f23d80a2920 0x7f23d80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.124+0000 7f23df7fe700 1 -- 172.21.15.112:0/3439353968 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f23e40cfcd0 con 0x7f23f0102d10 2024-03-16T09:14:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.124+0000 7f23df7fe700 1 --2- 172.21.15.112:0/3439353968 >> [v2:172.21.15.194:6816/3334422298,v1:172.21.15.194:6817/3334422298] conn(0x7f23d80ac100 0x7f23d80ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.124+0000 7f23df7fe700 1 -- 172.21.15.112:0/3439353968 --> [v2:172.21.15.194:6816/3334422298,v1:172.21.15.194:6817/3334422298] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f23e400bcd0 con 0x7f23d80ac100 2024-03-16T09:14:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.124+0000 7f23df7fe700 1 -- 172.21.15.112:0/3439353968 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f23e40d0080 con 0x7f23f0102d10 2024-03-16T09:14:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.124+0000 7f23ee59c700 1 --2- 172.21.15.112:0/3439353968 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f23d80a2920 0x7f23d80a4de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f23f0199ac0 tx=0x7f23e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.124+0000 7f23eed9d700 1 --2- 172.21.15.112:0/3439353968 >> [v2:172.21.15.194:6816/3334422298,v1:172.21.15.194:6817/3334422298] conn(0x7f23d80ac100 0x7f23d80ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.125+0000 7f23eed9d700 1 --2- 172.21.15.112:0/3439353968 >> [v2:172.21.15.194:6816/3334422298,v1:172.21.15.194:6817/3334422298] conn(0x7f23d80ac100 0x7f23d80ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.125+0000 7f23df7fe700 1 -- 172.21.15.112:0/3439353968 <== osd.8 v2:172.21.15.194:6816/3334422298 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f23e400bcd0 con 0x7f23d80ac100 2024-03-16T09:14:21.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.141+0000 7f23f4df4700 1 -- 172.21.15.112:0/3439353968 --> [v2:172.21.15.194:6816/3334422298,v1:172.21.15.194:6817/3334422298] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f23f0062e60 con 0x7f23d80ac100 2024-03-16T09:14:21.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.142+0000 7f23df7fe700 1 -- 172.21.15.112:0/3439353968 <== osd.8 v2:172.21.15.194:6816/3334422298 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f23f0062e60 con 0x7f23d80ac100 2024-03-16T09:14:21.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.142+0000 7f23f4df4700 1 -- 172.21.15.112:0/3439353968 >> [v2:172.21.15.194:6816/3334422298,v1:172.21.15.194:6817/3334422298] conn(0x7f23d80ac100 msgr2=0x7f23d80ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:21.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.142+0000 7f23f4df4700 1 --2- 172.21.15.112:0/3439353968 >> [v2:172.21.15.194:6816/3334422298,v1:172.21.15.194:6817/3334422298] conn(0x7f23d80ac100 0x7f23d80ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:21.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.142+0000 7f23f4df4700 1 -- 172.21.15.112:0/3439353968 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f23d80a2920 msgr2=0x7f23d80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:21.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.142+0000 7f23f4df4700 1 --2- 172.21.15.112:0/3439353968 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f23d80a2920 0x7f23d80a4de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f23f0199ac0 tx=0x7f23e000b410 comp rx=0 tx=0).stop 2024-03-16T09:14:21.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.142+0000 7f23f4df4700 1 -- 172.21.15.112:0/3439353968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23f0102d10 msgr2=0x7f23f019ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:21.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.142+0000 7f23f4df4700 1 --2- 172.21.15.112:0/3439353968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23f0102d10 0x7f23f019ed00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f23e4000c00 tx=0x7f23e400f460 comp rx=0 tx=0).stop 2024-03-16T09:14:21.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.142+0000 7f23eed9d700 1 -- 172.21.15.112:0/3439353968 reap_dead start 2024-03-16T09:14:21.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.142+0000 7f23f4df4700 1 -- 172.21.15.112:0/3439353968 shutdown_connections 2024-03-16T09:14:21.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.142+0000 7f23f4df4700 1 -- 172.21.15.112:0/3439353968 >> 172.21.15.112:0/3439353968 conn(0x7f23f0079390 msgr2=0x7f23f0100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:21.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.142+0000 7f23f4df4700 1 -- 172.21.15.112:0/3439353968 shutdown_connections 2024-03-16T09:14:21.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:21.142+0000 7f23f4df4700 1 -- 172.21.15.112:0/3439353968 wait complete. 2024-03-16T09:14:22.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:21 smithi159 ceph-mon[36739]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T09:14:22.319 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:21 smithi112 ceph-mon[32307]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T09:14:22.350 INFO:teuthology.orchestra.run.smithi112.stdout:253403070501 2024-03-16T09:14:22.350 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-16T09:14:22.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:21 smithi194 ceph-mon[36729]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T09:14:23.018 INFO:teuthology.orchestra.run.smithi112.stdout:34359738445 2024-03-16T09:14:23.019 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-16T09:14:23.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.344+0000 7f6418140700 1 -- 172.21.15.112:0/1322394913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6410100e70 msgr2=0x7f6410101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:23.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.344+0000 7f6418140700 1 --2- 172.21.15.112:0/1322394913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6410100e70 0x7f6410101250 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f640c0099c0 tx=0x7f640c009cd0 comp rx=0 tx=0).stop 2024-03-16T09:14:23.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.344+0000 7f6418140700 1 -- 172.21.15.112:0/1322394913 shutdown_connections 2024-03-16T09:14:23.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.344+0000 7f6418140700 1 --2- 172.21.15.112:0/1322394913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6410107690 0x7f641010f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:23.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.344+0000 7f6418140700 1 --2- 172.21.15.112:0/1322394913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6410100e70 0x7f6410101250 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:23.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.344+0000 7f6418140700 1 --2- 172.21.15.112:0/1322394913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6410101820 0x7f6410106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:23.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.344+0000 7f6418140700 1 -- 172.21.15.112:0/1322394913 >> 172.21.15.112:0/1322394913 conn(0x7f6410076510 msgr2=0x7f6410076920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:23.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.344+0000 7f6418140700 1 -- 172.21.15.112:0/1322394913 shutdown_connections 2024-03-16T09:14:23.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.344+0000 7f6418140700 1 -- 172.21.15.112:0/1322394913 wait complete. 2024-03-16T09:14:23.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.345+0000 7f6418140700 1 Processor -- start 2024-03-16T09:14:23.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.345+0000 7f6418140700 1 -- start start 2024-03-16T09:14:23.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.345+0000 7f6418140700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6410100e70 0x7f6410101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:23.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.345+0000 7f6418140700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6410101820 0x7f6410102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:23.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.345+0000 7f6418140700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6410107690 0x7f6410102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:23.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.345+0000 7f6418140700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6410112740 con 0x7f6410107690 2024-03-16T09:14:23.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.345+0000 7f6418140700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f64101125c0 con 0x7f6410101820 2024-03-16T09:14:23.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.345+0000 7f6418140700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f64101128c0 con 0x7f6410100e70 2024-03-16T09:14:23.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.345+0000 7f64166dd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6410107690 0x7f6410102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:23.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.345+0000 7f6415edc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6410100e70 0x7f6410101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:23.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.345+0000 7f64166dd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6410107690 0x7f6410102870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41382/0 (socket says 172.21.15.112:41382) 2024-03-16T09:14:23.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.345+0000 7f64156db700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6410101820 0x7f6410102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:23.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.345+0000 7f64166dd700 1 -- 172.21.15.112:0/1038455978 learned_addr learned my addr 172.21.15.112:0/1038455978 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:23.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.345+0000 7f64166dd700 1 -- 172.21.15.112:0/1038455978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6410100e70 msgr2=0x7f6410101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:23.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.345+0000 7f64166dd700 1 --2- 172.21.15.112:0/1038455978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6410100e70 0x7f6410101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:23.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.345+0000 7f64166dd700 1 -- 172.21.15.112:0/1038455978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6410101820 msgr2=0x7f6410102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:23.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.345+0000 7f64166dd700 1 --2- 172.21.15.112:0/1038455978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6410101820 0x7f6410102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:23.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.345+0000 7f64166dd700 1 -- 172.21.15.112:0/1038455978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64101a70e0 con 0x7f6410107690 2024-03-16T09:14:23.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.345+0000 7f6415edc700 1 --2- 172.21.15.112:0/1038455978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6410100e70 0x7f6410101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:23.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.347+0000 7f64166dd700 1 --2- 172.21.15.112:0/1038455978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6410107690 0x7f6410102870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f640800ef30 tx=0x7f640800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:23.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.347+0000 7f6406ffd700 1 -- 172.21.15.112:0/1038455978 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64080092e0 con 0x7f6410107690 2024-03-16T09:14:23.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.347+0000 7f6406ffd700 1 -- 172.21.15.112:0/1038455978 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6408004bb0 con 0x7f6410107690 2024-03-16T09:14:23.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.347+0000 7f6418140700 1 -- 172.21.15.112:0/1038455978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64101a7370 con 0x7f6410107690 2024-03-16T09:14:23.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.347+0000 7f6406ffd700 1 -- 172.21.15.112:0/1038455978 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408004280 con 0x7f6410107690 2024-03-16T09:14:23.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.347+0000 7f6418140700 1 -- 172.21.15.112:0/1038455978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64101a78b0 con 0x7f6410107690 2024-03-16T09:14:23.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.348+0000 7f6418140700 1 -- 172.21.15.112:0/1038455978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f64100629a0 con 0x7f6410107690 2024-03-16T09:14:23.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.349+0000 7f6406ffd700 1 -- 172.21.15.112:0/1038455978 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f640801b030 con 0x7f6410107690 2024-03-16T09:14:23.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.349+0000 7f6406ffd700 1 --2- 172.21.15.112:0/1038455978 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f63fc0a2920 0x7f63fc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:23.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.349+0000 7f6415edc700 1 --2- 172.21.15.112:0/1038455978 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f63fc0a2920 0x7f63fc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:23.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.349+0000 7f6406ffd700 1 -- 172.21.15.112:0/1038455978 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f64080cedf0 con 0x7f6410107690 2024-03-16T09:14:23.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.350+0000 7f6406ffd700 1 --2- 172.21.15.112:0/1038455978 >> [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923] conn(0x7f63fc0ac100 0x7f63fc0ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:23.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.350+0000 7f6406ffd700 1 -- 172.21.15.112:0/1038455978 --> [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f640801fe50 con 0x7f63fc0ac100 2024-03-16T09:14:23.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.350+0000 7f6406ffd700 1 -- 172.21.15.112:0/1038455978 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f64080cf2d0 con 0x7f6410107690 2024-03-16T09:14:23.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.350+0000 7f64156db700 1 --2- 172.21.15.112:0/1038455978 >> [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923] conn(0x7f63fc0ac100 0x7f63fc0ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:23.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.350+0000 7f6415edc700 1 --2- 172.21.15.112:0/1038455978 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f63fc0a2920 0x7f63fc0a4de0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f640c000c00 tx=0x7f640c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:23.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.350+0000 7f64156db700 1 --2- 172.21.15.112:0/1038455978 >> [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923] conn(0x7f63fc0ac100 0x7f63fc0ae520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:23.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.351+0000 7f6406ffd700 1 -- 172.21.15.112:0/1038455978 <== osd.6 v2:172.21.15.112:6818/3853469923 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f640801fe50 con 0x7f63fc0ac100 2024-03-16T09:14:23.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.367+0000 7f6418140700 1 -- 172.21.15.112:0/1038455978 --> [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6410062dd0 con 0x7f63fc0ac100 2024-03-16T09:14:23.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.367+0000 7f6406ffd700 1 -- 172.21.15.112:0/1038455978 <== osd.6 v2:172.21.15.112:6818/3853469923 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6410062dd0 con 0x7f63fc0ac100 2024-03-16T09:14:23.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.367+0000 7f6418140700 1 -- 172.21.15.112:0/1038455978 >> [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923] conn(0x7f63fc0ac100 msgr2=0x7f63fc0ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:23.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.367+0000 7f6418140700 1 --2- 172.21.15.112:0/1038455978 >> [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923] conn(0x7f63fc0ac100 0x7f63fc0ae520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:23.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.367+0000 7f6418140700 1 -- 172.21.15.112:0/1038455978 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f63fc0a2920 msgr2=0x7f63fc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:23.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.367+0000 7f6418140700 1 --2- 172.21.15.112:0/1038455978 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f63fc0a2920 0x7f63fc0a4de0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f640c000c00 tx=0x7f640c005fb0 comp rx=0 tx=0).stop 2024-03-16T09:14:23.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.367+0000 7f6418140700 1 -- 172.21.15.112:0/1038455978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6410107690 msgr2=0x7f6410102870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:23.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.367+0000 7f6418140700 1 --2- 172.21.15.112:0/1038455978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6410107690 0x7f6410102870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f640800ef30 tx=0x7f640800c3a0 comp rx=0 tx=0).stop 2024-03-16T09:14:23.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.367+0000 7f64166dd700 1 -- 172.21.15.112:0/1038455978 reap_dead start 2024-03-16T09:14:23.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.367+0000 7f6418140700 1 -- 172.21.15.112:0/1038455978 shutdown_connections 2024-03-16T09:14:23.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.367+0000 7f6418140700 1 -- 172.21.15.112:0/1038455978 >> 172.21.15.112:0/1038455978 conn(0x7f6410076510 msgr2=0x7f64100fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:23.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.367+0000 7f6418140700 1 -- 172.21.15.112:0/1038455978 shutdown_connections 2024-03-16T09:14:23.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:23.367+0000 7f6418140700 1 -- 172.21.15.112:0/1038455978 wait complete. 2024-03-16T09:14:24.008 INFO:teuthology.orchestra.run.smithi112.stdout:197568495664 2024-03-16T09:14:24.009 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-16T09:14:24.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:23 smithi159 ceph-mon[36739]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:14:24.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:23 smithi112 ceph-mon[32307]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:14:24.361 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:14:24.362 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:14:24.363 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:14:24.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:23 smithi194 ceph-mon[36729]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:14:24.415 INFO:teuthology.orchestra.run.smithi112.stdout:343597383702 2024-03-16T09:14:24.416 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-16T09:14:24.914 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:14:24.914 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:14:25.345 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:14:25.348 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:14:25.350 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:14:26.209 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:25 smithi112 ceph-mon[32307]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T09:14:26.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:25 smithi159 ceph-mon[36739]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T09:14:26.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:25 smithi194 ceph-mon[36729]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T09:14:27.088 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:14:28.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:27 smithi159 ceph-mon[36739]: pgmap v313: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:14:28.289 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:27 smithi112 ceph-mon[32307]: pgmap v313: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:14:28.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:27 smithi194 ceph-mon[36729]: pgmap v313: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:14:30.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:29 smithi112 ceph-mon[32307]: pgmap v314: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T09:14:30.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:29 smithi194 ceph-mon[36729]: pgmap v314: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T09:14:30.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:29 smithi159 ceph-mon[36739]: pgmap v314: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T09:14:31.262 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:30 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-03-16T09:14:31.262 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:30 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-03-16T09:14:31.262 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:30 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-16T09:14:31.262 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:30 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-03-16T09:14:31.262 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:30 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-03-16T09:14:31.262 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:30 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-16T09:14:31.262 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:30 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-16T09:14:31.262 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:30 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-03-16T09:14:31.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:30 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-03-16T09:14:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:30 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-03-16T09:14:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:30 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-16T09:14:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:30 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-03-16T09:14:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:30 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-03-16T09:14:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:30 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-16T09:14:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:30 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-16T09:14:31.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:30 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-03-16T09:14:31.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:30 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-03-16T09:14:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:30 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-03-16T09:14:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:30 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-03-16T09:14:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:30 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-03-16T09:14:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:30 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-03-16T09:14:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:30 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-16T09:14:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:30 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-16T09:14:31.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:30 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-03-16T09:14:32.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:32 smithi112 ceph-mon[32307]: pgmap v315: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T09:14:32.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:32 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-03-16T09:14:32.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:32 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-03-16T09:14:32.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:32 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-16T09:14:32.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:32 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-03-16T09:14:32.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:32 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-03-16T09:14:32.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:32 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-16T09:14:32.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:32 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-16T09:14:32.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:32 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-03-16T09:14:32.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:32 smithi112 ceph-mon[32307]: osdmap e86: 12 total, 12 up, 12 in 2024-03-16T09:14:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:32 smithi194 ceph-mon[36729]: pgmap v315: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T09:14:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:32 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-03-16T09:14:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:32 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-03-16T09:14:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:32 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-16T09:14:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:32 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-03-16T09:14:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:32 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-03-16T09:14:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:32 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-16T09:14:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:32 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-16T09:14:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:32 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-03-16T09:14:32.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:32 smithi194 ceph-mon[36729]: osdmap e86: 12 total, 12 up, 12 in 2024-03-16T09:14:32.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:32 smithi159 ceph-mon[36739]: pgmap v315: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T09:14:32.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:32 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-03-16T09:14:32.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:32 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-03-16T09:14:32.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:32 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-03-16T09:14:32.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:32 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-03-16T09:14:32.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:32 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-03-16T09:14:32.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:32 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-16T09:14:32.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:32 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-16T09:14:32.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:32 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-03-16T09:14:32.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:32 smithi159 ceph-mon[36739]: osdmap e86: 12 total, 12 up, 12 in 2024-03-16T09:14:33.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.182+0000 7fcb560ee700 1 -- 172.21.15.112:0/2872971701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb50100ed0 msgr2=0x7fcb501012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:33.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.182+0000 7fcb560ee700 1 --2- 172.21.15.112:0/2872971701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb50100ed0 0x7fcb501012b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009990 tx=0x7fcb44009ca0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.182+0000 7fcb560ee700 1 -- 172.21.15.112:0/2872971701 shutdown_connections 2024-03-16T09:14:33.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.182+0000 7fcb560ee700 1 --2- 172.21.15.112:0/2872971701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb50105d60 0x7fcb5010d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.182+0000 7fcb560ee700 1 --2- 172.21.15.112:0/2872971701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb50100ed0 0x7fcb501012b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.182+0000 7fcb560ee700 1 --2- 172.21.15.112:0/2872971701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb50101880 0x7fcb50105630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.182+0000 7fcb560ee700 1 -- 172.21.15.112:0/2872971701 >> 172.21.15.112:0/2872971701 conn(0x7fcb50076550 msgr2=0x7fcb50076960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:33.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.182+0000 7fcb560ee700 1 -- 172.21.15.112:0/2872971701 shutdown_connections 2024-03-16T09:14:33.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.182+0000 7fcb560ee700 1 -- 172.21.15.112:0/2872971701 wait complete. 2024-03-16T09:14:33.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.183+0000 7fcb560ee700 1 Processor -- start 2024-03-16T09:14:33.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.183+0000 7fcb560ee700 1 -- start start 2024-03-16T09:14:33.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.183+0000 7fcb560ee700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb50100ed0 0x7fcb5019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:33.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.183+0000 7fcb560ee700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb50101880 0x7fcb5019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:33.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.183+0000 7fcb560ee700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb50105d60 0x7fcb50198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:33.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.183+0000 7fcb560ee700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb501105c0 con 0x7fcb50100ed0 2024-03-16T09:14:33.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.183+0000 7fcb560ee700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcb50110440 con 0x7fcb50101880 2024-03-16T09:14:33.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.183+0000 7fcb560ee700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcb50110740 con 0x7fcb50105d60 2024-03-16T09:14:33.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.184+0000 7fcb4f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb50100ed0 0x7fcb5019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:33.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.184+0000 7fcb4f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb50100ed0 0x7fcb5019e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37302/0 (socket says 172.21.15.112:37302) 2024-03-16T09:14:33.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.184+0000 7fcb4f7fe700 1 -- 172.21.15.112:0/457077205 learned_addr learned my addr 172.21.15.112:0/457077205 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:33.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.184+0000 7fcb4effd700 1 --2- 172.21.15.112:0/457077205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb50101880 0x7fcb5019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:33.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.184+0000 7fcb4ffff700 1 --2- 172.21.15.112:0/457077205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb50105d60 0x7fcb50198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:33.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.184+0000 7fcb4f7fe700 1 -- 172.21.15.112:0/457077205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb50105d60 msgr2=0x7fcb50198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:33.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.184+0000 7fcb4f7fe700 1 --2- 172.21.15.112:0/457077205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb50105d60 0x7fcb50198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.184+0000 7fcb4f7fe700 1 -- 172.21.15.112:0/457077205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb50101880 msgr2=0x7fcb5019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:33.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.184+0000 7fcb4f7fe700 1 --2- 172.21.15.112:0/457077205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb50101880 0x7fcb5019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.184+0000 7fcb4f7fe700 1 -- 172.21.15.112:0/457077205 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb501991c0 con 0x7fcb50100ed0 2024-03-16T09:14:33.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.184+0000 7fcb4f7fe700 1 --2- 172.21.15.112:0/457077205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb50100ed0 0x7fcb5019e8a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcb440057c0 tx=0x7fcb440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:33.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.184+0000 7fcb4cff9700 1 -- 172.21.15.112:0/457077205 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44003c10 con 0x7fcb50100ed0 2024-03-16T09:14:33.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.184+0000 7fcb4cff9700 1 -- 172.21.15.112:0/457077205 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcb44024070 con 0x7fcb50100ed0 2024-03-16T09:14:33.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.184+0000 7fcb560ee700 1 -- 172.21.15.112:0/457077205 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb50199450 con 0x7fcb50100ed0 2024-03-16T09:14:33.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.184+0000 7fcb4cff9700 1 -- 172.21.15.112:0/457077205 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44016420 con 0x7fcb50100ed0 2024-03-16T09:14:33.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.184+0000 7fcb560ee700 1 -- 172.21.15.112:0/457077205 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb50102570 con 0x7fcb50100ed0 2024-03-16T09:14:33.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.185+0000 7fcb560ee700 1 -- 172.21.15.112:0/457077205 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb501020e0 con 0x7fcb50100ed0 2024-03-16T09:14:33.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.186+0000 7fcb4cff9700 1 -- 172.21.15.112:0/457077205 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fcb4401e070 con 0x7fcb50100ed0 2024-03-16T09:14:33.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.187+0000 7fcb4cff9700 1 --2- 172.21.15.112:0/457077205 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcb380a2920 0x7fcb380a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:33.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.187+0000 7fcb4effd700 1 --2- 172.21.15.112:0/457077205 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcb380a2920 0x7fcb380a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:33.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.187+0000 7fcb4cff9700 1 -- 172.21.15.112:0/457077205 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcb440cfe70 con 0x7fcb50100ed0 2024-03-16T09:14:33.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.187+0000 7fcb4effd700 1 --2- 172.21.15.112:0/457077205 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcb380a2920 0x7fcb380a4de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb50107280 tx=0x7fcb40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:33.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.191+0000 7fcb4cff9700 1 -- 172.21.15.112:0/457077205 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcb4409ce50 con 0x7fcb50100ed0 2024-03-16T09:14:33.289 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:33 smithi112 ceph-mon[32307]: osdmap e87: 12 total, 12 up, 12 in 2024-03-16T09:14:33.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.395+0000 7fcb560ee700 1 -- 172.21.15.112:0/457077205 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fcb50199f90 con 0x7fcb50100ed0 2024-03-16T09:14:33.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.395+0000 7fcb4cff9700 1 -- 172.21.15.112:0/457077205 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcb4409c750 con 0x7fcb50100ed0 2024-03-16T09:14:33.397 INFO:teuthology.orchestra.run.smithi112.stdout:141733920828 2024-03-16T09:14:33.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.388+0000 7fc4842d0700 1 -- 172.21.15.112:0/1762189477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc47c101e10 msgr2=0x7fc47c105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:33.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.388+0000 7fc4842d0700 1 --2- 172.21.15.112:0/1762189477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc47c101e10 0x7fc47c105dd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0098f0 tx=0x7fc46c009c00 comp rx=0 tx=0).stop 2024-03-16T09:14:33.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.388+0000 7fc4842d0700 1 -- 172.21.15.112:0/1762189477 shutdown_connections 2024-03-16T09:14:33.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.388+0000 7fc4842d0700 1 --2- 172.21.15.112:0/1762189477 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc47c1064b0 0x7fc47c10e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.388+0000 7fc4842d0700 1 --2- 172.21.15.112:0/1762189477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc47c101460 0x7fc47c101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.388+0000 7fc4842d0700 1 --2- 172.21.15.112:0/1762189477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc47c101e10 0x7fc47c105dd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.388+0000 7fc4842d0700 1 -- 172.21.15.112:0/1762189477 >> 172.21.15.112:0/1762189477 conn(0x7fc47c077220 msgr2=0x7fc47c077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:33.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.388+0000 7fc4842d0700 1 -- 172.21.15.112:0/1762189477 shutdown_connections 2024-03-16T09:14:33.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.388+0000 7fc4842d0700 1 -- 172.21.15.112:0/1762189477 wait complete. 2024-03-16T09:14:33.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.389+0000 7fc4842d0700 1 Processor -- start 2024-03-16T09:14:33.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.389+0000 7fc4842d0700 1 -- start start 2024-03-16T09:14:33.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.389+0000 7fc4842d0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc47c101460 0x7fc47c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:33.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.389+0000 7fc4842d0700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc47c101e10 0x7fc47c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:33.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.389+0000 7fc4842d0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc47c1064b0 0x7fc47c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:33.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.389+0000 7fc4842d0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c1115b0 con 0x7fc47c1064b0 2024-03-16T09:14:33.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.389+0000 7fc4842d0700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c111430 con 0x7fc47c101e10 2024-03-16T09:14:33.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.389+0000 7fc4842d0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c111730 con 0x7fc47c101460 2024-03-16T09:14:33.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.389+0000 7fc48186b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc47c101e10 0x7fc47c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:33.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.389+0000 7fc48286d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc47c1064b0 0x7fc47c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:33.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.389+0000 7fc48206c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc47c101460 0x7fc47c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:33.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.389+0000 7fc48186b700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc47c101e10 0x7fc47c19b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:37390/0 (socket says 172.21.15.112:37390) 2024-03-16T09:14:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.389+0000 7fc48186b700 1 -- 172.21.15.112:0/2728305236 learned_addr learned my addr 172.21.15.112:0/2728305236 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.389+0000 7fc48286d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc47c1064b0 0x7fc47c19f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37322/0 (socket says 172.21.15.112:37322) 2024-03-16T09:14:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.389+0000 7fc48206c700 1 -- 172.21.15.112:0/2728305236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc47c101e10 msgr2=0x7fc47c19b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.390+0000 7fc48206c700 1 --2- 172.21.15.112:0/2728305236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc47c101e10 0x7fc47c19b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.390+0000 7fc48206c700 1 -- 172.21.15.112:0/2728305236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc47c1064b0 msgr2=0x7fc47c19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.390+0000 7fc48206c700 1 --2- 172.21.15.112:0/2728305236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc47c1064b0 0x7fc47c19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.390+0000 7fc48206c700 1 -- 172.21.15.112:0/2728305236 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47c19fdb0 con 0x7fc47c101460 2024-03-16T09:14:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.390+0000 7fc48186b700 1 --2- 172.21.15.112:0/2728305236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc47c101e10 0x7fc47c19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.390+0000 7fc48286d700 1 --2- 172.21.15.112:0/2728305236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc47c1064b0 0x7fc47c19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.390+0000 7fc48206c700 1 --2- 172.21.15.112:0/2728305236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc47c101460 0x7fc47c19ada0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc47800eb90 tx=0x7fc47800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.390+0000 7fc4737fe700 1 -- 172.21.15.112:0/2728305236 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47800cc90 con 0x7fc47c101460 2024-03-16T09:14:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.390+0000 7fc4842d0700 1 -- 172.21.15.112:0/2728305236 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47c1a00a0 con 0x7fc47c101460 2024-03-16T09:14:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.390+0000 7fc4737fe700 1 -- 172.21.15.112:0/2728305236 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc478004520 con 0x7fc47c101460 2024-03-16T09:14:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.390+0000 7fc4737fe700 1 -- 172.21.15.112:0/2728305236 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478010450 con 0x7fc47c101460 2024-03-16T09:14:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.390+0000 7fc4842d0700 1 -- 172.21.15.112:0/2728305236 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc47c1a7920 con 0x7fc47c101460 2024-03-16T09:14:33.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.391+0000 7fc4842d0700 1 -- 172.21.15.112:0/2728305236 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc47c068f90 con 0x7fc47c101460 2024-03-16T09:14:33.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.392+0000 7fc4737fe700 1 -- 172.21.15.112:0/2728305236 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fc478004000 con 0x7fc47c101460 2024-03-16T09:14:33.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.393+0000 7fc4737fe700 1 --2- 172.21.15.112:0/2728305236 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc4680a2920 0x7fc4680a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:33.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.393+0000 7fc48186b700 1 --2- 172.21.15.112:0/2728305236 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc4680a2920 0x7fc4680a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:33.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.393+0000 7fc4737fe700 1 -- 172.21.15.112:0/2728305236 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc4780cf010 con 0x7fc47c101460 2024-03-16T09:14:33.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.393+0000 7fc48186b700 1 --2- 172.21.15.112:0/2728305236 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc4680a2920 0x7fc4680a4de0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc47c1010d0 tx=0x7fc46c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:33.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:33 smithi194 ceph-mon[36729]: osdmap e87: 12 total, 12 up, 12 in 2024-03-16T09:14:33.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.397+0000 7fc4737fe700 1 -- 172.21.15.112:0/2728305236 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc47801f340 con 0x7fc47c101460 2024-03-16T09:14:33.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.397+0000 7fcb560ee700 1 -- 172.21.15.112:0/457077205 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcb380a2920 msgr2=0x7fcb380a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:33.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.397+0000 7fcb560ee700 1 --2- 172.21.15.112:0/457077205 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcb380a2920 0x7fcb380a4de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb50107280 tx=0x7fcb40009260 comp rx=0 tx=0).stop 2024-03-16T09:14:33.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.397+0000 7fcb560ee700 1 -- 172.21.15.112:0/457077205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb50100ed0 msgr2=0x7fcb5019e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:33.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.397+0000 7fcb560ee700 1 --2- 172.21.15.112:0/457077205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb50100ed0 0x7fcb5019e8a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcb440057c0 tx=0x7fcb440057f0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.397+0000 7fcb560ee700 1 -- 172.21.15.112:0/457077205 shutdown_connections 2024-03-16T09:14:33.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.397+0000 7fcb560ee700 1 --2- 172.21.15.112:0/457077205 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcb380a2920 0x7fcb380a4de0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.397+0000 7fcb560ee700 1 --2- 172.21.15.112:0/457077205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb50105d60 0x7fcb50198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.397+0000 7fcb560ee700 1 --2- 172.21.15.112:0/457077205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb50100ed0 0x7fcb5019e8a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.397+0000 7fcb560ee700 1 --2- 172.21.15.112:0/457077205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb50101880 0x7fcb5019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.397+0000 7fcb560ee700 1 -- 172.21.15.112:0/457077205 >> 172.21.15.112:0/457077205 conn(0x7fcb50076550 msgr2=0x7fcb501003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:33.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.397+0000 7fcb560ee700 1 -- 172.21.15.112:0/457077205 shutdown_connections 2024-03-16T09:14:33.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.397+0000 7fcb560ee700 1 -- 172.21.15.112:0/457077205 wait complete. 2024-03-16T09:14:33.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:33 smithi159 ceph-mon[36739]: osdmap e87: 12 total, 12 up, 12 in 2024-03-16T09:14:33.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.600+0000 7fc4842d0700 1 -- 172.21.15.112:0/2728305236 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc47c02db70 con 0x7fc47c101460 2024-03-16T09:14:33.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.600+0000 7fc4737fe700 1 -- 172.21.15.112:0/2728305236 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc47809bff0 con 0x7fc47c101460 2024-03-16T09:14:33.602 INFO:teuthology.orchestra.run.smithi112.stdout:77309411399 2024-03-16T09:14:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.602+0000 7fc4842d0700 1 -- 172.21.15.112:0/2728305236 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc4680a2920 msgr2=0x7fc4680a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.602+0000 7fc4842d0700 1 --2- 172.21.15.112:0/2728305236 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc4680a2920 0x7fc4680a4de0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc47c1010d0 tx=0x7fc46c019040 comp rx=0 tx=0).stop 2024-03-16T09:14:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.603+0000 7fc4842d0700 1 -- 172.21.15.112:0/2728305236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc47c101460 msgr2=0x7fc47c19ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.603+0000 7fc4842d0700 1 --2- 172.21.15.112:0/2728305236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc47c101460 0x7fc47c19ada0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc47800eb90 tx=0x7fc47800eea0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.603+0000 7fc4842d0700 1 -- 172.21.15.112:0/2728305236 shutdown_connections 2024-03-16T09:14:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.603+0000 7fc4842d0700 1 --2- 172.21.15.112:0/2728305236 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc4680a2920 0x7fc4680a4de0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.603+0000 7fc4842d0700 1 --2- 172.21.15.112:0/2728305236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc47c1064b0 0x7fc47c19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.603+0000 7fc4842d0700 1 --2- 172.21.15.112:0/2728305236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc47c101460 0x7fc47c19ada0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.603+0000 7fc4842d0700 1 --2- 172.21.15.112:0/2728305236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc47c101e10 0x7fc47c19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.603+0000 7fc4842d0700 1 -- 172.21.15.112:0/2728305236 >> 172.21.15.112:0/2728305236 conn(0x7fc47c077220 msgr2=0x7fc47c10cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.603+0000 7fc4842d0700 1 -- 172.21.15.112:0/2728305236 shutdown_connections 2024-03-16T09:14:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.603+0000 7fc4842d0700 1 -- 172.21.15.112:0/2728305236 wait complete. 2024-03-16T09:14:33.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.978+0000 7f384b7de700 1 -- 172.21.15.112:0/3185598023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f384406a920 msgr2=0x7f384410f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:33.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.978+0000 7f384b7de700 1 --2- 172.21.15.112:0/3185598023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f384406a920 0x7f384410f160 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f383c00b0d0 tx=0x7f383c00b3e0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.978+0000 7f384b7de700 1 -- 172.21.15.112:0/3185598023 shutdown_connections 2024-03-16T09:14:33.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.978+0000 7f384b7de700 1 --2- 172.21.15.112:0/3185598023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f384406a920 0x7f384410f160 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.978+0000 7f384b7de700 1 --2- 172.21.15.112:0/3185598023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3844108320 0x7f3844108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.978+0000 7f384b7de700 1 --2- 172.21.15.112:0/3185598023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3844069f80 0x7f384406a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.978+0000 7f384b7de700 1 -- 172.21.15.112:0/3185598023 >> 172.21.15.112:0/3185598023 conn(0x7f3844078100 msgr2=0x7f3844078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:33.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.978+0000 7f384b7de700 1 -- 172.21.15.112:0/3185598023 shutdown_connections 2024-03-16T09:14:33.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.978+0000 7f384b7de700 1 -- 172.21.15.112:0/3185598023 wait complete. 2024-03-16T09:14:33.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.979+0000 7f384b7de700 1 Processor -- start 2024-03-16T09:14:33.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.979+0000 7f384b7de700 1 -- start start 2024-03-16T09:14:33.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.979+0000 7f384b7de700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3844069f80 0x7f38441012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:33.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.979+0000 7f384b7de700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f384406a920 0x7f38441017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:33.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.979+0000 7f384b7de700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3844108320 0x7f3844101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:33.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.979+0000 7f384b7de700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38441126c0 con 0x7f384406a920 2024-03-16T09:14:33.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.979+0000 7f384b7de700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3844112540 con 0x7f3844069f80 2024-03-16T09:14:33.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.979+0000 7f384b7de700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3844112840 con 0x7f3844108320 2024-03-16T09:14:33.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.979+0000 7f3848d79700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f384406a920 0x7f38441017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:33.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.979+0000 7f3848d79700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f384406a920 0x7f38441017e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37342/0 (socket says 172.21.15.112:37342) 2024-03-16T09:14:33.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.979+0000 7f3848d79700 1 -- 172.21.15.112:0/2713428517 learned_addr learned my addr 172.21.15.112:0/2713428517 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:33.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.979+0000 7f384957a700 1 --2- 172.21.15.112:0/2713428517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3844069f80 0x7f38441012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:33.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.979+0000 7f3849d7b700 1 --2- 172.21.15.112:0/2713428517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3844108320 0x7f3844101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:33.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.979+0000 7f3848d79700 1 -- 172.21.15.112:0/2713428517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3844108320 msgr2=0x7f3844101d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:33.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.979+0000 7f3848d79700 1 --2- 172.21.15.112:0/2713428517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3844108320 0x7f3844101d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.979+0000 7f3848d79700 1 -- 172.21.15.112:0/2713428517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3844069f80 msgr2=0x7f38441012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:33.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.979+0000 7f3848d79700 1 --2- 172.21.15.112:0/2713428517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3844069f80 0x7f38441012a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:33.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.979+0000 7f3848d79700 1 -- 172.21.15.112:0/2713428517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38441a7090 con 0x7f384406a920 2024-03-16T09:14:33.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.979+0000 7f3848d79700 1 --2- 172.21.15.112:0/2713428517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f384406a920 0x7f38441017e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f383400eb30 tx=0x7f383400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:33.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.980+0000 7f383a7fc700 1 -- 172.21.15.112:0/2713428517 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383400cbd0 con 0x7f384406a920 2024-03-16T09:14:33.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.980+0000 7f383a7fc700 1 -- 172.21.15.112:0/2713428517 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3834004520 con 0x7f384406a920 2024-03-16T09:14:33.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.980+0000 7f384b7de700 1 -- 172.21.15.112:0/2713428517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38441a7380 con 0x7f384406a920 2024-03-16T09:14:33.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.980+0000 7f383a7fc700 1 -- 172.21.15.112:0/2713428517 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834010430 con 0x7f384406a920 2024-03-16T09:14:33.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.980+0000 7f384b7de700 1 -- 172.21.15.112:0/2713428517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38441a78c0 con 0x7f384406a920 2024-03-16T09:14:33.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.981+0000 7f384b7de700 1 -- 172.21.15.112:0/2713428517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3844068f90 con 0x7f384406a920 2024-03-16T09:14:33.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.982+0000 7f383a7fc700 1 -- 172.21.15.112:0/2713428517 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f3834004000 con 0x7f384406a920 2024-03-16T09:14:33.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.982+0000 7f383a7fc700 1 --2- 172.21.15.112:0/2713428517 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f38300a2920 0x7f38300a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:33.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.982+0000 7f384957a700 1 --2- 172.21.15.112:0/2713428517 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f38300a2920 0x7f38300a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:33.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.982+0000 7f383a7fc700 1 -- 172.21.15.112:0/2713428517 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f38340cf170 con 0x7f384406a920 2024-03-16T09:14:33.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.983+0000 7f384957a700 1 --2- 172.21.15.112:0/2713428517 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f38300a2920 0x7f38300a4de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3840009be0 tx=0x7f3840009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:33.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:33.986+0000 7f383a7fc700 1 -- 172.21.15.112:0/2713428517 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3834053740 con 0x7f384406a920 2024-03-16T09:14:34.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:34.187+0000 7f384b7de700 1 -- 172.21.15.112:0/2713428517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f384402db70 con 0x7f384406a920 2024-03-16T09:14:34.190 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:34 smithi112 ceph-mon[32307]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-16T09:14:34.190 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:34 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/457077205' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-16T09:14:34.190 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:34 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2728305236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-16T09:14:34.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:34.187+0000 7f383a7fc700 1 -- 172.21.15.112:0/2713428517 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f383401f330 con 0x7f384406a920 2024-03-16T09:14:34.191 INFO:teuthology.orchestra.run.smithi112.stdout:287762808867 2024-03-16T09:14:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:34.189+0000 7f384b7de700 1 -- 172.21.15.112:0/2713428517 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f38300a2920 msgr2=0x7f38300a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:34.189+0000 7f384b7de700 1 --2- 172.21.15.112:0/2713428517 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f38300a2920 0x7f38300a4de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3840009be0 tx=0x7f3840009260 comp rx=0 tx=0).stop 2024-03-16T09:14:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:34.189+0000 7f384b7de700 1 -- 172.21.15.112:0/2713428517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f384406a920 msgr2=0x7f38441017e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:34.189+0000 7f384b7de700 1 --2- 172.21.15.112:0/2713428517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f384406a920 0x7f38441017e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f383400eb30 tx=0x7f383400ee40 comp rx=0 tx=0).stop 2024-03-16T09:14:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:34.190+0000 7f384b7de700 1 -- 172.21.15.112:0/2713428517 shutdown_connections 2024-03-16T09:14:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:34.190+0000 7f384b7de700 1 --2- 172.21.15.112:0/2713428517 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f38300a2920 0x7f38300a4de0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:34.190+0000 7f384b7de700 1 --2- 172.21.15.112:0/2713428517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3844069f80 0x7f38441012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:34.190+0000 7f384b7de700 1 --2- 172.21.15.112:0/2713428517 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3844108320 0x7f3844101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:34.190+0000 7f384b7de700 1 --2- 172.21.15.112:0/2713428517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f384406a920 0x7f38441017e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:34.190+0000 7f384b7de700 1 -- 172.21.15.112:0/2713428517 >> 172.21.15.112:0/2713428517 conn(0x7f3844078100 msgr2=0x7f38440ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:34.190+0000 7f384b7de700 1 -- 172.21.15.112:0/2713428517 shutdown_connections 2024-03-16T09:14:34.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:34.190+0000 7f384b7de700 1 -- 172.21.15.112:0/2713428517 wait complete. 2024-03-16T09:14:34.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:34 smithi194 ceph-mon[36729]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-16T09:14:34.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:34 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/457077205' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-16T09:14:34.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:34 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2728305236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-16T09:14:34.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:34 smithi159 ceph-mon[36739]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-16T09:14:34.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:34 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/457077205' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-16T09:14:34.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:34 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2728305236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-16T09:14:35.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:35 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2713428517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-16T09:14:35.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:35 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2713428517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-16T09:14:35.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:35 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2713428517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-16T09:14:35.690 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920823 got 141733920828 for osd.4 2024-03-16T09:14:35.690 DEBUG:teuthology.parallel:result is None 2024-03-16T09:14:36.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:36 smithi112 ceph-mon[32307]: pgmap v319: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 72 B/s, 1 keys/s, 0 objects/s recovering 2024-03-16T09:14:36.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:36 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:14:36.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:36 smithi194 ceph-mon[36729]: pgmap v319: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 72 B/s, 1 keys/s, 0 objects/s recovering 2024-03-16T09:14:36.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:36 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:14:36.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:36 smithi159 ceph-mon[36739]: pgmap v319: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 72 B/s, 1 keys/s, 0 objects/s recovering 2024-03-16T09:14:36.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:36 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:14:37.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:37 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1327957664' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-16T09:14:37.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:37 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1327957664' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-16T09:14:37.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.534+0000 7f8092df8700 1 -- 172.21.15.112:0/2080916006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f808c1064e0 msgr2=0x7f808c10e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:37.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.534+0000 7f8092df8700 1 --2- 172.21.15.112:0/2080916006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f808c1064e0 0x7f808c10e0b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f808400b0a0 tx=0x7f808400b3b0 comp rx=0 tx=0).stop 2024-03-16T09:14:37.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.534+0000 7f8092df8700 1 -- 172.21.15.112:0/2080916006 shutdown_connections 2024-03-16T09:14:37.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.534+0000 7f8092df8700 1 --2- 172.21.15.112:0/2080916006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f808c1064e0 0x7f808c10e0b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:37.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.534+0000 7f8092df8700 1 --2- 172.21.15.112:0/2080916006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f808c101460 0x7f808c101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:37.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.534+0000 7f8092df8700 1 --2- 172.21.15.112:0/2080916006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f808c101e10 0x7f808c105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:37.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.534+0000 7f8092df8700 1 -- 172.21.15.112:0/2080916006 >> 172.21.15.112:0/2080916006 conn(0x7f808c077210 msgr2=0x7f808c077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:37.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.534+0000 7f8092df8700 1 -- 172.21.15.112:0/2080916006 shutdown_connections 2024-03-16T09:14:37.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.534+0000 7f8092df8700 1 -- 172.21.15.112:0/2080916006 wait complete. 2024-03-16T09:14:37.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.535+0000 7f8092df8700 1 Processor -- start 2024-03-16T09:14:37.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.535+0000 7f8092df8700 1 -- start start 2024-03-16T09:14:37.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.535+0000 7f8092df8700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f808c101460 0x7f808c199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:37.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.535+0000 7f8092df8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f808c101e10 0x7f808c199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:37.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.535+0000 7f8092df8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f808c1064e0 0x7f808c1a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:37.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.535+0000 7f8092df8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f808c111620 con 0x7f808c1064e0 2024-03-16T09:14:37.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.535+0000 7f8092df8700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f808c1114a0 con 0x7f808c101460 2024-03-16T09:14:37.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.535+0000 7f8092df8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f808c1117a0 con 0x7f808c101e10 2024-03-16T09:14:37.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.535+0000 7f8091395700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f808c1064e0 0x7f808c1a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:37.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.535+0000 7f8090b94700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f808c101460 0x7f808c199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:37.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.535+0000 7f8091395700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f808c1064e0 0x7f808c1a49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37360/0 (socket says 172.21.15.112:37360) 2024-03-16T09:14:37.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.535+0000 7f8091395700 1 -- 172.21.15.112:0/1327957664 learned_addr learned my addr 172.21.15.112:0/1327957664 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:37.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.535+0000 7f808bfff700 1 --2- 172.21.15.112:0/1327957664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f808c101e10 0x7f808c199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:37.268 INFO:teuthology.orchestra.run.smithi112.stdout:163208757303 2024-03-16T09:14:37.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.536+0000 7f8091395700 1 -- 172.21.15.112:0/1327957664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f808c101e10 msgr2=0x7f808c199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:37.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.536+0000 7f8091395700 1 --2- 172.21.15.112:0/1327957664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f808c101e10 0x7f808c199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:37.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.536+0000 7f8091395700 1 -- 172.21.15.112:0/1327957664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f808c101460 msgr2=0x7f808c199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:37.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.536+0000 7f8091395700 1 --2- 172.21.15.112:0/1327957664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f808c101460 0x7f808c199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:37.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.536+0000 7f8091395700 1 -- 172.21.15.112:0/1327957664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c1a4f00 con 0x7f808c1064e0 2024-03-16T09:14:37.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.536+0000 7f808bfff700 1 --2- 172.21.15.112:0/1327957664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f808c101e10 0x7f808c199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:37.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.536+0000 7f8090b94700 1 --2- 172.21.15.112:0/1327957664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f808c101460 0x7f808c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.536+0000 7f8091395700 1 --2- 172.21.15.112:0/1327957664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f808c1064e0 0x7f808c1a49c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8084006070 tx=0x7f8084004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.536+0000 7f8089ffb700 1 -- 172.21.15.112:0/1327957664 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084021070 con 0x7f808c1064e0 2024-03-16T09:14:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.536+0000 7f8089ffb700 1 -- 172.21.15.112:0/1327957664 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8084003c00 con 0x7f808c1064e0 2024-03-16T09:14:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.536+0000 7f8089ffb700 1 -- 172.21.15.112:0/1327957664 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808401d450 con 0x7f808c1064e0 2024-03-16T09:14:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.536+0000 7f8092df8700 1 -- 172.21.15.112:0/1327957664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808c197730 con 0x7f808c1064e0 2024-03-16T09:14:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.536+0000 7f8092df8700 1 -- 172.21.15.112:0/1327957664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f808c197c70 con 0x7f808c1064e0 2024-03-16T09:14:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.537+0000 7f8092df8700 1 -- 172.21.15.112:0/1327957664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808c068f90 con 0x7f808c1064e0 2024-03-16T09:14:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.538+0000 7f8089ffb700 1 -- 172.21.15.112:0/1327957664 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f8084007980 con 0x7f808c1064e0 2024-03-16T09:14:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.539+0000 7f8089ffb700 1 --2- 172.21.15.112:0/1327957664 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f807c0a2920 0x7f807c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.539+0000 7f8090b94700 1 --2- 172.21.15.112:0/1327957664 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f807c0a2920 0x7f807c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.539+0000 7f8089ffb700 1 -- 172.21.15.112:0/1327957664 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f808400f020 con 0x7f808c1064e0 2024-03-16T09:14:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.539+0000 7f8090b94700 1 --2- 172.21.15.112:0/1327957664 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f807c0a2920 0x7f807c0a4de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8080009710 tx=0x7f808000f7d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.543+0000 7f8089ffb700 1 -- 172.21.15.112:0/1327957664 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f808402a020 con 0x7f808c1064e0 2024-03-16T09:14:37.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.747+0000 7f8092df8700 1 -- 172.21.15.112:0/1327957664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f808c02db70 con 0x7f808c1064e0 2024-03-16T09:14:37.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.748+0000 7f8089ffb700 1 -- 172.21.15.112:0/1327957664 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f80840074e0 con 0x7f808c1064e0 2024-03-16T09:14:37.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.750+0000 7f8092df8700 1 -- 172.21.15.112:0/1327957664 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f807c0a2920 msgr2=0x7f807c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:37.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.750+0000 7f8092df8700 1 --2- 172.21.15.112:0/1327957664 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f807c0a2920 0x7f807c0a4de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8080009710 tx=0x7f808000f7d0 comp rx=0 tx=0).stop 2024-03-16T09:14:37.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.750+0000 7f8092df8700 1 -- 172.21.15.112:0/1327957664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f808c1064e0 msgr2=0x7f808c1a49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:37.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.750+0000 7f8092df8700 1 --2- 172.21.15.112:0/1327957664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f808c1064e0 0x7f808c1a49c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8084006070 tx=0x7f8084004ce0 comp rx=0 tx=0).stop 2024-03-16T09:14:37.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.750+0000 7f8092df8700 1 -- 172.21.15.112:0/1327957664 shutdown_connections 2024-03-16T09:14:37.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.750+0000 7f8092df8700 1 --2- 172.21.15.112:0/1327957664 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f807c0a2920 0x7f807c0a4de0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:37.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.750+0000 7f8092df8700 1 --2- 172.21.15.112:0/1327957664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f808c101460 0x7f808c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:37.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.750+0000 7f8092df8700 1 --2- 172.21.15.112:0/1327957664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f808c1064e0 0x7f808c1a49c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:37.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.750+0000 7f8092df8700 1 --2- 172.21.15.112:0/1327957664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f808c101e10 0x7f808c199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:37.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.750+0000 7f8092df8700 1 -- 172.21.15.112:0/1327957664 >> 172.21.15.112:0/1327957664 conn(0x7f808c077210 msgr2=0x7f808c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:37.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.750+0000 7f8092df8700 1 -- 172.21.15.112:0/1327957664 shutdown_connections 2024-03-16T09:14:37.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:36.750+0000 7f8092df8700 1 -- 172.21.15.112:0/1327957664 wait complete. 2024-03-16T09:14:37.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:37 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1327957664' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-16T09:14:38.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:38 smithi112 ceph-mon[32307]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 145 B/s, 1 keys/s, 1 objects/s recovering 2024-03-16T09:14:38.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:38 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:38.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:38 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:38.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:38 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:38.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:38 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:38.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:38 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3092044986' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-16T09:14:38.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:38 smithi194 ceph-mon[36729]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 145 B/s, 1 keys/s, 1 objects/s recovering 2024-03-16T09:14:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:38 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:38 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:38 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:38 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:38 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3092044986' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-16T09:14:38.438 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411394 got 77309411399 for osd.2 2024-03-16T09:14:38.438 DEBUG:teuthology.parallel:result is None 2024-03-16T09:14:38.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:38 smithi159 ceph-mon[36739]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 145 B/s, 1 keys/s, 1 objects/s recovering 2024-03-16T09:14:38.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:38 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:38.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:38 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:38.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:38 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:38.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:38 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:38.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:38 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3092044986' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-16T09:14:38.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.697+0000 7fc8e8c7c700 1 -- 172.21.15.112:0/216829242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8e406a920 msgr2=0x7fc8e410f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:38.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.697+0000 7fc8e8c7c700 1 --2- 172.21.15.112:0/216829242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8e406a920 0x7fc8e410f160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00b0d0 tx=0x7fc8dc00b3e0 comp rx=0 tx=0).stop 2024-03-16T09:14:38.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.697+0000 7fc8e8c7c700 1 -- 172.21.15.112:0/216829242 shutdown_connections 2024-03-16T09:14:38.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.697+0000 7fc8e8c7c700 1 --2- 172.21.15.112:0/216829242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8e4108320 0x7fc8e4108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:38.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.697+0000 7fc8e8c7c700 1 --2- 172.21.15.112:0/216829242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8e406a920 0x7fc8e410f160 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:38.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.697+0000 7fc8e8c7c700 1 --2- 172.21.15.112:0/216829242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8e4069f80 0x7fc8e406a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:38.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.697+0000 7fc8e8c7c700 1 -- 172.21.15.112:0/216829242 >> 172.21.15.112:0/216829242 conn(0x7fc8e4078100 msgr2=0x7fc8e4078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:38.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.698+0000 7fc8e8c7c700 1 -- 172.21.15.112:0/216829242 shutdown_connections 2024-03-16T09:14:38.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.698+0000 7fc8e8c7c700 1 -- 172.21.15.112:0/216829242 wait complete. 2024-03-16T09:14:38.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.698+0000 7fc8e8c7c700 1 Processor -- start 2024-03-16T09:14:38.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.698+0000 7fc8e8c7c700 1 -- start start 2024-03-16T09:14:38.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.698+0000 7fc8e8c7c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8e4069f80 0x7fc8e419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:38.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.698+0000 7fc8e8c7c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8e406a920 0x7fc8e419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:38.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.698+0000 7fc8e8c7c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8e4108320 0x7fc8e4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:38.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.698+0000 7fc8e8c7c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e41126c0 con 0x7fc8e4108320 2024-03-16T09:14:38.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.698+0000 7fc8e8c7c700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e4112540 con 0x7fc8e4069f80 2024-03-16T09:14:38.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.698+0000 7fc8e8c7c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e4112840 con 0x7fc8e406a920 2024-03-16T09:14:38.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.699+0000 7fc8e2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8e4108320 0x7fc8e4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:38.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.699+0000 7fc8e259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8e4069f80 0x7fc8e419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:38.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.699+0000 7fc8e1d9b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8e406a920 0x7fc8e419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:38.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.699+0000 7fc8e259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8e4069f80 0x7fc8e419e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:37440/0 (socket says 172.21.15.112:37440) 2024-03-16T09:14:38.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.699+0000 7fc8e259c700 1 -- 172.21.15.112:0/3092044986 learned_addr learned my addr 172.21.15.112:0/3092044986 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:38.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.699+0000 7fc8e2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8e4108320 0x7fc8e4198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37374/0 (socket says 172.21.15.112:37374) 2024-03-16T09:14:38.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.699+0000 7fc8e2d9d700 1 -- 172.21.15.112:0/3092044986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8e406a920 msgr2=0x7fc8e419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:38.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.699+0000 7fc8e2d9d700 1 --2- 172.21.15.112:0/3092044986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8e406a920 0x7fc8e419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:38.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.699+0000 7fc8e2d9d700 1 -- 172.21.15.112:0/3092044986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8e4069f80 msgr2=0x7fc8e419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:38.527 INFO:teuthology.orchestra.run.smithi112.stdout:55834574923 2024-03-16T09:14:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.699+0000 7fc8e2d9d700 1 --2- 172.21.15.112:0/3092044986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8e4069f80 0x7fc8e419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.699+0000 7fc8e2d9d700 1 -- 172.21.15.112:0/3092044986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e41990a0 con 0x7fc8e4108320 2024-03-16T09:14:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.699+0000 7fc8e259c700 1 --2- 172.21.15.112:0/3092044986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8e4069f80 0x7fc8e419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.699+0000 7fc8e2d9d700 1 --2- 172.21.15.112:0/3092044986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8e4108320 0x7fc8e4198940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009cd0 tx=0x7fc8dc007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.699+0000 7fc8db7fe700 1 -- 172.21.15.112:0/3092044986 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc007e60 con 0x7fc8e4108320 2024-03-16T09:14:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.699+0000 7fc8db7fe700 1 -- 172.21.15.112:0/3092044986 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc8dc022070 con 0x7fc8e4108320 2024-03-16T09:14:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.699+0000 7fc8e8c7c700 1 -- 172.21.15.112:0/3092044986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e4199330 con 0x7fc8e4108320 2024-03-16T09:14:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.699+0000 7fc8db7fe700 1 -- 172.21.15.112:0/3092044986 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc01b6c0 con 0x7fc8e4108320 2024-03-16T09:14:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.699+0000 7fc8e8c7c700 1 -- 172.21.15.112:0/3092044986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e41011d0 con 0x7fc8e4108320 2024-03-16T09:14:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.700+0000 7fc8e8c7c700 1 -- 172.21.15.112:0/3092044986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e41039d0 con 0x7fc8e4108320 2024-03-16T09:14:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.702+0000 7fc8db7fe700 1 -- 172.21.15.112:0/3092044986 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fc8dc01d470 con 0x7fc8e4108320 2024-03-16T09:14:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.703+0000 7fc8db7fe700 1 --2- 172.21.15.112:0/3092044986 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc8d00a29a0 0x7fc8d00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:38.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.703+0000 7fc8e259c700 1 --2- 172.21.15.112:0/3092044986 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc8d00a29a0 0x7fc8d00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.703+0000 7fc8db7fe700 1 -- 172.21.15.112:0/3092044986 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc8dc0cfe50 con 0x7fc8e4108320 2024-03-16T09:14:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.704+0000 7fc8e259c700 1 --2- 172.21.15.112:0/3092044986 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc8d00a29a0 0x7fc8d00a4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4199a80 tx=0x7fc8d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.706+0000 7fc8db7fe700 1 -- 172.21.15.112:0/3092044986 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc8dc003c00 con 0x7fc8e4108320 2024-03-16T09:14:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.918+0000 7fc8e8c7c700 1 -- 172.21.15.112:0/3092044986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc8e4199e70 con 0x7fc8e4108320 2024-03-16T09:14:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.919+0000 7fc8db7fe700 1 -- 172.21.15.112:0/3092044986 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc8dc09cdd0 con 0x7fc8e4108320 2024-03-16T09:14:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.921+0000 7fc8e8c7c700 1 -- 172.21.15.112:0/3092044986 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc8d00a29a0 msgr2=0x7fc8d00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.921+0000 7fc8e8c7c700 1 --2- 172.21.15.112:0/3092044986 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc8d00a29a0 0x7fc8d00a4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4199a80 tx=0x7fc8d4009260 comp rx=0 tx=0).stop 2024-03-16T09:14:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.921+0000 7fc8e8c7c700 1 -- 172.21.15.112:0/3092044986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8e4108320 msgr2=0x7fc8e4198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.921+0000 7fc8e8c7c700 1 --2- 172.21.15.112:0/3092044986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8e4108320 0x7fc8e4198940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009cd0 tx=0x7fc8dc007a20 comp rx=0 tx=0).stop 2024-03-16T09:14:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.921+0000 7fc8e8c7c700 1 -- 172.21.15.112:0/3092044986 shutdown_connections 2024-03-16T09:14:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.921+0000 7fc8e8c7c700 1 --2- 172.21.15.112:0/3092044986 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc8d00a29a0 0x7fc8d00a4e60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.921+0000 7fc8e8c7c700 1 --2- 172.21.15.112:0/3092044986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8e4069f80 0x7fc8e419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.921+0000 7fc8e8c7c700 1 --2- 172.21.15.112:0/3092044986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8e4108320 0x7fc8e4198940 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.921+0000 7fc8e8c7c700 1 --2- 172.21.15.112:0/3092044986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8e406a920 0x7fc8e419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.921+0000 7fc8e8c7c700 1 -- 172.21.15.112:0/3092044986 >> 172.21.15.112:0/3092044986 conn(0x7fc8e4078100 msgr2=0x7fc8e40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.921+0000 7fc8e8c7c700 1 -- 172.21.15.112:0/3092044986 shutdown_connections 2024-03-16T09:14:38.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:37.921+0000 7fc8e8c7c700 1 -- 172.21.15.112:0/3092044986 wait complete. 2024-03-16T09:14:38.970 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:14:38.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.975+0000 7f91e7457700 1 -- 172.21.15.112:0/3168584964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e0107650 msgr2=0x7f91e010f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:38.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.975+0000 7f91e7457700 1 --2- 172.21.15.112:0/3168584964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e0107650 0x7f91e010f1d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f91d800b0d0 tx=0x7f91d800b3e0 comp rx=0 tx=0).stop 2024-03-16T09:14:38.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.975+0000 7f91e7457700 1 -- 172.21.15.112:0/3168584964 shutdown_connections 2024-03-16T09:14:38.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.975+0000 7f91e7457700 1 --2- 172.21.15.112:0/3168584964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91e006aa70 0x7f91e006ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:38.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.975+0000 7f91e7457700 1 --2- 172.21.15.112:0/3168584964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e0107650 0x7f91e010f1d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:38.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.975+0000 7f91e7457700 1 --2- 172.21.15.112:0/3168584964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f91e006b420 0x7f91e0106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:38.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.975+0000 7f91e7457700 1 -- 172.21.15.112:0/3168584964 >> 172.21.15.112:0/3168584964 conn(0x7f91e0077220 msgr2=0x7f91e0077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:38.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.975+0000 7f91e7457700 1 -- 172.21.15.112:0/3168584964 shutdown_connections 2024-03-16T09:14:38.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.975+0000 7f91e7457700 1 -- 172.21.15.112:0/3168584964 wait complete. 2024-03-16T09:14:38.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.976+0000 7f91e7457700 1 Processor -- start 2024-03-16T09:14:38.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.976+0000 7f91e7457700 1 -- start start 2024-03-16T09:14:38.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.976+0000 7f91e7457700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e006aa70 0x7f91e019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:38.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.976+0000 7f91e7457700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f91e006b420 0x7f91e019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:38.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.976+0000 7f91e7457700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91e0107650 0x7f91e0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:38.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.977+0000 7f91e7457700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f91e01127a0 con 0x7f91e006aa70 2024-03-16T09:14:38.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.977+0000 7f91e7457700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f91e0112620 con 0x7f91e0107650 2024-03-16T09:14:38.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.977+0000 7f91e7457700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f91e0112920 con 0x7f91e006b420 2024-03-16T09:14:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.977+0000 7f91e49f2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f91e006b420 0x7f91e019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.977+0000 7f91e51f3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e006aa70 0x7f91e019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.977+0000 7f91e59f4700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91e0107650 0x7f91e0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.977+0000 7f91e49f2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f91e006b420 0x7f91e019eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:41294/0 (socket says 172.21.15.112:41294) 2024-03-16T09:14:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.977+0000 7f91e59f4700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91e0107650 0x7f91e0198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:37454/0 (socket says 172.21.15.112:37454) 2024-03-16T09:14:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.977+0000 7f91e49f2700 1 -- 172.21.15.112:0/681345760 learned_addr learned my addr 172.21.15.112:0/681345760 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.977+0000 7f91e49f2700 1 -- 172.21.15.112:0/681345760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91e0107650 msgr2=0x7f91e0198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.977+0000 7f91e49f2700 1 --2- 172.21.15.112:0/681345760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91e0107650 0x7f91e0198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.977+0000 7f91e49f2700 1 -- 172.21.15.112:0/681345760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e006aa70 msgr2=0x7f91e019e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.977+0000 7f91e49f2700 1 --2- 172.21.15.112:0/681345760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e006aa70 0x7f91e019e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.977+0000 7f91e49f2700 1 -- 172.21.15.112:0/681345760 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e0199180 con 0x7f91e006b420 2024-03-16T09:14:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.977+0000 7f91e51f3700 1 --2- 172.21.15.112:0/681345760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e006aa70 0x7f91e019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:38.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.977+0000 7f91e49f2700 1 --2- 172.21.15.112:0/681345760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f91e006b420 0x7f91e019eda0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f91d000e9c0 tx=0x7f91d000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:38.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.978+0000 7f91d67fc700 1 -- 172.21.15.112:0/681345760 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0004a30 con 0x7f91e006b420 2024-03-16T09:14:38.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.978+0000 7f91d67fc700 1 -- 172.21.15.112:0/681345760 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f91d000ccc0 con 0x7f91e006b420 2024-03-16T09:14:38.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.978+0000 7f91e7457700 1 -- 172.21.15.112:0/681345760 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e0199470 con 0x7f91e006b420 2024-03-16T09:14:38.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.978+0000 7f91d67fc700 1 -- 172.21.15.112:0/681345760 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0005680 con 0x7f91e006b420 2024-03-16T09:14:38.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.978+0000 7f91e7457700 1 -- 172.21.15.112:0/681345760 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e01a5730 con 0x7f91e006b420 2024-03-16T09:14:38.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.979+0000 7f91e7457700 1 -- 172.21.15.112:0/681345760 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e010a800 con 0x7f91e006b420 2024-03-16T09:14:38.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.981+0000 7f91d67fc700 1 -- 172.21.15.112:0/681345760 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f91d000c7c0 con 0x7f91e006b420 2024-03-16T09:14:38.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.981+0000 7f91d67fc700 1 --2- 172.21.15.112:0/681345760 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f91cc0a29a0 0x7f91cc0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:38.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.981+0000 7f91d67fc700 1 -- 172.21.15.112:0/681345760 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f91d00ced80 con 0x7f91e006b420 2024-03-16T09:14:38.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.982+0000 7f91e51f3700 1 --2- 172.21.15.112:0/681345760 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f91cc0a29a0 0x7f91cc0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:38.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.982+0000 7f91e51f3700 1 --2- 172.21.15.112:0/681345760 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f91cc0a29a0 0x7f91cc0a4e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f91e0199b60 tx=0x7f91dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:38.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:38.986+0000 7f91d67fc700 1 -- 172.21.15.112:0/681345760 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f91d009bc70 con 0x7f91e006b420 2024-03-16T09:14:39.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:39.187+0000 7f91e7457700 1 -- 172.21.15.112:0/681345760 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f91e0199f50 con 0x7f91e006b420 2024-03-16T09:14:39.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:39.188+0000 7f91d67fc700 1 -- 172.21.15.112:0/681345760 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f91d009b570 con 0x7f91e006b420 2024-03-16T09:14:39.191 INFO:teuthology.orchestra.run.smithi112.stdout:219043332142 2024-03-16T09:14:39.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:39.190+0000 7f91e7457700 1 -- 172.21.15.112:0/681345760 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f91cc0a29a0 msgr2=0x7f91cc0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:39.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:39.190+0000 7f91e7457700 1 --2- 172.21.15.112:0/681345760 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f91cc0a29a0 0x7f91cc0a4e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f91e0199b60 tx=0x7f91dc009260 comp rx=0 tx=0).stop 2024-03-16T09:14:39.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:39.190+0000 7f91e7457700 1 -- 172.21.15.112:0/681345760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f91e006b420 msgr2=0x7f91e019eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:39.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:39.190+0000 7f91e7457700 1 --2- 172.21.15.112:0/681345760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f91e006b420 0x7f91e019eda0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f91d000e9c0 tx=0x7f91d000ecd0 comp rx=0 tx=0).stop 2024-03-16T09:14:39.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:39.190+0000 7f91e7457700 1 -- 172.21.15.112:0/681345760 shutdown_connections 2024-03-16T09:14:39.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:39.190+0000 7f91e7457700 1 --2- 172.21.15.112:0/681345760 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f91cc0a29a0 0x7f91cc0a4e60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:39.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:39.190+0000 7f91e7457700 1 --2- 172.21.15.112:0/681345760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91e0107650 0x7f91e0198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:39.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:39.190+0000 7f91e7457700 1 --2- 172.21.15.112:0/681345760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91e006aa70 0x7f91e019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:39.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:39.190+0000 7f91e7457700 1 --2- 172.21.15.112:0/681345760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f91e006b420 0x7f91e019eda0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:39.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:39.190+0000 7f91e7457700 1 -- 172.21.15.112:0/681345760 >> 172.21.15.112:0/681345760 conn(0x7f91e0077220 msgr2=0x7f91e00ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:39.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:39.190+0000 7f91e7457700 1 -- 172.21.15.112:0/681345760 shutdown_connections 2024-03-16T09:14:39.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:39.190+0000 7f91e7457700 1 -- 172.21.15.112:0/681345760 wait complete. 2024-03-16T09:14:39.638 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757299 got 163208757303 for osd.5 2024-03-16T09:14:39.638 DEBUG:teuthology.parallel:result is None 2024-03-16T09:14:40.127 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:40 smithi112 ceph-mon[32307]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 145 B/s, 1 keys/s, 1 objects/s recovering 2024-03-16T09:14:40.127 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:40 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/681345760' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-16T09:14:40.128 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574923 for osd.1 2024-03-16T09:14:40.129 DEBUG:teuthology.parallel:result is None 2024-03-16T09:14:40.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:40 smithi194 ceph-mon[36729]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 145 B/s, 1 keys/s, 1 objects/s recovering 2024-03-16T09:14:40.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:40 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/681345760' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-16T09:14:40.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:40 smithi159 ceph-mon[36739]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 145 B/s, 1 keys/s, 1 objects/s recovering 2024-03-16T09:14:40.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:40 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/681345760' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-16T09:14:40.547 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808863 got 287762808867 for osd.9 2024-03-16T09:14:40.547 DEBUG:teuthology.parallel:result is None 2024-03-16T09:14:40.751 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:14:40.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.959+0000 7fc9de439700 1 -- 172.21.15.112:0/871723709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9d806b420 msgr2=0x7fc9d8106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:40.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.959+0000 7fc9de439700 1 --2- 172.21.15.112:0/871723709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9d806b420 0x7fc9d8106f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0098f0 tx=0x7fc9cc009c00 comp rx=0 tx=0).stop 2024-03-16T09:14:40.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.959+0000 7fc9de439700 1 -- 172.21.15.112:0/871723709 shutdown_connections 2024-03-16T09:14:40.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.959+0000 7fc9de439700 1 --2- 172.21.15.112:0/871723709 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9d8107650 0x7fc9d810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:40.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.959+0000 7fc9de439700 1 --2- 172.21.15.112:0/871723709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9d806aa70 0x7fc9d806ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:40.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.959+0000 7fc9de439700 1 --2- 172.21.15.112:0/871723709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9d806b420 0x7fc9d8106f20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:40.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.959+0000 7fc9de439700 1 -- 172.21.15.112:0/871723709 >> 172.21.15.112:0/871723709 conn(0x7fc9d8077220 msgr2=0x7fc9d8077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:40.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.959+0000 7fc9de439700 1 -- 172.21.15.112:0/871723709 shutdown_connections 2024-03-16T09:14:40.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.959+0000 7fc9de439700 1 -- 172.21.15.112:0/871723709 wait complete. 2024-03-16T09:14:40.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.960+0000 7fc9de439700 1 Processor -- start 2024-03-16T09:14:40.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.960+0000 7fc9de439700 1 -- start start 2024-03-16T09:14:40.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.960+0000 7fc9de439700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9d806aa70 0x7fc9d819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:40.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.960+0000 7fc9de439700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9d806b420 0x7fc9d819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:40.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.960+0000 7fc9de439700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9d8107650 0x7fc9d81989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:40.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9de439700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d81126e0 con 0x7fc9d8107650 2024-03-16T09:14:40.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9de439700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d8112560 con 0x7fc9d806b420 2024-03-16T09:14:40.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9de439700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d8112860 con 0x7fc9d806aa70 2024-03-16T09:14:40.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9dc9d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9d8107650 0x7fc9d81989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:40.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9dc9d6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9d8107650 0x7fc9d81989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37414/0 (socket says 172.21.15.112:37414) 2024-03-16T09:14:40.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9dc9d6700 1 -- 172.21.15.112:0/3205728680 learned_addr learned my addr 172.21.15.112:0/3205728680 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:40.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9d7fff700 1 --2- 172.21.15.112:0/3205728680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9d806aa70 0x7fc9d819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:40.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9d77fe700 1 --2- 172.21.15.112:0/3205728680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9d806b420 0x7fc9d819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:40.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9d77fe700 1 -- 172.21.15.112:0/3205728680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9d806aa70 msgr2=0x7fc9d819e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:40.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9d77fe700 1 --2- 172.21.15.112:0/3205728680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9d806aa70 0x7fc9d819e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:40.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9d77fe700 1 -- 172.21.15.112:0/3205728680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9d8107650 msgr2=0x7fc9d81989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:40.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9d77fe700 1 --2- 172.21.15.112:0/3205728680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9d8107650 0x7fc9d81989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:40.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9d77fe700 1 -- 172.21.15.112:0/3205728680 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d8199120 con 0x7fc9d806b420 2024-03-16T09:14:40.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9dc9d6700 1 --2- 172.21.15.112:0/3205728680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9d8107650 0x7fc9d81989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:40.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9d77fe700 1 --2- 172.21.15.112:0/3205728680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9d806b420 0x7fc9d819ed40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0098c0 tx=0x7fc9cc00b950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:40.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9d57fa700 1 -- 172.21.15.112:0/3205728680 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc00f660 con 0x7fc9d806b420 2024-03-16T09:14:40.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9de439700 1 -- 172.21.15.112:0/3205728680 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d81993b0 con 0x7fc9d806b420 2024-03-16T09:14:40.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9d57fa700 1 -- 172.21.15.112:0/3205728680 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc9cc003680 con 0x7fc9d806b420 2024-03-16T09:14:40.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9d57fa700 1 -- 172.21.15.112:0/3205728680 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc020810 con 0x7fc9d806b420 2024-03-16T09:14:40.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.961+0000 7fc9de439700 1 -- 172.21.15.112:0/3205728680 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d81a56d0 con 0x7fc9d806b420 2024-03-16T09:14:40.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.963+0000 7fc9de439700 1 -- 172.21.15.112:0/3205728680 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d8103a00 con 0x7fc9d806b420 2024-03-16T09:14:40.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.964+0000 7fc9d57fa700 1 -- 172.21.15.112:0/3205728680 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fc9cc003c00 con 0x7fc9d806b420 2024-03-16T09:14:40.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.964+0000 7fc9d57fa700 1 --2- 172.21.15.112:0/3205728680 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc9c00a2920 0x7fc9c00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:40.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.964+0000 7fc9d7fff700 1 --2- 172.21.15.112:0/3205728680 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc9c00a2920 0x7fc9c00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:40.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.964+0000 7fc9d57fa700 1 -- 172.21.15.112:0/3205728680 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc9cc0cf970 con 0x7fc9d806b420 2024-03-16T09:14:40.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.965+0000 7fc9d7fff700 1 --2- 172.21.15.112:0/3205728680 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc9c00a2920 0x7fc9c00a4de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8199b00 tx=0x7fc9c8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:40.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:40.974+0000 7fc9d57fa700 1 -- 172.21.15.112:0/3205728680 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc9cc013030 con 0x7fc9d806b420 2024-03-16T09:14:41.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:41.176+0000 7fc9de439700 1 -- 172.21.15.112:0/3205728680 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc9d802db70 con 0x7fc9d806b420 2024-03-16T09:14:41.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:41.176+0000 7fc9d57fa700 1 -- 172.21.15.112:0/3205728680 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc9cc09c950 con 0x7fc9d806b420 2024-03-16T09:14:41.180 INFO:teuthology.orchestra.run.smithi112.stdout:107374182466 2024-03-16T09:14:41.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:41.180+0000 7fc9de439700 1 -- 172.21.15.112:0/3205728680 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc9c00a2920 msgr2=0x7fc9c00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:41.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:41.180+0000 7fc9de439700 1 --2- 172.21.15.112:0/3205728680 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc9c00a2920 0x7fc9c00a4de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8199b00 tx=0x7fc9c8009310 comp rx=0 tx=0).stop 2024-03-16T09:14:41.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:41.180+0000 7fc9de439700 1 -- 172.21.15.112:0/3205728680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9d806b420 msgr2=0x7fc9d819ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:41.180+0000 7fc9de439700 1 --2- 172.21.15.112:0/3205728680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9d806b420 0x7fc9d819ed40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0098c0 tx=0x7fc9cc00b950 comp rx=0 tx=0).stop 2024-03-16T09:14:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:41.180+0000 7fc9de439700 1 -- 172.21.15.112:0/3205728680 shutdown_connections 2024-03-16T09:14:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:41.180+0000 7fc9de439700 1 --2- 172.21.15.112:0/3205728680 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc9c00a2920 0x7fc9c00a4de0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:41.180+0000 7fc9de439700 1 --2- 172.21.15.112:0/3205728680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9d8107650 0x7fc9d81989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:41.180+0000 7fc9de439700 1 --2- 172.21.15.112:0/3205728680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9d806aa70 0x7fc9d819e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:41.180+0000 7fc9de439700 1 --2- 172.21.15.112:0/3205728680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9d806b420 0x7fc9d819ed40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:41.180+0000 7fc9de439700 1 -- 172.21.15.112:0/3205728680 >> 172.21.15.112:0/3205728680 conn(0x7fc9d8077220 msgr2=0x7fc9d8108780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:41.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:41.180+0000 7fc9de439700 1 -- 172.21.15.112:0/3205728680 shutdown_connections 2024-03-16T09:14:41.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:41.180+0000 7fc9de439700 1 -- 172.21.15.112:0/3205728680 wait complete. 2024-03-16T09:14:42.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:42 smithi112 ceph-mon[32307]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 117 B/s, 1 keys/s, 0 objects/s recovering 2024-03-16T09:14:42.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:42 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3205728680' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-16T09:14:42.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:42 smithi194 ceph-mon[36729]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 117 B/s, 1 keys/s, 0 objects/s recovering 2024-03-16T09:14:42.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:42 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3205728680' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-16T09:14:42.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:42 smithi159 ceph-mon[36739]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 117 B/s, 1 keys/s, 0 objects/s recovering 2024-03-16T09:14:42.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:42 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3205728680' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-16T09:14:42.593 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182461 got 107374182466 for osd.3 2024-03-16T09:14:42.594 DEBUG:teuthology.parallel:result is None 2024-03-16T09:14:44.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:44 smithi112 ceph-mon[32307]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 106 B/s, 0 keys/s, 0 objects/s recovering 2024-03-16T09:14:44.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:44 smithi194 ceph-mon[36729]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 106 B/s, 0 keys/s, 0 objects/s recovering 2024-03-16T09:14:44.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:44 smithi159 ceph-mon[36739]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 106 B/s, 0 keys/s, 0 objects/s recovering 2024-03-16T09:14:44.502 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332137 got 219043332142 for osd.7 2024-03-16T09:14:44.502 DEBUG:teuthology.parallel:result is None 2024-03-16T09:14:44.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.696+0000 7f6df8f22700 1 -- 172.21.15.112:0/2648897820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df4101e10 msgr2=0x7f6df4105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:44.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.696+0000 7f6df8f22700 1 --2- 172.21.15.112:0/2648897820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df4101e10 0x7f6df4105dd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6de80099c0 tx=0x7f6de8009cd0 comp rx=0 tx=0).stop 2024-03-16T09:14:44.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.696+0000 7f6df8f22700 1 -- 172.21.15.112:0/2648897820 shutdown_connections 2024-03-16T09:14:44.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.696+0000 7f6df8f22700 1 --2- 172.21.15.112:0/2648897820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6df4101460 0x7f6df4101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:44.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.696+0000 7f6df8f22700 1 --2- 172.21.15.112:0/2648897820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6df41064b0 0x7f6df410e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:44.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.696+0000 7f6df8f22700 1 --2- 172.21.15.112:0/2648897820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df4101e10 0x7f6df4105dd0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:44.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.696+0000 7f6df8f22700 1 -- 172.21.15.112:0/2648897820 >> 172.21.15.112:0/2648897820 conn(0x7f6df4077220 msgr2=0x7f6df4077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:44.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.696+0000 7f6df8f22700 1 -- 172.21.15.112:0/2648897820 shutdown_connections 2024-03-16T09:14:44.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.696+0000 7f6df8f22700 1 -- 172.21.15.112:0/2648897820 wait complete. 2024-03-16T09:14:44.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.697+0000 7f6df8f22700 1 Processor -- start 2024-03-16T09:14:44.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.697+0000 7f6df8f22700 1 -- start start 2024-03-16T09:14:44.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.697+0000 7f6df8f22700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6df4101460 0x7f6df419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:44.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.697+0000 7f6df8f22700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6df4101e10 0x7f6df419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:44.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.697+0000 7f6df8f22700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df41064b0 0x7f6df4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:44.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.697+0000 7f6df8f22700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6df4111580 con 0x7f6df41064b0 2024-03-16T09:14:44.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.697+0000 7f6df8f22700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6df4111400 con 0x7f6df4101460 2024-03-16T09:14:44.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.697+0000 7f6df8f22700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6df4111700 con 0x7f6df4101e10 2024-03-16T09:14:44.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.698+0000 7f6df259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6df4101460 0x7f6df419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:44.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.698+0000 7f6df2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df41064b0 0x7f6df4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:44.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.698+0000 7f6df259c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6df4101460 0x7f6df419e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:58376/0 (socket says 172.21.15.112:58376) 2024-03-16T09:14:44.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.698+0000 7f6df259c700 1 -- 172.21.15.112:0/2596714037 learned_addr learned my addr 172.21.15.112:0/2596714037 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:44.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.698+0000 7f6df1d9b700 1 --2- 172.21.15.112:0/2596714037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6df4101e10 0x7f6df419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:44.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.698+0000 7f6df259c700 1 -- 172.21.15.112:0/2596714037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6df4101e10 msgr2=0x7f6df419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:44.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.698+0000 7f6df259c700 1 --2- 172.21.15.112:0/2596714037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6df4101e10 0x7f6df419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:44.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.698+0000 7f6df259c700 1 -- 172.21.15.112:0/2596714037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df41064b0 msgr2=0x7f6df4198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:44.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.698+0000 7f6df259c700 1 --2- 172.21.15.112:0/2596714037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df41064b0 0x7f6df4198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:44.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.698+0000 7f6df259c700 1 -- 172.21.15.112:0/2596714037 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df41990a0 con 0x7f6df4101460 2024-03-16T09:14:44.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.698+0000 7f6df259c700 1 --2- 172.21.15.112:0/2596714037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6df4101460 0x7f6df419e780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6de400eb30 tx=0x7f6de400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:44.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.698+0000 7f6de37fe700 1 -- 172.21.15.112:0/2596714037 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de400cbd0 con 0x7f6df4101460 2024-03-16T09:14:44.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.698+0000 7f6df8f22700 1 -- 172.21.15.112:0/2596714037 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df4199390 con 0x7f6df4101460 2024-03-16T09:14:44.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.698+0000 7f6de37fe700 1 -- 172.21.15.112:0/2596714037 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6de4004520 con 0x7f6df4101460 2024-03-16T09:14:44.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.698+0000 7f6de37fe700 1 -- 172.21.15.112:0/2596714037 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4010430 con 0x7f6df4101460 2024-03-16T09:14:44.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.698+0000 7f6df8f22700 1 -- 172.21.15.112:0/2596714037 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df41a5650 con 0x7f6df4101460 2024-03-16T09:14:44.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.700+0000 7f6df8f22700 1 -- 172.21.15.112:0/2596714037 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df4068f90 con 0x7f6df4101460 2024-03-16T09:14:44.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.700+0000 7f6de37fe700 1 -- 172.21.15.112:0/2596714037 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f6de4004000 con 0x7f6df4101460 2024-03-16T09:14:44.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.701+0000 7f6de37fe700 1 --2- 172.21.15.112:0/2596714037 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ddc0a2920 0x7f6ddc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:44.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.701+0000 7f6df1d9b700 1 --2- 172.21.15.112:0/2596714037 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ddc0a2920 0x7f6ddc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:44.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.701+0000 7f6de37fe700 1 -- 172.21.15.112:0/2596714037 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f6de40cef20 con 0x7f6df4101460 2024-03-16T09:14:44.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.701+0000 7f6df1d9b700 1 --2- 172.21.15.112:0/2596714037 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ddc0a2920 0x7f6ddc0a4de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6de8000c00 tx=0x7f6de8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:44.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.706+0000 7f6de37fe700 1 -- 172.21.15.112:0/2596714037 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6de409be10 con 0x7f6df4101460 2024-03-16T09:14:44.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.911+0000 7f6df8f22700 1 -- 172.21.15.112:0/2596714037 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f6df4199e70 con 0x7f6df4101460 2024-03-16T09:14:44.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.911+0000 7f6de37fe700 1 -- 172.21.15.112:0/2596714037 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f6de409b710 con 0x7f6df4101460 2024-03-16T09:14:44.915 INFO:teuthology.orchestra.run.smithi112.stdout:309237645343 2024-03-16T09:14:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.914+0000 7f6df8f22700 1 -- 172.21.15.112:0/2596714037 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ddc0a2920 msgr2=0x7f6ddc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.914+0000 7f6df8f22700 1 --2- 172.21.15.112:0/2596714037 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ddc0a2920 0x7f6ddc0a4de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6de8000c00 tx=0x7f6de8005fb0 comp rx=0 tx=0).stop 2024-03-16T09:14:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.914+0000 7f6df8f22700 1 -- 172.21.15.112:0/2596714037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6df4101460 msgr2=0x7f6df419e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.914+0000 7f6df8f22700 1 --2- 172.21.15.112:0/2596714037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6df4101460 0x7f6df419e780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6de400eb30 tx=0x7f6de400ee40 comp rx=0 tx=0).stop 2024-03-16T09:14:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.914+0000 7f6df8f22700 1 -- 172.21.15.112:0/2596714037 shutdown_connections 2024-03-16T09:14:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.914+0000 7f6df8f22700 1 --2- 172.21.15.112:0/2596714037 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ddc0a2920 0x7f6ddc0a4de0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.914+0000 7f6df8f22700 1 --2- 172.21.15.112:0/2596714037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6df4101460 0x7f6df419e780 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:44.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.914+0000 7f6df8f22700 1 --2- 172.21.15.112:0/2596714037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df41064b0 0x7f6df4198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:44.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.914+0000 7f6df8f22700 1 --2- 172.21.15.112:0/2596714037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6df4101e10 0x7f6df419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:44.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.914+0000 7f6df8f22700 1 -- 172.21.15.112:0/2596714037 >> 172.21.15.112:0/2596714037 conn(0x7f6df4077220 msgr2=0x7f6df406aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:44.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.914+0000 7f6df8f22700 1 -- 172.21.15.112:0/2596714037 shutdown_connections 2024-03-16T09:14:44.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:44.914+0000 7f6df8f22700 1 -- 172.21.15.112:0/2596714037 wait complete. 2024-03-16T09:14:45.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:45 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2596714037' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-16T09:14:45.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:45 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2596714037' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-16T09:14:45.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:45 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2596714037' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-16T09:14:45.840 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645335 got 309237645343 for osd.10 2024-03-16T09:14:45.840 DEBUG:teuthology.parallel:result is None 2024-03-16T09:14:46.089 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:46 smithi112 ceph-mon[32307]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 96 B/s, 0 keys/s, 0 objects/s recovering 2024-03-16T09:14:46.387 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:46 smithi159 ceph-mon[36739]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 96 B/s, 0 keys/s, 0 objects/s recovering 2024-03-16T09:14:46.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:46 smithi194 ceph-mon[36729]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 96 B/s, 0 keys/s, 0 objects/s recovering 2024-03-16T09:14:46.641 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:14:47.115 INFO:teuthology.orchestra.run.smithi112.stdout:253403070506 2024-03-16T09:14:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.262+0000 7f0b4b94a700 1 -- 172.21.15.112:0/3980843483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b44102ce0 msgr2=0x7f0b4410f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.262+0000 7f0b4b94a700 1 --2- 172.21.15.112:0/3980843483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b44102ce0 0x7f0b4410f120 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00b0d0 tx=0x7f0b3c00b3e0 comp rx=0 tx=0).stop 2024-03-16T09:14:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.262+0000 7f0b4b94a700 1 -- 172.21.15.112:0/3980843483 shutdown_connections 2024-03-16T09:14:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.262+0000 7f0b4b94a700 1 --2- 172.21.15.112:0/3980843483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b44102ce0 0x7f0b4410f120 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.262+0000 7f0b4b94a700 1 --2- 172.21.15.112:0/3980843483 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b44108300 0x7f0b441086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.262+0000 7f0b4b94a700 1 --2- 172.21.15.112:0/3980843483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b44102320 0x7f0b441027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.262+0000 7f0b4b94a700 1 -- 172.21.15.112:0/3980843483 >> 172.21.15.112:0/3980843483 conn(0x7f0b44079400 msgr2=0x7f0b44079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.262+0000 7f0b4b94a700 1 -- 172.21.15.112:0/3980843483 shutdown_connections 2024-03-16T09:14:47.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.262+0000 7f0b4b94a700 1 -- 172.21.15.112:0/3980843483 wait complete. 2024-03-16T09:14:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.263+0000 7f0b4b94a700 1 Processor -- start 2024-03-16T09:14:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.263+0000 7f0b4b94a700 1 -- start start 2024-03-16T09:14:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.263+0000 7f0b4b94a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b44102320 0x7f0b440745d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.263+0000 7f0b4b94a700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b44102ce0 0x7f0b44074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.263+0000 7f0b4b94a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b44108300 0x7f0b44075050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.263+0000 7f0b4b94a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0b441126e0 con 0x7f0b44108300 2024-03-16T09:14:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.263+0000 7f0b4b94a700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0b44112560 con 0x7f0b44102ce0 2024-03-16T09:14:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.263+0000 7f0b4b94a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0b44112860 con 0x7f0b44102320 2024-03-16T09:14:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.263+0000 7f0b49ee7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b44108300 0x7f0b44075050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.263+0000 7f0b49ee7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b44108300 0x7f0b44075050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53734/0 (socket says 172.21.15.112:53734) 2024-03-16T09:14:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.263+0000 7f0b49ee7700 1 -- 172.21.15.112:0/1197849188 learned_addr learned my addr 172.21.15.112:0/1197849188 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.263+0000 7f0b48ee5700 1 --2- 172.21.15.112:0/1197849188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b44102ce0 0x7f0b44074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.263+0000 7f0b496e6700 1 --2- 172.21.15.112:0/1197849188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b44102320 0x7f0b440745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.264+0000 7f0b49ee7700 1 -- 172.21.15.112:0/1197849188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b44102320 msgr2=0x7f0b440745d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:47.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.264+0000 7f0b49ee7700 1 --2- 172.21.15.112:0/1197849188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b44102320 0x7f0b440745d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:47.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.264+0000 7f0b49ee7700 1 -- 172.21.15.112:0/1197849188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b44102ce0 msgr2=0x7f0b44074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:47.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.264+0000 7f0b49ee7700 1 --2- 172.21.15.112:0/1197849188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b44102ce0 0x7f0b44074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:47.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.264+0000 7f0b49ee7700 1 -- 172.21.15.112:0/1197849188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b441a7020 con 0x7f0b44108300 2024-03-16T09:14:47.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.264+0000 7f0b48ee5700 1 --2- 172.21.15.112:0/1197849188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b44102ce0 0x7f0b44074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:47.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.264+0000 7f0b49ee7700 1 --2- 172.21.15.112:0/1197849188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b44108300 0x7f0b44075050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c007a50 tx=0x7f0b3c007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:47.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.264+0000 7f0b3a7fc700 1 -- 172.21.15.112:0/1197849188 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c01f070 con 0x7f0b44108300 2024-03-16T09:14:47.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.264+0000 7f0b3a7fc700 1 -- 172.21.15.112:0/1197849188 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0b3c003c00 con 0x7f0b44108300 2024-03-16T09:14:47.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.264+0000 7f0b4b94a700 1 -- 172.21.15.112:0/1197849188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b441a72b0 con 0x7f0b44108300 2024-03-16T09:14:47.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.264+0000 7f0b3a7fc700 1 -- 172.21.15.112:0/1197849188 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c01b6a0 con 0x7f0b44108300 2024-03-16T09:14:47.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.264+0000 7f0b4b94a700 1 -- 172.21.15.112:0/1197849188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b441a76b0 con 0x7f0b44108300 2024-03-16T09:14:47.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.265+0000 7f0b4b94a700 1 -- 172.21.15.112:0/1197849188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b44068f90 con 0x7f0b44108300 2024-03-16T09:14:47.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.266+0000 7f0b3a7fc700 1 -- 172.21.15.112:0/1197849188 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f0b3c022070 con 0x7f0b44108300 2024-03-16T09:14:47.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.267+0000 7f0b3a7fc700 1 --2- 172.21.15.112:0/1197849188 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f0b300a2920 0x7f0b300a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:47.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.267+0000 7f0b496e6700 1 --2- 172.21.15.112:0/1197849188 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f0b300a2920 0x7f0b300a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:47.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.267+0000 7f0b3a7fc700 1 -- 172.21.15.112:0/1197849188 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f0b3c0cfa80 con 0x7f0b44108300 2024-03-16T09:14:47.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.267+0000 7f0b496e6700 1 --2- 172.21.15.112:0/1197849188 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f0b300a2920 0x7f0b300a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009be0 tx=0x7f0b40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:47.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.270+0000 7f0b3a7fc700 1 -- 172.21.15.112:0/1197849188 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0b3c09ca60 con 0x7f0b44108300 2024-03-16T09:14:47.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.470+0000 7f0b4b94a700 1 -- 172.21.15.112:0/1197849188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f0b4402dd30 con 0x7f0b44108300 2024-03-16T09:14:47.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.470+0000 7f0b3a7fc700 1 -- 172.21.15.112:0/1197849188 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0b3c09c360 con 0x7f0b44108300 2024-03-16T09:14:47.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.473+0000 7f0b4b94a700 1 -- 172.21.15.112:0/1197849188 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f0b300a2920 msgr2=0x7f0b300a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:47.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.473+0000 7f0b4b94a700 1 --2- 172.21.15.112:0/1197849188 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f0b300a2920 0x7f0b300a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009be0 tx=0x7f0b40009260 comp rx=0 tx=0).stop 2024-03-16T09:14:47.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.473+0000 7f0b4b94a700 1 -- 172.21.15.112:0/1197849188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b44108300 msgr2=0x7f0b44075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:47.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.473+0000 7f0b4b94a700 1 --2- 172.21.15.112:0/1197849188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b44108300 0x7f0b44075050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c007a50 tx=0x7f0b3c007a80 comp rx=0 tx=0).stop 2024-03-16T09:14:47.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.473+0000 7f0b4b94a700 1 -- 172.21.15.112:0/1197849188 shutdown_connections 2024-03-16T09:14:47.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.473+0000 7f0b4b94a700 1 --2- 172.21.15.112:0/1197849188 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f0b300a2920 0x7f0b300a4de0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:47.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.473+0000 7f0b4b94a700 1 --2- 172.21.15.112:0/1197849188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b44108300 0x7f0b44075050 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:47.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.473+0000 7f0b4b94a700 1 --2- 172.21.15.112:0/1197849188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b44102320 0x7f0b440745d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:47.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.473+0000 7f0b4b94a700 1 --2- 172.21.15.112:0/1197849188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b44102ce0 0x7f0b44074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:47.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.473+0000 7f0b4b94a700 1 -- 172.21.15.112:0/1197849188 >> 172.21.15.112:0/1197849188 conn(0x7f0b44079400 msgr2=0x7f0b44100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:47.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.473+0000 7f0b4b94a700 1 -- 172.21.15.112:0/1197849188 shutdown_connections 2024-03-16T09:14:47.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.473+0000 7f0b4b94a700 1 -- 172.21.15.112:0/1197849188 wait complete. 2024-03-16T09:14:47.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:47 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1197849188' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-16T09:14:47.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:47 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1197849188' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-16T09:14:47.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:47 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1197849188' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-16T09:14:47.855 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070501 got 253403070506 for osd.8 2024-03-16T09:14:47.856 DEBUG:teuthology.parallel:result is None 2024-03-16T09:14:48.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:48 smithi112 ceph-mon[32307]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 48 B/s, 0 objects/s recovering 2024-03-16T09:14:48.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:48 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3302558648' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-16T09:14:48.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:48 smithi194 ceph-mon[36729]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 48 B/s, 0 objects/s recovering 2024-03-16T09:14:48.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:48 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3302558648' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-16T09:14:48.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:48 smithi159 ceph-mon[36739]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 48 B/s, 0 objects/s recovering 2024-03-16T09:14:48.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:48 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3302558648' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-16T09:14:49.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.885+0000 7f6ee93da700 1 -- 172.21.15.112:0/1547561173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ee4101b70 msgr2=0x7f6ee410e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:49.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.885+0000 7f6ee93da700 1 --2- 172.21.15.112:0/1547561173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ee4101b70 0x7f6ee410e050 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00b0a0 tx=0x7f6edc00b3b0 comp rx=0 tx=0).stop 2024-03-16T09:14:49.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.885+0000 7f6ee93da700 1 -- 172.21.15.112:0/1547561173 shutdown_connections 2024-03-16T09:14:49.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.885+0000 7f6ee93da700 1 --2- 172.21.15.112:0/1547561173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ee4107210 0x7f6ee41075f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:49.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.885+0000 7f6ee93da700 1 --2- 172.21.15.112:0/1547561173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ee4101b70 0x7f6ee410e050 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:49.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.885+0000 7f6ee93da700 1 --2- 172.21.15.112:0/1547561173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee41011d0 0x7f6ee4101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:49.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.885+0000 7f6ee93da700 1 -- 172.21.15.112:0/1547561173 >> 172.21.15.112:0/1547561173 conn(0x7f6ee4078100 msgr2=0x7f6ee4078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:49.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.885+0000 7f6ee93da700 1 -- 172.21.15.112:0/1547561173 shutdown_connections 2024-03-16T09:14:49.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.885+0000 7f6ee93da700 1 -- 172.21.15.112:0/1547561173 wait complete. 2024-03-16T09:14:49.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.886+0000 7f6ee93da700 1 Processor -- start 2024-03-16T09:14:49.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.886+0000 7f6ee93da700 1 -- start start 2024-03-16T09:14:49.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.886+0000 7f6ee93da700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ee41011d0 0x7f6ee419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:49.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.886+0000 7f6ee93da700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ee4101b70 0x7f6ee419b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:49.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.886+0000 7f6ee93da700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee4107210 0x7f6ee419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:49.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.886+0000 7f6ee93da700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee41115b0 con 0x7f6ee41011d0 2024-03-16T09:14:49.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.886+0000 7f6ee93da700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee4111430 con 0x7f6ee4101b70 2024-03-16T09:14:49.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.886+0000 7f6ee93da700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee4111730 con 0x7f6ee4107210 2024-03-16T09:14:49.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.886+0000 7f6ee2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ee41011d0 0x7f6ee419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:49.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.886+0000 7f6ee2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ee41011d0 0x7f6ee419ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53752/0 (socket says 172.21.15.112:53752) 2024-03-16T09:14:49.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.887+0000 7f6ee2ffd700 1 -- 172.21.15.112:0/3302558648 learned_addr learned my addr 172.21.15.112:0/3302558648 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:49.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.887+0000 7f6ee27fc700 1 --2- 172.21.15.112:0/3302558648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ee4101b70 0x7f6ee419b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:49.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.887+0000 7f6ee37fe700 1 --2- 172.21.15.112:0/3302558648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee4107210 0x7f6ee419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:49.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.887+0000 7f6ee27fc700 1 -- 172.21.15.112:0/3302558648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee4107210 msgr2=0x7f6ee419f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:49.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.887+0000 7f6ee27fc700 1 --2- 172.21.15.112:0/3302558648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee4107210 0x7f6ee419f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:49.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.887+0000 7f6ee27fc700 1 -- 172.21.15.112:0/3302558648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ee41011d0 msgr2=0x7f6ee419ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:49.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.887+0000 7f6ee27fc700 1 --2- 172.21.15.112:0/3302558648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ee41011d0 0x7f6ee419ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:49.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.887+0000 7f6ee27fc700 1 -- 172.21.15.112:0/3302558648 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee419fd30 con 0x7f6ee4101b70 2024-03-16T09:14:49.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.887+0000 7f6ee2ffd700 1 --2- 172.21.15.112:0/3302558648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ee41011d0 0x7f6ee419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:49.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.887+0000 7f6ee27fc700 1 --2- 172.21.15.112:0/3302558648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ee4101b70 0x7f6ee419b260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ed800e9f0 tx=0x7f6ed800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:49.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.887+0000 7f6ecbfff700 1 -- 172.21.15.112:0/3302558648 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8004220 con 0x7f6ee4101b70 2024-03-16T09:14:49.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.887+0000 7f6ee93da700 1 -- 172.21.15.112:0/3302558648 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee41a0020 con 0x7f6ee4101b70 2024-03-16T09:14:49.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.887+0000 7f6ecbfff700 1 -- 172.21.15.112:0/3302558648 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6ed8004d30 con 0x7f6ee4101b70 2024-03-16T09:14:49.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.887+0000 7f6ecbfff700 1 -- 172.21.15.112:0/3302558648 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8005080 con 0x7f6ee4101b70 2024-03-16T09:14:49.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.887+0000 7f6ee93da700 1 -- 172.21.15.112:0/3302558648 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee41a78a0 con 0x7f6ee4101b70 2024-03-16T09:14:49.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.888+0000 7f6ee93da700 1 -- 172.21.15.112:0/3302558648 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee4068f90 con 0x7f6ee4101b70 2024-03-16T09:14:49.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.889+0000 7f6ecbfff700 1 -- 172.21.15.112:0/3302558648 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f6ed8004750 con 0x7f6ee4101b70 2024-03-16T09:14:49.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.890+0000 7f6ecbfff700 1 --2- 172.21.15.112:0/3302558648 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ecc0a2920 0x7f6ecc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:49.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.890+0000 7f6ee2ffd700 1 --2- 172.21.15.112:0/3302558648 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ecc0a2920 0x7f6ecc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:49.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.890+0000 7f6ecbfff700 1 -- 172.21.15.112:0/3302558648 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f6ed80ceb10 con 0x7f6ee4101b70 2024-03-16T09:14:49.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.890+0000 7f6ee2ffd700 1 --2- 172.21.15.112:0/3302558648 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ecc0a2920 0x7f6ecc0a4de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4011550 tx=0x7f6ed4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:49.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:46.893+0000 7f6ecbfff700 1 -- 172.21.15.112:0/3302558648 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6ed809ba60 con 0x7f6ee4101b70 2024-03-16T09:14:49.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:47.096+0000 7f6ee93da700 1 -- 172.21.15.112:0/3302558648 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f6ee41a7ad0 con 0x7f6ee4101b70 2024-03-16T09:14:49.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:47.097+0000 7f6ecbfff700 1 -- 172.21.15.112:0/3302558648 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f6ed8017e40 con 0x7f6ee4101b70 2024-03-16T09:14:49.420 INFO:teuthology.orchestra.run.smithi112.stdout:343597383707 2024-03-16T09:14:49.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:47.099+0000 7f6ee93da700 1 -- 172.21.15.112:0/3302558648 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ecc0a2920 msgr2=0x7f6ecc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:49.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:47.099+0000 7f6ee93da700 1 --2- 172.21.15.112:0/3302558648 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ecc0a2920 0x7f6ecc0a4de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4011550 tx=0x7f6ed4009260 comp rx=0 tx=0).stop 2024-03-16T09:14:49.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:47.099+0000 7f6ee93da700 1 -- 172.21.15.112:0/3302558648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ee4101b70 msgr2=0x7f6ee419b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:49.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:47.099+0000 7f6ee93da700 1 --2- 172.21.15.112:0/3302558648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ee4101b70 0x7f6ee419b260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6ed800e9f0 tx=0x7f6ed800ed00 comp rx=0 tx=0).stop 2024-03-16T09:14:49.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:47.099+0000 7f6ee93da700 1 -- 172.21.15.112:0/3302558648 shutdown_connections 2024-03-16T09:14:49.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:47.099+0000 7f6ee93da700 1 --2- 172.21.15.112:0/3302558648 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ecc0a2920 0x7f6ecc0a4de0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:49.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:47.099+0000 7f6ee93da700 1 --2- 172.21.15.112:0/3302558648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ee4107210 0x7f6ee419f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:49.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:47.099+0000 7f6ee93da700 1 --2- 172.21.15.112:0/3302558648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ee41011d0 0x7f6ee419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:49.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:47.099+0000 7f6ee93da700 1 --2- 172.21.15.112:0/3302558648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ee4101b70 0x7f6ee419b260 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:49.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:47.099+0000 7f6ee93da700 1 -- 172.21.15.112:0/3302558648 >> 172.21.15.112:0/3302558648 conn(0x7f6ee4078100 msgr2=0x7f6ee4069f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:49.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:47.099+0000 7f6ee93da700 1 -- 172.21.15.112:0/3302558648 shutdown_connections 2024-03-16T09:14:49.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:47.099+0000 7f6ee93da700 1 -- 172.21.15.112:0/3302558648 wait complete. 2024-03-16T09:14:50.059 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383702 got 343597383707 for osd.11 2024-03-16T09:14:50.059 DEBUG:teuthology.parallel:result is None 2024-03-16T09:14:50.310 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:50 smithi112 ceph-mon[32307]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T09:14:50.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:50 smithi194 ceph-mon[36729]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T09:14:50.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:50 smithi159 ceph-mon[36739]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T09:14:50.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.881+0000 7fb41f41e700 1 -- 172.21.15.112:0/645560711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb418108320 msgr2=0x7fb418108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:50.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.881+0000 7fb41f41e700 1 --2- 172.21.15.112:0/645560711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb418108320 0x7fb418108700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb4140099c0 tx=0x7fb414009cd0 comp rx=0 tx=0).stop 2024-03-16T09:14:50.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.881+0000 7fb41f41e700 1 -- 172.21.15.112:0/645560711 shutdown_connections 2024-03-16T09:14:50.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.881+0000 7fb41f41e700 1 --2- 172.21.15.112:0/645560711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb41806a920 0x7fb41810f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:50.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.881+0000 7fb41f41e700 1 --2- 172.21.15.112:0/645560711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb418108320 0x7fb418108700 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:50.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.881+0000 7fb41f41e700 1 --2- 172.21.15.112:0/645560711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb418069f80 0x7fb41806a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:50.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.882+0000 7fb41f41e700 1 -- 172.21.15.112:0/645560711 >> 172.21.15.112:0/645560711 conn(0x7fb418078100 msgr2=0x7fb418078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:50.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.882+0000 7fb41f41e700 1 -- 172.21.15.112:0/645560711 shutdown_connections 2024-03-16T09:14:50.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.882+0000 7fb41f41e700 1 -- 172.21.15.112:0/645560711 wait complete. 2024-03-16T09:14:50.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.882+0000 7fb41f41e700 1 Processor -- start 2024-03-16T09:14:50.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.882+0000 7fb41f41e700 1 -- start start 2024-03-16T09:14:50.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.882+0000 7fb41f41e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb418069f80 0x7fb4181012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:50.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.882+0000 7fb41f41e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb41806a920 0x7fb4181017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:50.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.882+0000 7fb41f41e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb418108320 0x7fb418101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:50.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb41f41e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb418112650 con 0x7fb418108320 2024-03-16T09:14:50.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb41f41e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb4181124d0 con 0x7fb418069f80 2024-03-16T09:14:50.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb41f41e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb4181127d0 con 0x7fb41806a920 2024-03-16T09:14:50.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb41d9bb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb418108320 0x7fb418101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:50.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb41d9bb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb418108320 0x7fb418101d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53798/0 (socket says 172.21.15.112:53798) 2024-03-16T09:14:50.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb41d1ba700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb418069f80 0x7fb4181012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:50.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb41d9bb700 1 -- 172.21.15.112:0/2692049260 learned_addr learned my addr 172.21.15.112:0/2692049260 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:50.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb41c9b9700 1 --2- 172.21.15.112:0/2692049260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb41806a920 0x7fb4181017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:50.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb41d1ba700 1 -- 172.21.15.112:0/2692049260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb41806a920 msgr2=0x7fb4181017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:50.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb41d1ba700 1 --2- 172.21.15.112:0/2692049260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb41806a920 0x7fb4181017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:50.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb41d1ba700 1 -- 172.21.15.112:0/2692049260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb418108320 msgr2=0x7fb418101d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:50.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb41d1ba700 1 --2- 172.21.15.112:0/2692049260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb418108320 0x7fb418101d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:50.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb41d1ba700 1 -- 172.21.15.112:0/2692049260 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4181a7050 con 0x7fb418069f80 2024-03-16T09:14:50.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb41d9bb700 1 --2- 172.21.15.112:0/2692049260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb418108320 0x7fb418101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:50.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb41d1ba700 1 --2- 172.21.15.112:0/2692049260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb418069f80 0x7fb4181012a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb4140098e0 tx=0x7fb41400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:50.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb40e7fc700 1 -- 172.21.15.112:0/2692049260 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414027070 con 0x7fb418069f80 2024-03-16T09:14:50.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb40e7fc700 1 -- 172.21.15.112:0/2692049260 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb41400be40 con 0x7fb418069f80 2024-03-16T09:14:50.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb41f41e700 1 -- 172.21.15.112:0/2692049260 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4181a72e0 con 0x7fb418069f80 2024-03-16T09:14:50.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb40e7fc700 1 -- 172.21.15.112:0/2692049260 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414022db0 con 0x7fb418069f80 2024-03-16T09:14:50.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.883+0000 7fb41f41e700 1 -- 172.21.15.112:0/2692049260 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4181a7730 con 0x7fb418069f80 2024-03-16T09:14:50.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.885+0000 7fb41f41e700 1 -- 172.21.15.112:0/2692049260 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb418068f90 con 0x7fb418069f80 2024-03-16T09:14:50.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.885+0000 7fb40e7fc700 1 -- 172.21.15.112:0/2692049260 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7fb41402a070 con 0x7fb418069f80 2024-03-16T09:14:50.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.886+0000 7fb40e7fc700 1 --2- 172.21.15.112:0/2692049260 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb4040a2920 0x7fb4040a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:50.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.886+0000 7fb41c9b9700 1 --2- 172.21.15.112:0/2692049260 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb4040a2920 0x7fb4040a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:50.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.886+0000 7fb40e7fc700 1 -- 172.21.15.112:0/2692049260 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb4140cf700 con 0x7fb418069f80 2024-03-16T09:14:50.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.886+0000 7fb41c9b9700 1 --2- 172.21.15.112:0/2692049260 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb4040a2920 0x7fb4040a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb418078770 tx=0x7fb408009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:50.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.891+0000 7fb40e7fc700 1 -- 172.21.15.112:0/2692049260 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb41405e740 con 0x7fb418069f80 2024-03-16T09:14:50.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.784+0000 7f1fa4a0d700 1 -- 172.21.15.112:0/2488305686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa0108320 msgr2=0x7f1fa0108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.784+0000 7f1fa4a0d700 1 --2- 172.21.15.112:0/2488305686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa0108320 0x7f1fa0108700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1f900099c0 tx=0x7f1f90009cd0 comp rx=0 tx=0).stop 2024-03-16T09:14:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.784+0000 7f1fa4a0d700 1 -- 172.21.15.112:0/2488305686 shutdown_connections 2024-03-16T09:14:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.784+0000 7f1fa4a0d700 1 --2- 172.21.15.112:0/2488305686 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1fa006a920 0x7f1fa010f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.784+0000 7f1fa4a0d700 1 --2- 172.21.15.112:0/2488305686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa0108320 0x7f1fa0108700 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.784+0000 7f1fa4a0d700 1 --2- 172.21.15.112:0/2488305686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fa0069f80 0x7f1fa006a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.784+0000 7f1fa4a0d700 1 -- 172.21.15.112:0/2488305686 >> 172.21.15.112:0/2488305686 conn(0x7f1fa0078100 msgr2=0x7f1fa0078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.784+0000 7f1fa4a0d700 1 -- 172.21.15.112:0/2488305686 shutdown_connections 2024-03-16T09:14:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.784+0000 7f1fa4a0d700 1 -- 172.21.15.112:0/2488305686 wait complete. 2024-03-16T09:14:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1fa4a0d700 1 Processor -- start 2024-03-16T09:14:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1fa4a0d700 1 -- start start 2024-03-16T09:14:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1fa4a0d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fa0069f80 0x7f1fa01012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1fa4a0d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa006a920 0x7f1fa01017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1fa4a0d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1fa0108320 0x7f1fa0101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1fa4a0d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa0112650 con 0x7f1fa006a920 2024-03-16T09:14:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1fa4a0d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa01124d0 con 0x7f1fa0069f80 2024-03-16T09:14:50.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1fa4a0d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa01127d0 con 0x7f1fa0108320 2024-03-16T09:14:50.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1f9e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fa0069f80 0x7f1fa01012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:50.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1f9dd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa006a920 0x7f1fa01017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:50.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1f9e59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fa0069f80 0x7f1fa01012a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:58418/0 (socket says 172.21.15.112:58418) 2024-03-16T09:14:50.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1f9dd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa006a920 0x7f1fa01017e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53768/0 (socket says 172.21.15.112:53768) 2024-03-16T09:14:50.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1f9e59c700 1 -- 172.21.15.112:0/3462722866 learned_addr learned my addr 172.21.15.112:0/3462722866 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:50.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1f9ed9d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1fa0108320 0x7f1fa0101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:50.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1f9e59c700 1 -- 172.21.15.112:0/3462722866 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1fa0108320 msgr2=0x7f1fa0101d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:50.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1f9e59c700 1 --2- 172.21.15.112:0/3462722866 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1fa0108320 0x7f1fa0101d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:50.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1f9e59c700 1 -- 172.21.15.112:0/3462722866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa006a920 msgr2=0x7f1fa01017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:50.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1f9e59c700 1 --2- 172.21.15.112:0/3462722866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa006a920 0x7f1fa01017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:50.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1f9e59c700 1 -- 172.21.15.112:0/3462722866 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa01a7070 con 0x7f1fa0069f80 2024-03-16T09:14:50.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1f9dd9b700 1 --2- 172.21.15.112:0/3462722866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa006a920 0x7f1fa01017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:50.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.785+0000 7f1f9ed9d700 1 --2- 172.21.15.112:0/3462722866 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1fa0108320 0x7f1fa0101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:50.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.786+0000 7f1f9e59c700 1 --2- 172.21.15.112:0/3462722866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fa0069f80 0x7f1fa01012a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1f900059d0 tx=0x7f1f9000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.786+0000 7f1f8f7fe700 1 -- 172.21.15.112:0/3462722866 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90027070 con 0x7f1fa0069f80 2024-03-16T09:14:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.786+0000 7f1fa4a0d700 1 -- 172.21.15.112:0/3462722866 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa01a7300 con 0x7f1fa0069f80 2024-03-16T09:14:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.786+0000 7f1f8f7fe700 1 -- 172.21.15.112:0/3462722866 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1f9000be40 con 0x7f1fa0069f80 2024-03-16T09:14:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.787+0000 7f1f8f7fe700 1 -- 172.21.15.112:0/3462722866 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f900173f0 con 0x7f1fa0069f80 2024-03-16T09:14:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.787+0000 7f1fa4a0d700 1 -- 172.21.15.112:0/3462722866 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa01a7760 con 0x7f1fa0069f80 2024-03-16T09:14:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.787+0000 7f1fa4a0d700 1 -- 172.21.15.112:0/3462722866 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa0068f90 con 0x7f1fa0069f80 2024-03-16T09:14:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.788+0000 7f1f8f7fe700 1 -- 172.21.15.112:0/3462722866 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f1f9002a070 con 0x7f1fa0069f80 2024-03-16T09:14:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.789+0000 7f1f8f7fe700 1 --2- 172.21.15.112:0/3462722866 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1f880a2920 0x7f1f880a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.789+0000 7f1f9dd9b700 1 --2- 172.21.15.112:0/3462722866 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1f880a2920 0x7f1f880a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.789+0000 7f1f8f7fe700 1 -- 172.21.15.112:0/3462722866 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f1f900cf810 con 0x7f1fa0069f80 2024-03-16T09:14:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.789+0000 7f1f9dd9b700 1 --2- 172.21.15.112:0/3462722866 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1f880a2920 0x7f1f880a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0078690 tx=0x7f1f94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:50.798+0000 7f1f8f7fe700 1 -- 172.21.15.112:0/3462722866 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1f9005e740 con 0x7f1fa0069f80 2024-03-16T09:14:51.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.039+0000 7f1fa4a0d700 1 -- 172.21.15.112:0/3462722866 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f1fa002db70 con 0x7f1fa0069f80 2024-03-16T09:14:51.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.040+0000 7f1f8f7fe700 1 -- 172.21.15.112:0/3462722866 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1f90013070 con 0x7f1fa0069f80 2024-03-16T09:14:51.042 INFO:teuthology.orchestra.run.smithi112.stdout:197568495669 2024-03-16T09:14:51.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.042+0000 7f1fa4a0d700 1 -- 172.21.15.112:0/3462722866 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1f880a2920 msgr2=0x7f1f880a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:51.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.042+0000 7f1fa4a0d700 1 --2- 172.21.15.112:0/3462722866 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1f880a2920 0x7f1f880a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0078690 tx=0x7f1f94009260 comp rx=0 tx=0).stop 2024-03-16T09:14:51.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.042+0000 7f1fa4a0d700 1 -- 172.21.15.112:0/3462722866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fa0069f80 msgr2=0x7f1fa01012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:51.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.042+0000 7f1fa4a0d700 1 --2- 172.21.15.112:0/3462722866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fa0069f80 0x7f1fa01012a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1f900059d0 tx=0x7f1f9000f690 comp rx=0 tx=0).stop 2024-03-16T09:14:51.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.042+0000 7f1fa4a0d700 1 -- 172.21.15.112:0/3462722866 shutdown_connections 2024-03-16T09:14:51.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.042+0000 7f1fa4a0d700 1 --2- 172.21.15.112:0/3462722866 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1f880a2920 0x7f1f880a4de0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:51.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.042+0000 7f1fa4a0d700 1 --2- 172.21.15.112:0/3462722866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fa0069f80 0x7f1fa01012a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:51.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.042+0000 7f1fa4a0d700 1 --2- 172.21.15.112:0/3462722866 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1fa0108320 0x7f1fa0101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:51.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.042+0000 7f1fa4a0d700 1 --2- 172.21.15.112:0/3462722866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1fa006a920 0x7f1fa01017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:51.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.042+0000 7f1fa4a0d700 1 -- 172.21.15.112:0/3462722866 >> 172.21.15.112:0/3462722866 conn(0x7f1fa0078100 msgr2=0x7f1fa00ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:51.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.042+0000 7f1fa4a0d700 1 -- 172.21.15.112:0/3462722866 shutdown_connections 2024-03-16T09:14:51.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.042+0000 7f1fa4a0d700 1 -- 172.21.15.112:0/3462722866 wait complete. 2024-03-16T09:14:51.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.132+0000 7fb41f41e700 1 -- 172.21.15.112:0/2692049260 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb41802db70 con 0x7fb418069f80 2024-03-16T09:14:51.135 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:51 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3462722866' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-16T09:14:51.135 INFO:teuthology.orchestra.run.smithi112.stdout:34359738452 2024-03-16T09:14:51.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.132+0000 7fb40e7fc700 1 -- 172.21.15.112:0/2692049260 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb414013070 con 0x7fb418069f80 2024-03-16T09:14:51.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.135+0000 7fb41f41e700 1 -- 172.21.15.112:0/2692049260 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb4040a2920 msgr2=0x7fb4040a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:51.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.135+0000 7fb41f41e700 1 --2- 172.21.15.112:0/2692049260 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb4040a2920 0x7fb4040a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb418078770 tx=0x7fb408009260 comp rx=0 tx=0).stop 2024-03-16T09:14:51.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.135+0000 7fb41f41e700 1 -- 172.21.15.112:0/2692049260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb418069f80 msgr2=0x7fb4181012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:51.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.135+0000 7fb41f41e700 1 --2- 172.21.15.112:0/2692049260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb418069f80 0x7fb4181012a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb4140098e0 tx=0x7fb41400f690 comp rx=0 tx=0).stop 2024-03-16T09:14:51.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.135+0000 7fb41f41e700 1 -- 172.21.15.112:0/2692049260 shutdown_connections 2024-03-16T09:14:51.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.135+0000 7fb41f41e700 1 --2- 172.21.15.112:0/2692049260 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb4040a2920 0x7fb4040a4de0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:51.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.135+0000 7fb41f41e700 1 --2- 172.21.15.112:0/2692049260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb418069f80 0x7fb4181012a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:51.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.135+0000 7fb41f41e700 1 --2- 172.21.15.112:0/2692049260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb418108320 0x7fb418101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:51.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.135+0000 7fb41f41e700 1 --2- 172.21.15.112:0/2692049260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb41806a920 0x7fb4181017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:51.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.135+0000 7fb41f41e700 1 -- 172.21.15.112:0/2692049260 >> 172.21.15.112:0/2692049260 conn(0x7fb418078100 msgr2=0x7fb4180ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:51.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.135+0000 7fb41f41e700 1 -- 172.21.15.112:0/2692049260 shutdown_connections 2024-03-16T09:14:51.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:51.135+0000 7fb41f41e700 1 -- 172.21.15.112:0/2692049260 wait complete. 2024-03-16T09:14:51.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:51 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3462722866' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-16T09:14:51.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:51 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3462722866' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-16T09:14:51.713 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495664 got 197568495669 for osd.6 2024-03-16T09:14:51.713 DEBUG:teuthology.parallel:result is None 2024-03-16T09:14:52.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:52 smithi112 ceph-mon[32307]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T09:14:52.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:52 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2692049260' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-16T09:14:52.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:52 smithi194 ceph-mon[36729]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T09:14:52.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:52 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2692049260' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-16T09:14:52.450 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738452 for osd.0 2024-03-16T09:14:52.450 DEBUG:teuthology.parallel:result is None 2024-03-16T09:14:52.450 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-16T09:14:52.451 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-16T09:14:52.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:52 smithi159 ceph-mon[36739]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T09:14:52.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:52 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2692049260' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-16T09:14:52.715 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:14:53.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:53 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:53.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:53 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:53.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:53 smithi112 ceph-mon[32307]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:14:54.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:53 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:54.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:53 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:54.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:53 smithi194 ceph-mon[36729]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:14:54.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:53 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:54.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:53 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:54.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:53 smithi159 ceph-mon[36739]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:14:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.468+0000 7f8e79c94700 1 -- 172.21.15.112:0/4026004089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e7406aa20 msgr2=0x7f8e7406aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.468+0000 7f8e79c94700 1 --2- 172.21.15.112:0/4026004089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e7406aa20 0x7f8e7406aea0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8e680099c0 tx=0x7f8e68009cd0 comp rx=0 tx=0).stop 2024-03-16T09:14:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.468+0000 7f8e79c94700 1 -- 172.21.15.112:0/4026004089 shutdown_connections 2024-03-16T09:14:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.468+0000 7f8e79c94700 1 --2- 172.21.15.112:0/4026004089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e74108300 0x7f8e741086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.468+0000 7f8e79c94700 1 --2- 172.21.15.112:0/4026004089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e7406b3e0 0x7f8e7410f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.468+0000 7f8e79c94700 1 --2- 172.21.15.112:0/4026004089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e7406aa20 0x7f8e7406aea0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.468+0000 7f8e79c94700 1 -- 172.21.15.112:0/4026004089 >> 172.21.15.112:0/4026004089 conn(0x7f8e740781e0 msgr2=0x7f8e740785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.468+0000 7f8e79c94700 1 -- 172.21.15.112:0/4026004089 shutdown_connections 2024-03-16T09:14:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.468+0000 7f8e79c94700 1 -- 172.21.15.112:0/4026004089 wait complete. 2024-03-16T09:14:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.469+0000 7f8e79c94700 1 Processor -- start 2024-03-16T09:14:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.469+0000 7f8e79c94700 1 -- start start 2024-03-16T09:14:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.469+0000 7f8e79c94700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e7406aa20 0x7f8e741a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.469+0000 7f8e79c94700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e7406b3e0 0x7f8e741a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.469+0000 7f8e79c94700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e74108300 0x7f8e7419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:54.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.469+0000 7f8e79c94700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8e741126a0 con 0x7f8e7406aa20 2024-03-16T09:14:54.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.469+0000 7f8e79c94700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8e74112520 con 0x7f8e74108300 2024-03-16T09:14:54.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.469+0000 7f8e79c94700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8e74112820 con 0x7f8e7406b3e0 2024-03-16T09:14:54.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.469+0000 7f8e737fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e7406aa20 0x7f8e741a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:54.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.469+0000 7f8e737fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e7406aa20 0x7f8e741a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45638/0 (socket says 172.21.15.112:45638) 2024-03-16T09:14:54.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.469+0000 7f8e737fe700 1 -- 172.21.15.112:0/3876089757 learned_addr learned my addr 172.21.15.112:0/3876089757 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:54.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.469+0000 7f8e73fff700 1 --2- 172.21.15.112:0/3876089757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e74108300 0x7f8e7419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:54.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.469+0000 7f8e72ffd700 1 --2- 172.21.15.112:0/3876089757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e7406b3e0 0x7f8e741a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:54.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.470+0000 7f8e737fe700 1 -- 172.21.15.112:0/3876089757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e7406b3e0 msgr2=0x7f8e741a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:54.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.470+0000 7f8e737fe700 1 --2- 172.21.15.112:0/3876089757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e7406b3e0 0x7f8e741a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:54.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.470+0000 7f8e737fe700 1 -- 172.21.15.112:0/3876089757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e74108300 msgr2=0x7f8e7419aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:54.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.470+0000 7f8e737fe700 1 --2- 172.21.15.112:0/3876089757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e74108300 0x7f8e7419aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:54.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.470+0000 7f8e737fe700 1 -- 172.21.15.112:0/3876089757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7419b200 con 0x7f8e7406aa20 2024-03-16T09:14:54.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.470+0000 7f8e72ffd700 1 --2- 172.21.15.112:0/3876089757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e7406b3e0 0x7f8e741a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:54.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.470+0000 7f8e73fff700 1 --2- 172.21.15.112:0/3876089757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e74108300 0x7f8e7419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:14:54.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.470+0000 7f8e737fe700 1 --2- 172.21.15.112:0/3876089757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e7406aa20 0x7f8e741a08e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400e9a0 tx=0x7f8e6400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:54.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.470+0000 7f8e70ff9700 1 -- 172.21.15.112:0/3876089757 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6400ca10 con 0x7f8e7406aa20 2024-03-16T09:14:54.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.470+0000 7f8e79c94700 1 -- 172.21.15.112:0/3876089757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7419b4f0 con 0x7f8e7406aa20 2024-03-16T09:14:54.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.470+0000 7f8e70ff9700 1 -- 172.21.15.112:0/3876089757 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8e6400cbb0 con 0x7f8e7406aa20 2024-03-16T09:14:54.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.470+0000 7f8e70ff9700 1 -- 172.21.15.112:0/3876089757 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64018710 con 0x7f8e7406aa20 2024-03-16T09:14:54.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.470+0000 7f8e79c94700 1 -- 172.21.15.112:0/3876089757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e741a77b0 con 0x7f8e7406aa20 2024-03-16T09:14:54.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.471+0000 7f8e79c94700 1 -- 172.21.15.112:0/3876089757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e74068f90 con 0x7f8e7406aa20 2024-03-16T09:14:54.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.472+0000 7f8e70ff9700 1 -- 172.21.15.112:0/3876089757 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f8e64010430 con 0x7f8e7406aa20 2024-03-16T09:14:54.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.473+0000 7f8e70ff9700 1 --2- 172.21.15.112:0/3876089757 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8e5c0a2920 0x7f8e5c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:54.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.473+0000 7f8e72ffd700 1 --2- 172.21.15.112:0/3876089757 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8e5c0a2920 0x7f8e5c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:54.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.473+0000 7f8e70ff9700 1 -- 172.21.15.112:0/3876089757 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8e640ced70 con 0x7f8e7406aa20 2024-03-16T09:14:54.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.473+0000 7f8e72ffd700 1 --2- 172.21.15.112:0/3876089757 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8e5c0a2920 0x7f8e5c0a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8e741031b0 tx=0x7f8e68005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:54.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.477+0000 7f8e70ff9700 1 -- 172.21.15.112:0/3876089757 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8e6409bcc0 con 0x7f8e7406aa20 2024-03-16T09:14:54.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.694+0000 7f8e79c94700 1 -- 172.21.15.112:0/3876089757 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8e7410db60 con 0x7f8e5c0a2920 2024-03-16T09:14:54.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.702+0000 7f8e70ff9700 1 -- 172.21.15.112:0/3876089757 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258940 (secure 0 0 0) 0x7f8e7410db60 con 0x7f8e5c0a2920 2024-03-16T09:14:54.708 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:14:54.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.705+0000 7f8e79c94700 1 -- 172.21.15.112:0/3876089757 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8e5c0a2920 msgr2=0x7f8e5c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:54.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.705+0000 7f8e79c94700 1 --2- 172.21.15.112:0/3876089757 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8e5c0a2920 0x7f8e5c0a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8e741031b0 tx=0x7f8e68005fb0 comp rx=0 tx=0).stop 2024-03-16T09:14:54.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.705+0000 7f8e79c94700 1 -- 172.21.15.112:0/3876089757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e7406aa20 msgr2=0x7f8e741a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:54.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.705+0000 7f8e79c94700 1 --2- 172.21.15.112:0/3876089757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e7406aa20 0x7f8e741a08e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400e9a0 tx=0x7f8e6400ecb0 comp rx=0 tx=0).stop 2024-03-16T09:14:54.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.705+0000 7f8e79c94700 1 -- 172.21.15.112:0/3876089757 shutdown_connections 2024-03-16T09:14:54.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.705+0000 7f8e79c94700 1 --2- 172.21.15.112:0/3876089757 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8e5c0a2920 0x7f8e5c0a4de0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:54.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.705+0000 7f8e79c94700 1 --2- 172.21.15.112:0/3876089757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e74108300 0x7f8e7419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:54.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.705+0000 7f8e79c94700 1 --2- 172.21.15.112:0/3876089757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e7406aa20 0x7f8e741a08e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:54.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.705+0000 7f8e79c94700 1 --2- 172.21.15.112:0/3876089757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e7406b3e0 0x7f8e741a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:54.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.705+0000 7f8e79c94700 1 -- 172.21.15.112:0/3876089757 >> 172.21.15.112:0/3876089757 conn(0x7f8e740781e0 msgr2=0x7f8e7410d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:54.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.705+0000 7f8e79c94700 1 -- 172.21.15.112:0/3876089757 shutdown_connections 2024-03-16T09:14:54.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:54.705+0000 7f8e79c94700 1 -- 172.21.15.112:0/3876089757 wait complete. 2024-03-16T09:14:54.711 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-03-16T09:14:55.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:54 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:14:55.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:54 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:14:55.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:54 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:55.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:54 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:14:55.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:54 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:14:55.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:54 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:55.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:54 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:14:55.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:54 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:14:55.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:54 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:14:55.443 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-03-16T09:14:52.894256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653677,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":571,"num_read_kb":397,"num_write":337,"num_write_kb":3046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181900,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352476,"kb_used_data":11556,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589348,"statfs":{"total":1151940427776,"available":1151579492352,"internally_reserved":0,"allocated":11833344,"data_stored":7428411,"data_compressed":112689,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003134"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996728+0000","last_change":"2024-03-16T09:13:37.119969+0000","last_active":"2024-03-16T09:14:31.996728+0000","last_peered":"2024-03-16T09:14:31.996728+0000","last_clean":"2024-03-16T09:14:31.996728+0000","last_became_active":"2024-03-16T09:13:37.119177+0000","last_became_peered":"2024-03-16T09:13:37.119177+0000","last_unstale":"2024-03-16T09:14:31.996728+0000","last_undegraded":"2024-03-16T09:14:31.996728+0000","last_fullsized":"2024-03-16T09:14:31.996728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:25:28.038898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994824+0000","last_change":"2024-03-16T09:13:37.127564+0000","last_active":"2024-03-16T09:14:31.994824+0000","last_peered":"2024-03-16T09:14:31.994824+0000","last_clean":"2024-03-16T09:14:31.994824+0000","last_became_active":"2024-03-16T09:13:37.127214+0000","last_became_peered":"2024-03-16T09:13:37.127214+0000","last_unstale":"2024-03-16T09:14:31.994824+0000","last_undegraded":"2024-03-16T09:14:31.994824+0000","last_fullsized":"2024-03-16T09:14:31.994824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:46:14.655707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994554+0000","last_change":"2024-03-16T09:13:37.126288+0000","last_active":"2024-03-16T09:14:31.994554+0000","last_peered":"2024-03-16T09:14:31.994554+0000","last_clean":"2024-03-16T09:14:31.994554+0000","last_became_active":"2024-03-16T09:13:37.125383+0000","last_became_peered":"2024-03-16T09:13:37.125383+0000","last_unstale":"2024-03-16T09:14:31.994554+0000","last_undegraded":"2024-03-16T09:14:31.994554+0000","last_fullsized":"2024-03-16T09:14:31.994554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:47:09.901508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997078+0000","last_change":"2024-03-16T09:13:37.125503+0000","last_active":"2024-03-16T09:14:31.997078+0000","last_peered":"2024-03-16T09:14:31.997078+0000","last_clean":"2024-03-16T09:14:31.997078+0000","last_became_active":"2024-03-16T09:13:37.125144+0000","last_became_peered":"2024-03-16T09:13:37.125144+0000","last_unstale":"2024-03-16T09:14:31.997078+0000","last_undegraded":"2024-03-16T09:14:31.997078+0000","last_fullsized":"2024-03-16T09:14:31.997078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:30:26.529080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994824+0000","last_change":"2024-03-16T09:13:37.114237+0000","last_active":"2024-03-16T09:14:31.994824+0000","last_peered":"2024-03-16T09:14:31.994824+0000","last_clean":"2024-03-16T09:14:31.994824+0000","last_became_active":"2024-03-16T09:13:37.113595+0000","last_became_peered":"2024-03-16T09:13:37.113595+0000","last_unstale":"2024-03-16T09:14:31.994824+0000","last_undegraded":"2024-03-16T09:14:31.994824+0000","last_fullsized":"2024-03-16T09:14:31.994824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:06:17.735301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.230671+0000","last_change":"2024-03-16T09:13:37.118843+0000","last_active":"2024-03-16T09:14:32.230671+0000","last_peered":"2024-03-16T09:14:32.230671+0000","last_clean":"2024-03-16T09:14:32.230671+0000","last_became_active":"2024-03-16T09:13:37.117086+0000","last_became_peered":"2024-03-16T09:13:37.117086+0000","last_unstale":"2024-03-16T09:14:32.230671+0000","last_undegraded":"2024-03-16T09:14:32.230671+0000","last_fullsized":"2024-03-16T09:14:32.230671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:23:12.929931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997583+0000","last_change":"2024-03-16T09:13:37.133427+0000","last_active":"2024-03-16T09:14:31.997583+0000","last_peered":"2024-03-16T09:14:31.997583+0000","last_clean":"2024-03-16T09:14:31.997583+0000","last_became_active":"2024-03-16T09:13:37.133032+0000","last_became_peered":"2024-03-16T09:13:37.133032+0000","last_unstale":"2024-03-16T09:14:31.997583+0000","last_undegraded":"2024-03-16T09:14:31.997583+0000","last_fullsized":"2024-03-16T09:14:31.997583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:49:28.179883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517199+0000","last_change":"2024-03-16T09:13:37.132132+0000","last_active":"2024-03-16T09:14:32.517199+0000","last_peered":"2024-03-16T09:14:32.517199+0000","last_clean":"2024-03-16T09:14:32.517199+0000","last_became_active":"2024-03-16T09:13:37.131801+0000","last_became_peered":"2024-03-16T09:13:37.131801+0000","last_unstale":"2024-03-16T09:14:32.517199+0000","last_undegraded":"2024-03-16T09:14:32.517199+0000","last_fullsized":"2024-03-16T09:14:32.517199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:45:03.891725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.001301+0000","last_change":"2024-03-16T09:13:37.116378+0000","last_active":"2024-03-16T09:14:32.001301+0000","last_peered":"2024-03-16T09:14:32.001301+0000","last_clean":"2024-03-16T09:14:32.001301+0000","last_became_active":"2024-03-16T09:13:37.116027+0000","last_became_peered":"2024-03-16T09:13:37.116027+0000","last_unstale":"2024-03-16T09:14:32.001301+0000","last_undegraded":"2024-03-16T09:14:32.001301+0000","last_fullsized":"2024-03-16T09:14:32.001301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:14:38.681613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.230911+0000","last_change":"2024-03-16T09:13:37.121417+0000","last_active":"2024-03-16T09:14:32.230911+0000","last_peered":"2024-03-16T09:14:32.230911+0000","last_clean":"2024-03-16T09:14:32.230911+0000","last_became_active":"2024-03-16T09:13:37.121112+0000","last_became_peered":"2024-03-16T09:13:37.121112+0000","last_unstale":"2024-03-16T09:14:32.230911+0000","last_undegraded":"2024-03-16T09:14:32.230911+0000","last_fullsized":"2024-03-16T09:14:32.230911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:38:07.823604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997932+0000","last_change":"2024-03-16T09:13:37.121368+0000","last_active":"2024-03-16T09:14:31.997932+0000","last_peered":"2024-03-16T09:14:31.997932+0000","last_clean":"2024-03-16T09:14:31.997932+0000","last_became_active":"2024-03-16T09:13:37.121060+0000","last_became_peered":"2024-03-16T09:13:37.121060+0000","last_unstale":"2024-03-16T09:14:31.997932+0000","last_undegraded":"2024-03-16T09:14:31.997932+0000","last_fullsized":"2024-03-16T09:14:31.997932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:48:49.168285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997040+0000","last_change":"2024-03-16T09:13:37.127004+0000","last_active":"2024-03-16T09:14:31.997040+0000","last_peered":"2024-03-16T09:14:31.997040+0000","last_clean":"2024-03-16T09:14:31.997040+0000","last_became_active":"2024-03-16T09:13:37.126579+0000","last_became_peered":"2024-03-16T09:13:37.126579+0000","last_unstale":"2024-03-16T09:14:31.997040+0000","last_undegraded":"2024-03-16T09:14:31.997040+0000","last_fullsized":"2024-03-16T09:14:31.997040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:54:18.382514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385477+0000","last_change":"2024-03-16T09:13:37.104534+0000","last_active":"2024-03-16T09:14:32.385477+0000","last_peered":"2024-03-16T09:14:32.385477+0000","last_clean":"2024-03-16T09:14:32.385477+0000","last_became_active":"2024-03-16T09:13:37.104202+0000","last_became_peered":"2024-03-16T09:13:37.104202+0000","last_unstale":"2024-03-16T09:14:32.385477+0000","last_undegraded":"2024-03-16T09:14:32.385477+0000","last_fullsized":"2024-03-16T09:14:32.385477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:40:55.448238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994426+0000","last_change":"2024-03-16T09:13:37.117166+0000","last_active":"2024-03-16T09:14:31.994426+0000","last_peered":"2024-03-16T09:14:31.994426+0000","last_clean":"2024-03-16T09:14:31.994426+0000","last_became_active":"2024-03-16T09:13:37.116747+0000","last_became_peered":"2024-03-16T09:13:37.116747+0000","last_unstale":"2024-03-16T09:14:31.994426+0000","last_undegraded":"2024-03-16T09:14:31.994426+0000","last_fullsized":"2024-03-16T09:14:31.994426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:50:58.705683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.000383+0000","last_change":"2024-03-16T09:14:32.000383+0000","last_active":"2024-03-16T09:14:32.000383+0000","last_peered":"2024-03-16T09:14:32.000383+0000","last_clean":"2024-03-16T09:14:32.000383+0000","last_became_active":"2024-03-16T09:14:32.000027+0000","last_became_peered":"2024-03-16T09:14:32.000027+0000","last_unstale":"2024-03-16T09:14:32.000383+0000","last_undegraded":"2024-03-16T09:14:32.000383+0000","last_fullsized":"2024-03-16T09:14:32.000383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:04:03.478849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.000294+0000","last_change":"2024-03-16T09:13:37.120583+0000","last_active":"2024-03-16T09:14:32.000294+0000","last_peered":"2024-03-16T09:14:32.000294+0000","last_clean":"2024-03-16T09:14:32.000294+0000","last_became_active":"2024-03-16T09:13:37.120217+0000","last_became_peered":"2024-03-16T09:13:37.120217+0000","last_unstale":"2024-03-16T09:14:32.000294+0000","last_undegraded":"2024-03-16T09:14:32.000294+0000","last_fullsized":"2024-03-16T09:14:32.000294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:00:17.653736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996525+0000","last_change":"2024-03-16T09:13:37.121179+0000","last_active":"2024-03-16T09:14:31.996525+0000","last_peered":"2024-03-16T09:14:31.996525+0000","last_clean":"2024-03-16T09:14:31.996525+0000","last_became_active":"2024-03-16T09:13:37.120070+0000","last_became_peered":"2024-03-16T09:13:37.120070+0000","last_unstale":"2024-03-16T09:14:31.996525+0000","last_undegraded":"2024-03-16T09:14:31.996525+0000","last_fullsized":"2024-03-16T09:14:31.996525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:39:24.937068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996804+0000","last_change":"2024-03-16T09:13:37.110074+0000","last_active":"2024-03-16T09:14:31.996804+0000","last_peered":"2024-03-16T09:14:31.996804+0000","last_clean":"2024-03-16T09:14:31.996804+0000","last_became_active":"2024-03-16T09:13:37.109767+0000","last_became_peered":"2024-03-16T09:13:37.109767+0000","last_unstale":"2024-03-16T09:14:31.996804+0000","last_undegraded":"2024-03-16T09:14:31.996804+0000","last_fullsized":"2024-03-16T09:14:31.996804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:20:37.408262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.999135+0000","last_change":"2024-03-16T09:13:37.130791+0000","last_active":"2024-03-16T09:14:31.999135+0000","last_peered":"2024-03-16T09:14:31.999135+0000","last_clean":"2024-03-16T09:14:31.999135+0000","last_became_active":"2024-03-16T09:13:37.130428+0000","last_became_peered":"2024-03-16T09:13:37.130428+0000","last_unstale":"2024-03-16T09:14:31.999135+0000","last_undegraded":"2024-03-16T09:14:31.999135+0000","last_fullsized":"2024-03-16T09:14:31.999135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:32:33.185589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.006081+0000","last_change":"2024-03-16T09:14:32.006081+0000","last_active":"2024-03-16T09:14:32.006081+0000","last_peered":"2024-03-16T09:14:32.006081+0000","last_clean":"2024-03-16T09:14:32.006081+0000","last_became_active":"2024-03-16T09:14:32.000165+0000","last_became_peered":"2024-03-16T09:14:32.000165+0000","last_unstale":"2024-03-16T09:14:32.006081+0000","last_undegraded":"2024-03-16T09:14:32.006081+0000","last_fullsized":"2024-03-16T09:14:32.006081+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:12:54.685988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517549+0000","last_change":"2024-03-16T09:13:37.124946+0000","last_active":"2024-03-16T09:14:32.517549+0000","last_peered":"2024-03-16T09:14:32.517549+0000","last_clean":"2024-03-16T09:14:32.517549+0000","last_became_active":"2024-03-16T09:13:37.124624+0000","last_became_peered":"2024-03-16T09:13:37.124624+0000","last_unstale":"2024-03-16T09:14:32.517549+0000","last_undegraded":"2024-03-16T09:14:32.517549+0000","last_fullsized":"2024-03-16T09:14:32.517549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:18:15.188823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517090+0000","last_change":"2024-03-16T09:13:36.116312+0000","last_active":"2024-03-16T09:14:32.517090+0000","last_peered":"2024-03-16T09:14:32.517090+0000","last_clean":"2024-03-16T09:14:32.517090+0000","last_became_active":"2024-03-16T09:13:36.115999+0000","last_became_peered":"2024-03-16T09:13:36.115999+0000","last_unstale":"2024-03-16T09:14:32.517090+0000","last_undegraded":"2024-03-16T09:14:32.517090+0000","last_fullsized":"2024-03-16T09:14:32.517090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:04:46.795291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994711+0000","last_change":"2024-03-16T09:13:37.126024+0000","last_active":"2024-03-16T09:14:31.994711+0000","last_peered":"2024-03-16T09:14:31.994711+0000","last_clean":"2024-03-16T09:14:31.994711+0000","last_became_active":"2024-03-16T09:13:37.125722+0000","last_became_peered":"2024-03-16T09:13:37.125722+0000","last_unstale":"2024-03-16T09:14:31.994711+0000","last_undegraded":"2024-03-16T09:14:31.994711+0000","last_fullsized":"2024-03-16T09:14:31.994711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:27:24.923022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.005879+0000","last_change":"2024-03-16T09:14:32.005295+0000","last_active":"2024-03-16T09:14:32.005879+0000","last_peered":"2024-03-16T09:14:32.005879+0000","last_clean":"2024-03-16T09:14:32.005879+0000","last_became_active":"2024-03-16T09:14:31.999981+0000","last_became_peered":"2024-03-16T09:14:31.999981+0000","last_unstale":"2024-03-16T09:14:32.005879+0000","last_undegraded":"2024-03-16T09:14:32.005879+0000","last_fullsized":"2024-03-16T09:14:32.005879+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:13:32.661689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.000732+0000","last_change":"2024-03-16T09:13:37.127593+0000","last_active":"2024-03-16T09:14:32.000732+0000","last_peered":"2024-03-16T09:14:32.000732+0000","last_clean":"2024-03-16T09:14:32.000732+0000","last_became_active":"2024-03-16T09:13:37.127255+0000","last_became_peered":"2024-03-16T09:13:37.127255+0000","last_unstale":"2024-03-16T09:14:32.000732+0000","last_undegraded":"2024-03-16T09:14:32.000732+0000","last_fullsized":"2024-03-16T09:14:32.000732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:07:51.263266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.995143+0000","last_change":"2024-03-16T09:13:36.117583+0000","last_active":"2024-03-16T09:14:31.995143+0000","last_peered":"2024-03-16T09:14:31.995143+0000","last_clean":"2024-03-16T09:14:31.995143+0000","last_became_active":"2024-03-16T09:13:36.117270+0000","last_became_peered":"2024-03-16T09:13:36.117270+0000","last_unstale":"2024-03-16T09:14:31.995143+0000","last_undegraded":"2024-03-16T09:14:31.995143+0000","last_fullsized":"2024-03-16T09:14:31.995143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:10:53.541872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248667+0000","last_change":"2024-03-16T09:13:37.119019+0000","last_active":"2024-03-16T09:14:32.248667+0000","last_peered":"2024-03-16T09:14:32.248667+0000","last_clean":"2024-03-16T09:14:32.248667+0000","last_became_active":"2024-03-16T09:13:37.118148+0000","last_became_peered":"2024-03-16T09:13:37.118148+0000","last_unstale":"2024-03-16T09:14:32.248667+0000","last_undegraded":"2024-03-16T09:14:32.248667+0000","last_fullsized":"2024-03-16T09:14:32.248667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:17:43.042779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:50.401230+0000","last_change":"2024-03-16T09:13:36.114415+0000","last_active":"2024-03-16T09:14:50.401230+0000","last_peered":"2024-03-16T09:14:50.401230+0000","last_clean":"2024-03-16T09:14:50.401230+0000","last_became_active":"2024-03-16T09:13:36.114056+0000","last_became_peered":"2024-03-16T09:13:36.114056+0000","last_unstale":"2024-03-16T09:14:50.401230+0000","last_undegraded":"2024-03-16T09:14:50.401230+0000","last_fullsized":"2024-03-16T09:14:50.401230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:50:28.322627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994972+0000","last_change":"2024-03-16T09:13:37.124997+0000","last_active":"2024-03-16T09:14:31.994972+0000","last_peered":"2024-03-16T09:14:31.994972+0000","last_clean":"2024-03-16T09:14:31.994972+0000","last_became_active":"2024-03-16T09:13:37.124664+0000","last_became_peered":"2024-03-16T09:13:37.124664+0000","last_unstale":"2024-03-16T09:14:31.994972+0000","last_undegraded":"2024-03-16T09:14:31.994972+0000","last_fullsized":"2024-03-16T09:14:31.994972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:24:30.104576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.230143+0000","last_change":"2024-03-16T09:13:36.121980+0000","last_active":"2024-03-16T09:14:32.230143+0000","last_peered":"2024-03-16T09:14:32.230143+0000","last_clean":"2024-03-16T09:14:32.230143+0000","last_became_active":"2024-03-16T09:13:36.121703+0000","last_became_peered":"2024-03-16T09:13:36.121703+0000","last_unstale":"2024-03-16T09:14:32.230143+0000","last_undegraded":"2024-03-16T09:14:32.230143+0000","last_fullsized":"2024-03-16T09:14:32.230143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:40:26.601688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248438+0000","last_change":"2024-03-16T09:13:37.123443+0000","last_active":"2024-03-16T09:14:32.248438+0000","last_peered":"2024-03-16T09:14:32.248438+0000","last_clean":"2024-03-16T09:14:32.248438+0000","last_became_active":"2024-03-16T09:13:37.121987+0000","last_became_peered":"2024-03-16T09:13:37.121987+0000","last_unstale":"2024-03-16T09:14:32.248438+0000","last_undegraded":"2024-03-16T09:14:32.248438+0000","last_fullsized":"2024-03-16T09:14:32.248438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:37:08.478552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385000+0000","last_change":"2024-03-16T09:13:36.106391+0000","last_active":"2024-03-16T09:14:32.385000+0000","last_peered":"2024-03-16T09:14:32.385000+0000","last_clean":"2024-03-16T09:14:32.385000+0000","last_became_active":"2024-03-16T09:13:36.106046+0000","last_became_peered":"2024-03-16T09:13:36.106046+0000","last_unstale":"2024-03-16T09:14:32.385000+0000","last_undegraded":"2024-03-16T09:14:32.385000+0000","last_fullsized":"2024-03-16T09:14:32.385000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:03:23.255519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517168+0000","last_change":"2024-03-16T09:13:37.114337+0000","last_active":"2024-03-16T09:14:32.517168+0000","last_peered":"2024-03-16T09:14:32.517168+0000","last_clean":"2024-03-16T09:14:32.517168+0000","last_became_active":"2024-03-16T09:13:37.113991+0000","last_became_peered":"2024-03-16T09:13:37.113991+0000","last_unstale":"2024-03-16T09:14:32.517168+0000","last_undegraded":"2024-03-16T09:14:32.517168+0000","last_fullsized":"2024-03-16T09:14:32.517168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:23:54.141479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248113+0000","last_change":"2024-03-16T09:13:36.113297+0000","last_active":"2024-03-16T09:14:32.248113+0000","last_peered":"2024-03-16T09:14:32.248113+0000","last_clean":"2024-03-16T09:14:32.248113+0000","last_became_active":"2024-03-16T09:13:36.112950+0000","last_became_peered":"2024-03-16T09:13:36.112950+0000","last_unstale":"2024-03-16T09:14:32.248113+0000","last_undegraded":"2024-03-16T09:14:32.248113+0000","last_fullsized":"2024-03-16T09:14:32.248113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:40:17.682450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996565+0000","last_change":"2024-03-16T09:13:37.112239+0000","last_active":"2024-03-16T09:14:31.996565+0000","last_peered":"2024-03-16T09:14:31.996565+0000","last_clean":"2024-03-16T09:14:31.996565+0000","last_became_active":"2024-03-16T09:13:37.111885+0000","last_became_peered":"2024-03-16T09:13:37.111885+0000","last_unstale":"2024-03-16T09:14:31.996565+0000","last_undegraded":"2024-03-16T09:14:31.996565+0000","last_fullsized":"2024-03-16T09:14:31.996565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:49.565846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:49.399879+0000","last_change":"2024-03-16T09:13:36.111341+0000","last_active":"2024-03-16T09:14:49.399879+0000","last_peered":"2024-03-16T09:14:49.399879+0000","last_clean":"2024-03-16T09:14:49.399879+0000","last_became_active":"2024-03-16T09:13:36.110558+0000","last_became_peered":"2024-03-16T09:13:36.110558+0000","last_unstale":"2024-03-16T09:14:49.399879+0000","last_undegraded":"2024-03-16T09:14:49.399879+0000","last_fullsized":"2024-03-16T09:14:49.399879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:56:44.252002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.000951+0000","last_change":"2024-03-16T09:13:37.126808+0000","last_active":"2024-03-16T09:14:32.000951+0000","last_peered":"2024-03-16T09:14:32.000951+0000","last_clean":"2024-03-16T09:14:32.000951+0000","last_became_active":"2024-03-16T09:13:37.126442+0000","last_became_peered":"2024-03-16T09:13:37.126442+0000","last_unstale":"2024-03-16T09:14:32.000951+0000","last_undegraded":"2024-03-16T09:14:32.000951+0000","last_fullsized":"2024-03-16T09:14:32.000951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:04:39.216835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.004330+0000","last_change":"2024-03-16T09:14:32.004330+0000","last_active":"2024-03-16T09:14:32.004330+0000","last_peered":"2024-03-16T09:14:32.004330+0000","last_clean":"2024-03-16T09:14:32.004330+0000","last_became_active":"2024-03-16T09:14:32.001741+0000","last_became_peered":"2024-03-16T09:14:32.001741+0000","last_unstale":"2024-03-16T09:14:32.004330+0000","last_undegraded":"2024-03-16T09:14:32.004330+0000","last_fullsized":"2024-03-16T09:14:32.004330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:08:19.002873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994663+0000","last_change":"2024-03-16T09:13:37.125833+0000","last_active":"2024-03-16T09:14:31.994663+0000","last_peered":"2024-03-16T09:14:31.994663+0000","last_clean":"2024-03-16T09:14:31.994663+0000","last_became_active":"2024-03-16T09:13:37.125508+0000","last_became_peered":"2024-03-16T09:13:37.125508+0000","last_unstale":"2024-03-16T09:14:31.994663+0000","last_undegraded":"2024-03-16T09:14:31.994663+0000","last_fullsized":"2024-03-16T09:14:31.994663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:14:47.749715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248156+0000","last_change":"2024-03-16T09:13:36.112200+0000","last_active":"2024-03-16T09:14:32.248156+0000","last_peered":"2024-03-16T09:14:32.248156+0000","last_clean":"2024-03-16T09:14:32.248156+0000","last_became_active":"2024-03-16T09:13:36.111830+0000","last_became_peered":"2024-03-16T09:13:36.111830+0000","last_unstale":"2024-03-16T09:14:32.248156+0000","last_undegraded":"2024-03-16T09:14:32.248156+0000","last_fullsized":"2024-03-16T09:14:32.248156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:07:57.147960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.000844+0000","last_change":"2024-03-16T09:13:37.118434+0000","last_active":"2024-03-16T09:14:32.000844+0000","last_peered":"2024-03-16T09:14:32.000844+0000","last_clean":"2024-03-16T09:14:32.000844+0000","last_became_active":"2024-03-16T09:13:37.118069+0000","last_became_peered":"2024-03-16T09:13:37.118069+0000","last_unstale":"2024-03-16T09:14:32.000844+0000","last_undegraded":"2024-03-16T09:14:32.000844+0000","last_fullsized":"2024-03-16T09:14:32.000844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:02:56.559384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'27","reported_seq":195,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:50.400964+0000","last_change":"2024-03-16T09:13:36.114591+0000","last_active":"2024-03-16T09:14:50.400964+0000","last_peered":"2024-03-16T09:14:50.400964+0000","last_clean":"2024-03-16T09:14:50.400964+0000","last_became_active":"2024-03-16T09:13:36.114252+0000","last_became_peered":"2024-03-16T09:13:36.114252+0000","last_unstale":"2024-03-16T09:14:50.400964+0000","last_undegraded":"2024-03-16T09:14:50.400964+0000","last_fullsized":"2024-03-16T09:14:50.400964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:37:52.637394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59465,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":149,"num_read_kb":132,"num_write":28,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994943+0000","last_change":"2024-03-16T09:13:37.133102+0000","last_active":"2024-03-16T09:14:31.994943+0000","last_peered":"2024-03-16T09:14:31.994943+0000","last_clean":"2024-03-16T09:14:31.994943+0000","last_became_active":"2024-03-16T09:13:37.132728+0000","last_became_peered":"2024-03-16T09:13:37.132728+0000","last_unstale":"2024-03-16T09:14:31.994943+0000","last_undegraded":"2024-03-16T09:14:31.994943+0000","last_fullsized":"2024-03-16T09:14:31.994943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:54:27.511997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998770+0000","last_change":"2024-03-16T09:13:36.114195+0000","last_active":"2024-03-16T09:14:31.998770+0000","last_peered":"2024-03-16T09:14:31.998770+0000","last_clean":"2024-03-16T09:14:31.998770+0000","last_became_active":"2024-03-16T09:13:36.113829+0000","last_became_peered":"2024-03-16T09:13:36.113829+0000","last_unstale":"2024-03-16T09:14:31.998770+0000","last_undegraded":"2024-03-16T09:14:31.998770+0000","last_fullsized":"2024-03-16T09:14:31.998770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:28:40.509877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385216+0000","last_change":"2024-03-16T09:13:37.132974+0000","last_active":"2024-03-16T09:14:32.385216+0000","last_peered":"2024-03-16T09:14:32.385216+0000","last_clean":"2024-03-16T09:14:32.385216+0000","last_became_active":"2024-03-16T09:13:37.132474+0000","last_became_peered":"2024-03-16T09:13:37.132474+0000","last_unstale":"2024-03-16T09:14:32.385216+0000","last_undegraded":"2024-03-16T09:14:32.385216+0000","last_fullsized":"2024-03-16T09:14:32.385216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:58:04.480441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.247992+0000","last_change":"2024-03-16T09:13:36.115242+0000","last_active":"2024-03-16T09:14:32.247992+0000","last_peered":"2024-03-16T09:14:32.247992+0000","last_clean":"2024-03-16T09:14:32.247992+0000","last_became_active":"2024-03-16T09:13:36.113566+0000","last_became_peered":"2024-03-16T09:13:36.113566+0000","last_unstale":"2024-03-16T09:14:32.247992+0000","last_undegraded":"2024-03-16T09:14:32.247992+0000","last_fullsized":"2024-03-16T09:14:32.247992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:47:50.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.386530+0000","last_change":"2024-03-16T09:14:32.386530+0000","last_active":"2024-03-16T09:14:32.386530+0000","last_peered":"2024-03-16T09:14:32.386530+0000","last_clean":"2024-03-16T09:14:32.386530+0000","last_became_active":"2024-03-16T09:14:32.386088+0000","last_became_peered":"2024-03-16T09:14:32.386088+0000","last_unstale":"2024-03-16T09:14:32.386530+0000","last_undegraded":"2024-03-16T09:14:32.386530+0000","last_fullsized":"2024-03-16T09:14:32.386530+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:44:57.427154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":92,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:46.401864+0000","last_change":"2024-03-16T09:13:36.109585+0000","last_active":"2024-03-16T09:14:46.401864+0000","last_peered":"2024-03-16T09:14:46.401864+0000","last_clean":"2024-03-16T09:14:46.401864+0000","last_became_active":"2024-03-16T09:13:36.109267+0000","last_became_peered":"2024-03-16T09:13:36.109267+0000","last_unstale":"2024-03-16T09:14:46.401864+0000","last_undegraded":"2024-03-16T09:14:46.401864+0000","last_fullsized":"2024-03-16T09:14:46.401864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:26:55.700791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997586+0000","last_change":"2024-03-16T09:13:37.136465+0000","last_active":"2024-03-16T09:14:31.997586+0000","last_peered":"2024-03-16T09:14:31.997586+0000","last_clean":"2024-03-16T09:14:31.997586+0000","last_became_active":"2024-03-16T09:13:37.135933+0000","last_became_peered":"2024-03-16T09:13:37.135933+0000","last_unstale":"2024-03-16T09:14:31.997586+0000","last_undegraded":"2024-03-16T09:14:31.997586+0000","last_fullsized":"2024-03-16T09:14:31.997586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:08:28.490533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994732+0000","last_change":"2024-03-16T09:13:36.119428+0000","last_active":"2024-03-16T09:14:31.994732+0000","last_peered":"2024-03-16T09:14:31.994732+0000","last_clean":"2024-03-16T09:14:31.994732+0000","last_became_active":"2024-03-16T09:13:36.119083+0000","last_became_peered":"2024-03-16T09:13:36.119083+0000","last_unstale":"2024-03-16T09:14:31.994732+0000","last_undegraded":"2024-03-16T09:14:31.994732+0000","last_fullsized":"2024-03-16T09:14:31.994732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:58:56.065151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248311+0000","last_change":"2024-03-16T09:13:37.118082+0000","last_active":"2024-03-16T09:14:32.248311+0000","last_peered":"2024-03-16T09:14:32.248311+0000","last_clean":"2024-03-16T09:14:32.248311+0000","last_became_active":"2024-03-16T09:13:37.117671+0000","last_became_peered":"2024-03-16T09:13:37.117671+0000","last_unstale":"2024-03-16T09:14:32.248311+0000","last_undegraded":"2024-03-16T09:14:32.248311+0000","last_fullsized":"2024-03-16T09:14:32.248311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:14:05.352191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.995286+0000","last_change":"2024-03-16T09:13:36.121077+0000","last_active":"2024-03-16T09:14:31.995286+0000","last_peered":"2024-03-16T09:14:31.995286+0000","last_clean":"2024-03-16T09:14:31.995286+0000","last_became_active":"2024-03-16T09:13:36.120723+0000","last_became_peered":"2024-03-16T09:13:36.120723+0000","last_unstale":"2024-03-16T09:14:31.995286+0000","last_undegraded":"2024-03-16T09:14:31.995286+0000","last_fullsized":"2024-03-16T09:14:31.995286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:58:14.746215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996410+0000","last_change":"2024-03-16T09:13:37.116725+0000","last_active":"2024-03-16T09:14:31.996410+0000","last_peered":"2024-03-16T09:14:31.996410+0000","last_clean":"2024-03-16T09:14:31.996410+0000","last_became_active":"2024-03-16T09:13:37.116406+0000","last_became_peered":"2024-03-16T09:13:37.116406+0000","last_unstale":"2024-03-16T09:14:31.996410+0000","last_undegraded":"2024-03-16T09:14:31.996410+0000","last_fullsized":"2024-03-16T09:14:31.996410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:56:59.136649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996706+0000","last_change":"2024-03-16T09:13:36.110568+0000","last_active":"2024-03-16T09:14:31.996706+0000","last_peered":"2024-03-16T09:14:31.996706+0000","last_clean":"2024-03-16T09:14:31.996706+0000","last_became_active":"2024-03-16T09:13:36.110246+0000","last_became_peered":"2024-03-16T09:13:36.110246+0000","last_unstale":"2024-03-16T09:14:31.996706+0000","last_undegraded":"2024-03-16T09:14:31.996706+0000","last_fullsized":"2024-03-16T09:14:31.996706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:36:50.011941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.230834+0000","last_change":"2024-03-16T09:13:37.122296+0000","last_active":"2024-03-16T09:14:32.230834+0000","last_peered":"2024-03-16T09:14:32.230834+0000","last_clean":"2024-03-16T09:14:32.230834+0000","last_became_active":"2024-03-16T09:13:37.121968+0000","last_became_peered":"2024-03-16T09:13:37.121968+0000","last_unstale":"2024-03-16T09:14:32.230834+0000","last_undegraded":"2024-03-16T09:14:32.230834+0000","last_fullsized":"2024-03-16T09:14:32.230834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:10:43.449397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997895+0000","last_change":"2024-03-16T09:13:36.112262+0000","last_active":"2024-03-16T09:14:31.997895+0000","last_peered":"2024-03-16T09:14:31.997895+0000","last_clean":"2024-03-16T09:14:31.997895+0000","last_became_active":"2024-03-16T09:13:36.111916+0000","last_became_peered":"2024-03-16T09:13:36.111916+0000","last_unstale":"2024-03-16T09:14:31.997895+0000","last_undegraded":"2024-03-16T09:14:31.997895+0000","last_fullsized":"2024-03-16T09:14:31.997895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:27:41.163508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.230779+0000","last_change":"2024-03-16T09:13:37.119814+0000","last_active":"2024-03-16T09:14:32.230779+0000","last_peered":"2024-03-16T09:14:32.230779+0000","last_clean":"2024-03-16T09:14:32.230779+0000","last_became_active":"2024-03-16T09:13:37.119454+0000","last_became_peered":"2024-03-16T09:13:37.119454+0000","last_unstale":"2024-03-16T09:14:32.230779+0000","last_undegraded":"2024-03-16T09:14:32.230779+0000","last_fullsized":"2024-03-16T09:14:32.230779+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:57:25.988044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.000336+0000","last_change":"2024-03-16T09:14:31.999902+0000","last_active":"2024-03-16T09:14:32.000336+0000","last_peered":"2024-03-16T09:14:32.000336+0000","last_clean":"2024-03-16T09:14:32.000336+0000","last_became_active":"2024-03-16T09:14:31.999609+0000","last_became_peered":"2024-03-16T09:14:31.999609+0000","last_unstale":"2024-03-16T09:14:32.000336+0000","last_undegraded":"2024-03-16T09:14:32.000336+0000","last_fullsized":"2024-03-16T09:14:32.000336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:58:47.495746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":91,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:46.401409+0000","last_change":"2024-03-16T09:13:36.111300+0000","last_active":"2024-03-16T09:14:46.401409+0000","last_peered":"2024-03-16T09:14:46.401409+0000","last_clean":"2024-03-16T09:14:46.401409+0000","last_became_active":"2024-03-16T09:13:36.110966+0000","last_became_peered":"2024-03-16T09:13:36.110966+0000","last_unstale":"2024-03-16T09:14:46.401409+0000","last_undegraded":"2024-03-16T09:14:46.401409+0000","last_fullsized":"2024-03-16T09:14:46.401409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:18:33.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517362+0000","last_change":"2024-03-16T09:13:37.125791+0000","last_active":"2024-03-16T09:14:32.517362+0000","last_peered":"2024-03-16T09:14:32.517362+0000","last_clean":"2024-03-16T09:14:32.517362+0000","last_became_active":"2024-03-16T09:13:37.125448+0000","last_became_peered":"2024-03-16T09:13:37.125448+0000","last_unstale":"2024-03-16T09:14:32.517362+0000","last_undegraded":"2024-03-16T09:14:32.517362+0000","last_fullsized":"2024-03-16T09:14:32.517362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:51:30.767750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996257+0000","last_change":"2024-03-16T09:13:36.107462+0000","last_active":"2024-03-16T09:14:31.996257+0000","last_peered":"2024-03-16T09:14:31.996257+0000","last_clean":"2024-03-16T09:14:31.996257+0000","last_became_active":"2024-03-16T09:13:36.107115+0000","last_became_peered":"2024-03-16T09:13:36.107115+0000","last_unstale":"2024-03-16T09:14:31.996257+0000","last_undegraded":"2024-03-16T09:14:31.996257+0000","last_fullsized":"2024-03-16T09:14:31.996257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:37.796719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":341,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997675+0000","last_change":"2024-03-16T09:12:24.331159+0000","last_active":"2024-03-16T09:14:31.997675+0000","last_peered":"2024-03-16T09:14:31.997675+0000","last_clean":"2024-03-16T09:14:31.997675+0000","last_became_active":"2024-03-16T09:12:24.330554+0000","last_became_peered":"2024-03-16T09:12:24.330554+0000","last_unstale":"2024-03-16T09:14:31.997675+0000","last_undegraded":"2024-03-16T09:14:31.997675+0000","last_fullsized":"2024-03-16T09:14:31.997675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:08:57.697012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:08:57.697012+0000","last_clean_scrub_stamp":"2024-03-16T09:08:57.697012+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:43:02.986865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385248+0000","last_change":"2024-03-16T09:13:37.114299+0000","last_active":"2024-03-16T09:14:32.385248+0000","last_peered":"2024-03-16T09:14:32.385248+0000","last_clean":"2024-03-16T09:14:32.385248+0000","last_became_active":"2024-03-16T09:13:37.113981+0000","last_became_peered":"2024-03-16T09:13:37.113981+0000","last_unstale":"2024-03-16T09:14:32.385248+0000","last_undegraded":"2024-03-16T09:14:32.385248+0000","last_fullsized":"2024-03-16T09:14:32.385248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:17:15.882130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997873+0000","last_change":"2024-03-16T09:13:36.113962+0000","last_active":"2024-03-16T09:14:31.997873+0000","last_peered":"2024-03-16T09:14:31.997873+0000","last_clean":"2024-03-16T09:14:31.997873+0000","last_became_active":"2024-03-16T09:13:36.113616+0000","last_became_peered":"2024-03-16T09:13:36.113616+0000","last_unstale":"2024-03-16T09:14:31.997873+0000","last_undegraded":"2024-03-16T09:14:31.997873+0000","last_fullsized":"2024-03-16T09:14:31.997873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:15:15.004360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.995031+0000","last_change":"2024-03-16T09:13:36.121946+0000","last_active":"2024-03-16T09:14:31.995031+0000","last_peered":"2024-03-16T09:14:31.995031+0000","last_clean":"2024-03-16T09:14:31.995031+0000","last_became_active":"2024-03-16T09:13:36.121209+0000","last_became_peered":"2024-03-16T09:13:36.121209+0000","last_unstale":"2024-03-16T09:14:31.995031+0000","last_undegraded":"2024-03-16T09:14:31.995031+0000","last_fullsized":"2024-03-16T09:14:31.995031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:09:51.139468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385432+0000","last_change":"2024-03-16T09:13:37.128931+0000","last_active":"2024-03-16T09:14:32.385432+0000","last_peered":"2024-03-16T09:14:32.385432+0000","last_clean":"2024-03-16T09:14:32.385432+0000","last_became_active":"2024-03-16T09:13:37.128515+0000","last_became_peered":"2024-03-16T09:13:37.128515+0000","last_unstale":"2024-03-16T09:14:32.385432+0000","last_undegraded":"2024-03-16T09:14:32.385432+0000","last_fullsized":"2024-03-16T09:14:32.385432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:58:46.429144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996860+0000","last_change":"2024-03-16T09:13:36.118831+0000","last_active":"2024-03-16T09:14:31.996860+0000","last_peered":"2024-03-16T09:14:31.996860+0000","last_clean":"2024-03-16T09:14:31.996860+0000","last_became_active":"2024-03-16T09:13:36.118489+0000","last_became_peered":"2024-03-16T09:13:36.118489+0000","last_unstale":"2024-03-16T09:14:31.996860+0000","last_undegraded":"2024-03-16T09:14:31.996860+0000","last_fullsized":"2024-03-16T09:14:31.996860+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:55:34.089451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517648+0000","last_change":"2024-03-16T09:13:37.120581+0000","last_active":"2024-03-16T09:14:32.517648+0000","last_peered":"2024-03-16T09:14:32.517648+0000","last_clean":"2024-03-16T09:14:32.517648+0000","last_became_active":"2024-03-16T09:13:37.120266+0000","last_became_peered":"2024-03-16T09:13:37.120266+0000","last_unstale":"2024-03-16T09:14:32.517648+0000","last_undegraded":"2024-03-16T09:14:32.517648+0000","last_fullsized":"2024-03-16T09:14:32.517648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:16:10.784229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996414+0000","last_change":"2024-03-16T09:13:36.107125+0000","last_active":"2024-03-16T09:14:31.996414+0000","last_peered":"2024-03-16T09:14:31.996414+0000","last_clean":"2024-03-16T09:14:31.996414+0000","last_became_active":"2024-03-16T09:13:36.106795+0000","last_became_peered":"2024-03-16T09:13:36.106795+0000","last_unstale":"2024-03-16T09:14:31.996414+0000","last_undegraded":"2024-03-16T09:14:31.996414+0000","last_fullsized":"2024-03-16T09:14:31.996414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:29:10.130565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994815+0000","last_change":"2024-03-16T09:13:37.123013+0000","last_active":"2024-03-16T09:14:31.994815+0000","last_peered":"2024-03-16T09:14:31.994815+0000","last_clean":"2024-03-16T09:14:31.994815+0000","last_became_active":"2024-03-16T09:13:37.122642+0000","last_became_peered":"2024-03-16T09:13:37.122642+0000","last_unstale":"2024-03-16T09:14:31.994815+0000","last_undegraded":"2024-03-16T09:14:31.994815+0000","last_fullsized":"2024-03-16T09:14:31.994815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:35:05.297337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.000335+0000","last_change":"2024-03-16T09:13:36.110868+0000","last_active":"2024-03-16T09:14:32.000335+0000","last_peered":"2024-03-16T09:14:32.000335+0000","last_clean":"2024-03-16T09:14:32.000335+0000","last_became_active":"2024-03-16T09:13:36.110536+0000","last_became_peered":"2024-03-16T09:13:36.110536+0000","last_unstale":"2024-03-16T09:14:32.000335+0000","last_undegraded":"2024-03-16T09:14:32.000335+0000","last_fullsized":"2024-03-16T09:14:32.000335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:07:06.829458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994742+0000","last_change":"2024-03-16T09:13:37.116579+0000","last_active":"2024-03-16T09:14:31.994742+0000","last_peered":"2024-03-16T09:14:31.994742+0000","last_clean":"2024-03-16T09:14:31.994742+0000","last_became_active":"2024-03-16T09:13:37.116226+0000","last_became_peered":"2024-03-16T09:13:37.116226+0000","last_unstale":"2024-03-16T09:14:31.994742+0000","last_undegraded":"2024-03-16T09:14:31.994742+0000","last_fullsized":"2024-03-16T09:14:31.994742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:08:39.015162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517053+0000","last_change":"2024-03-16T09:13:36.107881+0000","last_active":"2024-03-16T09:14:32.517053+0000","last_peered":"2024-03-16T09:14:32.517053+0000","last_clean":"2024-03-16T09:14:32.517053+0000","last_became_active":"2024-03-16T09:13:36.107538+0000","last_became_peered":"2024-03-16T09:13:36.107538+0000","last_unstale":"2024-03-16T09:14:32.517053+0000","last_undegraded":"2024-03-16T09:14:32.517053+0000","last_fullsized":"2024-03-16T09:14:32.517053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:49:41.932089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996243+0000","last_change":"2024-03-16T09:13:37.118580+0000","last_active":"2024-03-16T09:14:31.996243+0000","last_peered":"2024-03-16T09:14:31.996243+0000","last_clean":"2024-03-16T09:14:31.996243+0000","last_became_active":"2024-03-16T09:13:37.118166+0000","last_became_peered":"2024-03-16T09:13:37.118166+0000","last_unstale":"2024-03-16T09:14:31.996243+0000","last_undegraded":"2024-03-16T09:14:31.996243+0000","last_fullsized":"2024-03-16T09:14:31.996243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:10:13.957439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998331+0000","last_change":"2024-03-16T09:13:36.122307+0000","last_active":"2024-03-16T09:14:31.998331+0000","last_peered":"2024-03-16T09:14:31.998331+0000","last_clean":"2024-03-16T09:14:31.998331+0000","last_became_active":"2024-03-16T09:13:36.121961+0000","last_became_peered":"2024-03-16T09:13:36.121961+0000","last_unstale":"2024-03-16T09:14:31.998331+0000","last_undegraded":"2024-03-16T09:14:31.998331+0000","last_fullsized":"2024-03-16T09:14:31.998331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:50:03.482300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517496+0000","last_change":"2024-03-16T09:13:37.131918+0000","last_active":"2024-03-16T09:14:32.517496+0000","last_peered":"2024-03-16T09:14:32.517496+0000","last_clean":"2024-03-16T09:14:32.517496+0000","last_became_active":"2024-03-16T09:13:37.131560+0000","last_became_peered":"2024-03-16T09:13:37.131560+0000","last_unstale":"2024-03-16T09:14:32.517496+0000","last_undegraded":"2024-03-16T09:14:32.517496+0000","last_fullsized":"2024-03-16T09:14:32.517496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:57.315125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994873+0000","last_change":"2024-03-16T09:13:36.120192+0000","last_active":"2024-03-16T09:14:31.994873+0000","last_peered":"2024-03-16T09:14:31.994873+0000","last_clean":"2024-03-16T09:14:31.994873+0000","last_became_active":"2024-03-16T09:13:36.119063+0000","last_became_peered":"2024-03-16T09:13:36.119063+0000","last_unstale":"2024-03-16T09:14:31.994873+0000","last_undegraded":"2024-03-16T09:14:31.994873+0000","last_fullsized":"2024-03-16T09:14:31.994873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:18:56.616624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998065+0000","last_change":"2024-03-16T09:13:37.129597+0000","last_active":"2024-03-16T09:14:31.998065+0000","last_peered":"2024-03-16T09:14:31.998065+0000","last_clean":"2024-03-16T09:14:31.998065+0000","last_became_active":"2024-03-16T09:13:37.129248+0000","last_became_peered":"2024-03-16T09:13:37.129248+0000","last_unstale":"2024-03-16T09:14:31.998065+0000","last_undegraded":"2024-03-16T09:14:31.998065+0000","last_fullsized":"2024-03-16T09:14:31.998065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:41:11.118192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.010274+0000","last_change":"2024-03-16T09:14:32.009577+0000","last_active":"2024-03-16T09:14:32.010274+0000","last_peered":"2024-03-16T09:14:32.010274+0000","last_clean":"2024-03-16T09:14:32.010274+0000","last_became_active":"2024-03-16T09:14:32.000521+0000","last_became_peered":"2024-03-16T09:14:32.000521+0000","last_unstale":"2024-03-16T09:14:32.010274+0000","last_undegraded":"2024-03-16T09:14:32.010274+0000","last_fullsized":"2024-03-16T09:14:32.010274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:29:07.221252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998028+0000","last_change":"2024-03-16T09:13:37.120317+0000","last_active":"2024-03-16T09:14:31.998028+0000","last_peered":"2024-03-16T09:14:31.998028+0000","last_clean":"2024-03-16T09:14:31.998028+0000","last_became_active":"2024-03-16T09:13:37.119974+0000","last_became_peered":"2024-03-16T09:13:37.119974+0000","last_unstale":"2024-03-16T09:14:31.998028+0000","last_undegraded":"2024-03-16T09:14:31.998028+0000","last_fullsized":"2024-03-16T09:14:31.998028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:25:34.774377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248531+0000","last_change":"2024-03-16T09:13:36.112395+0000","last_active":"2024-03-16T09:14:32.248531+0000","last_peered":"2024-03-16T09:14:32.248531+0000","last_clean":"2024-03-16T09:14:32.248531+0000","last_became_active":"2024-03-16T09:13:36.112036+0000","last_became_peered":"2024-03-16T09:13:36.112036+0000","last_unstale":"2024-03-16T09:14:32.248531+0000","last_undegraded":"2024-03-16T09:14:32.248531+0000","last_fullsized":"2024-03-16T09:14:32.248531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:41:02.895820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998157+0000","last_change":"2024-03-16T09:13:37.133845+0000","last_active":"2024-03-16T09:14:31.998157+0000","last_peered":"2024-03-16T09:14:31.998157+0000","last_clean":"2024-03-16T09:14:31.998157+0000","last_became_active":"2024-03-16T09:13:37.132833+0000","last_became_peered":"2024-03-16T09:13:37.132833+0000","last_unstale":"2024-03-16T09:14:31.998157+0000","last_undegraded":"2024-03-16T09:14:31.998157+0000","last_fullsized":"2024-03-16T09:14:31.998157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:12:18.049210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385012+0000","last_change":"2024-03-16T09:13:36.114830+0000","last_active":"2024-03-16T09:14:32.385012+0000","last_peered":"2024-03-16T09:14:32.385012+0000","last_clean":"2024-03-16T09:14:32.385012+0000","last_became_active":"2024-03-16T09:13:36.113898+0000","last_became_peered":"2024-03-16T09:13:36.113898+0000","last_unstale":"2024-03-16T09:14:32.385012+0000","last_undegraded":"2024-03-16T09:14:32.385012+0000","last_fullsized":"2024-03-16T09:14:32.385012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:27:38.910819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998466+0000","last_change":"2024-03-16T09:13:37.115991+0000","last_active":"2024-03-16T09:14:31.998466+0000","last_peered":"2024-03-16T09:14:31.998466+0000","last_clean":"2024-03-16T09:14:31.998466+0000","last_became_active":"2024-03-16T09:13:37.115671+0000","last_became_peered":"2024-03-16T09:13:37.115671+0000","last_unstale":"2024-03-16T09:14:31.998466+0000","last_undegraded":"2024-03-16T09:14:31.998466+0000","last_fullsized":"2024-03-16T09:14:31.998466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:05:59.703675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996265+0000","last_change":"2024-03-16T09:13:37.115122+0000","last_active":"2024-03-16T09:14:31.996265+0000","last_peered":"2024-03-16T09:14:31.996265+0000","last_clean":"2024-03-16T09:14:31.996265+0000","last_became_active":"2024-03-16T09:13:37.114779+0000","last_became_peered":"2024-03-16T09:13:37.114779+0000","last_unstale":"2024-03-16T09:14:31.996265+0000","last_undegraded":"2024-03-16T09:14:31.996265+0000","last_fullsized":"2024-03-16T09:14:31.996265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:50:02.051104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.999014+0000","last_change":"2024-03-16T09:13:37.116542+0000","last_active":"2024-03-16T09:14:31.999014+0000","last_peered":"2024-03-16T09:14:31.999014+0000","last_clean":"2024-03-16T09:14:31.999014+0000","last_became_active":"2024-03-16T09:13:37.116224+0000","last_became_peered":"2024-03-16T09:13:37.116224+0000","last_unstale":"2024-03-16T09:14:31.999014+0000","last_undegraded":"2024-03-16T09:14:31.999014+0000","last_fullsized":"2024-03-16T09:14:31.999014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:32:40.106979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998208+0000","last_change":"2024-03-16T09:13:37.128750+0000","last_active":"2024-03-16T09:14:31.998208+0000","last_peered":"2024-03-16T09:14:31.998208+0000","last_clean":"2024-03-16T09:14:31.998208+0000","last_became_active":"2024-03-16T09:13:37.128411+0000","last_became_peered":"2024-03-16T09:13:37.128411+0000","last_unstale":"2024-03-16T09:14:31.998208+0000","last_undegraded":"2024-03-16T09:14:31.998208+0000","last_fullsized":"2024-03-16T09:14:31.998208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:01:02.119578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994531+0000","last_change":"2024-03-16T09:13:37.124024+0000","last_active":"2024-03-16T09:14:31.994531+0000","last_peered":"2024-03-16T09:14:31.994531+0000","last_clean":"2024-03-16T09:14:31.994531+0000","last_became_active":"2024-03-16T09:13:37.123721+0000","last_became_peered":"2024-03-16T09:13:37.123721+0000","last_unstale":"2024-03-16T09:14:31.994531+0000","last_undegraded":"2024-03-16T09:14:31.994531+0000","last_fullsized":"2024-03-16T09:14:31.994531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:29:28.079163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385428+0000","last_change":"2024-03-16T09:13:37.116603+0000","last_active":"2024-03-16T09:14:32.385428+0000","last_peered":"2024-03-16T09:14:32.385428+0000","last_clean":"2024-03-16T09:14:32.385428+0000","last_became_active":"2024-03-16T09:13:37.116279+0000","last_became_peered":"2024-03-16T09:13:37.116279+0000","last_unstale":"2024-03-16T09:14:32.385428+0000","last_undegraded":"2024-03-16T09:14:32.385428+0000","last_fullsized":"2024-03-16T09:14:32.385428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:25:13.001817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997029+0000","last_change":"2024-03-16T09:13:37.116863+0000","last_active":"2024-03-16T09:14:31.997029+0000","last_peered":"2024-03-16T09:14:31.997029+0000","last_clean":"2024-03-16T09:14:31.997029+0000","last_became_active":"2024-03-16T09:13:37.116543+0000","last_became_peered":"2024-03-16T09:13:37.116543+0000","last_unstale":"2024-03-16T09:14:31.997029+0000","last_undegraded":"2024-03-16T09:14:31.997029+0000","last_fullsized":"2024-03-16T09:14:31.997029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:06:35.501566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998909+0000","last_change":"2024-03-16T09:13:37.120325+0000","last_active":"2024-03-16T09:14:31.998909+0000","last_peered":"2024-03-16T09:14:31.998909+0000","last_clean":"2024-03-16T09:14:31.998909+0000","last_became_active":"2024-03-16T09:13:37.119985+0000","last_became_peered":"2024-03-16T09:13:37.119985+0000","last_unstale":"2024-03-16T09:14:31.998909+0000","last_undegraded":"2024-03-16T09:14:31.998909+0000","last_fullsized":"2024-03-16T09:14:31.998909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:35.419225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517644+0000","last_change":"2024-03-16T09:13:37.119604+0000","last_active":"2024-03-16T09:14:32.517644+0000","last_peered":"2024-03-16T09:14:32.517644+0000","last_clean":"2024-03-16T09:14:32.517644+0000","last_became_active":"2024-03-16T09:13:37.119019+0000","last_became_peered":"2024-03-16T09:13:37.119019+0000","last_unstale":"2024-03-16T09:14:32.517644+0000","last_undegraded":"2024-03-16T09:14:32.517644+0000","last_fullsized":"2024-03-16T09:14:32.517644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:42:57.902370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248334+0000","last_change":"2024-03-16T09:13:37.123845+0000","last_active":"2024-03-16T09:14:32.248334+0000","last_peered":"2024-03-16T09:14:32.248334+0000","last_clean":"2024-03-16T09:14:32.248334+0000","last_became_active":"2024-03-16T09:13:37.121753+0000","last_became_peered":"2024-03-16T09:13:37.121753+0000","last_unstale":"2024-03-16T09:14:32.248334+0000","last_undegraded":"2024-03-16T09:14:32.248334+0000","last_fullsized":"2024-03-16T09:14:32.248334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:23:01.289378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517187+0000","last_change":"2024-03-16T09:13:37.118182+0000","last_active":"2024-03-16T09:14:32.517187+0000","last_peered":"2024-03-16T09:14:32.517187+0000","last_clean":"2024-03-16T09:14:32.517187+0000","last_became_active":"2024-03-16T09:13:37.117820+0000","last_became_peered":"2024-03-16T09:13:37.117820+0000","last_unstale":"2024-03-16T09:14:32.517187+0000","last_undegraded":"2024-03-16T09:14:32.517187+0000","last_fullsized":"2024-03-16T09:14:32.517187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:27:31.393266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.000632+0000","last_change":"2024-03-16T09:13:37.121348+0000","last_active":"2024-03-16T09:14:32.000632+0000","last_peered":"2024-03-16T09:14:32.000632+0000","last_clean":"2024-03-16T09:14:32.000632+0000","last_became_active":"2024-03-16T09:13:37.120826+0000","last_became_peered":"2024-03-16T09:13:37.120826+0000","last_unstale":"2024-03-16T09:14:32.000632+0000","last_undegraded":"2024-03-16T09:14:32.000632+0000","last_fullsized":"2024-03-16T09:14:32.000632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:36:33.456230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385469+0000","last_change":"2024-03-16T09:13:37.131889+0000","last_active":"2024-03-16T09:14:32.385469+0000","last_peered":"2024-03-16T09:14:32.385469+0000","last_clean":"2024-03-16T09:14:32.385469+0000","last_became_active":"2024-03-16T09:13:37.131564+0000","last_became_peered":"2024-03-16T09:13:37.131564+0000","last_unstale":"2024-03-16T09:14:32.385469+0000","last_undegraded":"2024-03-16T09:14:32.385469+0000","last_fullsized":"2024-03-16T09:14:32.385469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:43:34.255378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998395+0000","last_change":"2024-03-16T09:14:31.998395+0000","last_active":"2024-03-16T09:14:31.998395+0000","last_peered":"2024-03-16T09:14:31.998395+0000","last_clean":"2024-03-16T09:14:31.998395+0000","last_became_active":"2024-03-16T09:14:31.998063+0000","last_became_peered":"2024-03-16T09:14:31.998063+0000","last_unstale":"2024-03-16T09:14:31.998395+0000","last_undegraded":"2024-03-16T09:14:31.998395+0000","last_fullsized":"2024-03-16T09:14:31.998395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:50:57.125765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280795,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63075,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":445,"num_read_kb":288,"num_write":154,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1164,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":189225,"data_compressed":5547,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383708,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":456779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645345,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":1107143,"data_compressed":37563,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808871,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070507,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":456353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332145,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":1047310,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495669,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":455822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":515080,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":457093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":515848,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":456133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":456841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757307,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":1047576,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":59465,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":60233,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":61043,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T09:14:55.448 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-16T09:14:55.688 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:14:56.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:55 smithi112 ceph-mon[32307]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:14:56.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:55 smithi112 ceph-mon[32307]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T09:14:56.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:55 smithi194 ceph-mon[36729]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:14:56.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:55 smithi194 ceph-mon[36729]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T09:14:56.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:55 smithi159 ceph-mon[36739]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:14:56.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:55 smithi159 ceph-mon[36739]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T09:14:57.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.445+0000 7f8aa0c9e700 1 -- 172.21.15.112:0/4222799881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a9c101b70 msgr2=0x7f8a9c10e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:57.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.445+0000 7f8aa0c9e700 1 --2- 172.21.15.112:0/4222799881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a9c101b70 0x7f8a9c10e010 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400b0a0 tx=0x7f8a9400b3b0 comp rx=0 tx=0).stop 2024-03-16T09:14:57.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.445+0000 7f8aa0c9e700 1 -- 172.21.15.112:0/4222799881 shutdown_connections 2024-03-16T09:14:57.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.445+0000 7f8aa0c9e700 1 --2- 172.21.15.112:0/4222799881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a9c101b70 0x7f8a9c10e010 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:57.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.445+0000 7f8aa0c9e700 1 --2- 172.21.15.112:0/4222799881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a9c107260 0x7f8a9c107640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:57.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.445+0000 7f8aa0c9e700 1 --2- 172.21.15.112:0/4222799881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a9c1011b0 0x7f8a9c101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:57.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.445+0000 7f8aa0c9e700 1 -- 172.21.15.112:0/4222799881 >> 172.21.15.112:0/4222799881 conn(0x7f8a9c0781e0 msgr2=0x7f8a9c0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:57.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.445+0000 7f8aa0c9e700 1 -- 172.21.15.112:0/4222799881 shutdown_connections 2024-03-16T09:14:57.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.445+0000 7f8aa0c9e700 1 -- 172.21.15.112:0/4222799881 wait complete. 2024-03-16T09:14:57.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.445+0000 7f8aa0c9e700 1 Processor -- start 2024-03-16T09:14:57.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.446+0000 7f8aa0c9e700 1 -- start start 2024-03-16T09:14:57.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.446+0000 7f8aa0c9e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a9c1011b0 0x7f8a9c1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:57.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.446+0000 7f8aa0c9e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a9c101b70 0x7f8a9c199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:57.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.446+0000 7f8aa0c9e700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a9c107260 0x7f8a9c1a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:57.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.446+0000 7f8aa0c9e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c1115d0 con 0x7f8a9c101b70 2024-03-16T09:14:57.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.446+0000 7f8aa0c9e700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c111450 con 0x7f8a9c107260 2024-03-16T09:14:57.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.446+0000 7f8aa0c9e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c111750 con 0x7f8a9c1011b0 2024-03-16T09:14:57.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.446+0000 7f8a91d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a9c101b70 0x7f8a9c199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:57.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.446+0000 7f8a91d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a9c101b70 0x7f8a9c199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45656/0 (socket says 172.21.15.112:45656) 2024-03-16T09:14:57.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.446+0000 7f8a91d9b700 1 -- 172.21.15.112:0/2870099929 learned_addr learned my addr 172.21.15.112:0/2870099929 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:14:57.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.446+0000 7f8a9a59c700 1 --2- 172.21.15.112:0/2870099929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a9c1011b0 0x7f8a9c1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:57.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.446+0000 7f8a9ad9d700 1 --2- 172.21.15.112:0/2870099929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a9c107260 0x7f8a9c1a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:57.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.446+0000 7f8a9a59c700 1 -- 172.21.15.112:0/2870099929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a9c107260 msgr2=0x7f8a9c1a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:57.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.446+0000 7f8a9a59c700 1 --2- 172.21.15.112:0/2870099929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a9c107260 0x7f8a9c1a4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:57.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.446+0000 7f8a9a59c700 1 -- 172.21.15.112:0/2870099929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a9c101b70 msgr2=0x7f8a9c199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:57.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.446+0000 7f8a9a59c700 1 --2- 172.21.15.112:0/2870099929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a9c101b70 0x7f8a9c199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:57.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.446+0000 7f8a9a59c700 1 -- 172.21.15.112:0/2870099929 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9c1a4eb0 con 0x7f8a9c1011b0 2024-03-16T09:14:57.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.446+0000 7f8a9a59c700 1 --2- 172.21.15.112:0/2870099929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a9c1011b0 0x7f8a9c1998f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00e9f0 tx=0x7f8a8c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:57.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.447+0000 7f8a93fff700 1 -- 172.21.15.112:0/2870099929 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c004220 con 0x7f8a9c1011b0 2024-03-16T09:14:57.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.447+0000 7f8aa0c9e700 1 -- 172.21.15.112:0/2870099929 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a9c1a50e0 con 0x7f8a9c1011b0 2024-03-16T09:14:57.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.447+0000 7f8a93fff700 1 -- 172.21.15.112:0/2870099929 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8a8c004d30 con 0x7f8a9c1011b0 2024-03-16T09:14:57.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.447+0000 7f8a93fff700 1 -- 172.21.15.112:0/2870099929 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c005080 con 0x7f8a9c1011b0 2024-03-16T09:14:57.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.447+0000 7f8aa0c9e700 1 -- 172.21.15.112:0/2870099929 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a9c1a55d0 con 0x7f8a9c1011b0 2024-03-16T09:14:57.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.448+0000 7f8aa0c9e700 1 -- 172.21.15.112:0/2870099929 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a9c068f90 con 0x7f8a9c1011b0 2024-03-16T09:14:57.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.449+0000 7f8a93fff700 1 -- 172.21.15.112:0/2870099929 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f8a8c004750 con 0x7f8a9c1011b0 2024-03-16T09:14:57.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.449+0000 7f8a93fff700 1 --2- 172.21.15.112:0/2870099929 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8a880a2920 0x7f8a880a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:14:57.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.449+0000 7f8a91d9b700 1 --2- 172.21.15.112:0/2870099929 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8a880a2920 0x7f8a880a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:14:57.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.449+0000 7f8a93fff700 1 -- 172.21.15.112:0/2870099929 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8a8c0ceb30 con 0x7f8a9c1011b0 2024-03-16T09:14:57.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.450+0000 7f8a91d9b700 1 --2- 172.21.15.112:0/2870099929 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8a880a2920 0x7f8a880a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c078890 tx=0x7f8a84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:14:57.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.455+0000 7f8a93fff700 1 -- 172.21.15.112:0/2870099929 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8a8c09ba20 con 0x7f8a9c1011b0 2024-03-16T09:14:57.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.649+0000 7f8aa0c9e700 1 -- 172.21.15.112:0/2870099929 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a9c062710 con 0x7f8a880a2920 2024-03-16T09:14:57.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.657+0000 7f8a93fff700 1 -- 172.21.15.112:0/2870099929 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258940 (secure 0 0 0) 0x7f8a9c062710 con 0x7f8a880a2920 2024-03-16T09:14:57.662 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:14:57.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.659+0000 7f8aa0c9e700 1 -- 172.21.15.112:0/2870099929 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8a880a2920 msgr2=0x7f8a880a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:57.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.659+0000 7f8aa0c9e700 1 --2- 172.21.15.112:0/2870099929 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8a880a2920 0x7f8a880a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c078890 tx=0x7f8a84009260 comp rx=0 tx=0).stop 2024-03-16T09:14:57.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.659+0000 7f8aa0c9e700 1 -- 172.21.15.112:0/2870099929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a9c1011b0 msgr2=0x7f8a9c1998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:14:57.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.659+0000 7f8aa0c9e700 1 --2- 172.21.15.112:0/2870099929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a9c1011b0 0x7f8a9c1998f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00e9f0 tx=0x7f8a8c00ed00 comp rx=0 tx=0).stop 2024-03-16T09:14:57.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.659+0000 7f8aa0c9e700 1 -- 172.21.15.112:0/2870099929 shutdown_connections 2024-03-16T09:14:57.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.659+0000 7f8aa0c9e700 1 --2- 172.21.15.112:0/2870099929 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8a880a2920 0x7f8a880a4de0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:57.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.659+0000 7f8aa0c9e700 1 --2- 172.21.15.112:0/2870099929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a9c107260 0x7f8a9c1a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:57.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.659+0000 7f8aa0c9e700 1 --2- 172.21.15.112:0/2870099929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a9c1011b0 0x7f8a9c1998f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:57.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.659+0000 7f8aa0c9e700 1 --2- 172.21.15.112:0/2870099929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a9c101b70 0x7f8a9c199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:14:57.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.659+0000 7f8aa0c9e700 1 -- 172.21.15.112:0/2870099929 >> 172.21.15.112:0/2870099929 conn(0x7f8a9c0781e0 msgr2=0x7f8a9c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:14:57.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.660+0000 7f8aa0c9e700 1 -- 172.21.15.112:0/2870099929 shutdown_connections 2024-03-16T09:14:57.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:14:57.660+0000 7f8aa0c9e700 1 -- 172.21.15.112:0/2870099929 wait complete. 2024-03-16T09:14:57.665 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-03-16T09:14:58.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:57 smithi159 ceph-mon[36739]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:14:58.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:57 smithi112 ceph-mon[32307]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:14:58.306 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-03-16T09:14:56.895483+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653677,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":596,"num_read_kb":412,"num_write":337,"num_write_kb":3046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181900,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352476,"kb_used_data":11556,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589348,"statfs":{"total":1151940427776,"available":1151579492352,"internally_reserved":0,"allocated":11833344,"data_stored":7428411,"data_compressed":112689,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003202"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996728+0000","last_change":"2024-03-16T09:13:37.119969+0000","last_active":"2024-03-16T09:14:31.996728+0000","last_peered":"2024-03-16T09:14:31.996728+0000","last_clean":"2024-03-16T09:14:31.996728+0000","last_became_active":"2024-03-16T09:13:37.119177+0000","last_became_peered":"2024-03-16T09:13:37.119177+0000","last_unstale":"2024-03-16T09:14:31.996728+0000","last_undegraded":"2024-03-16T09:14:31.996728+0000","last_fullsized":"2024-03-16T09:14:31.996728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:25:28.038898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994824+0000","last_change":"2024-03-16T09:13:37.127564+0000","last_active":"2024-03-16T09:14:31.994824+0000","last_peered":"2024-03-16T09:14:31.994824+0000","last_clean":"2024-03-16T09:14:31.994824+0000","last_became_active":"2024-03-16T09:13:37.127214+0000","last_became_peered":"2024-03-16T09:13:37.127214+0000","last_unstale":"2024-03-16T09:14:31.994824+0000","last_undegraded":"2024-03-16T09:14:31.994824+0000","last_fullsized":"2024-03-16T09:14:31.994824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:46:14.655707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994554+0000","last_change":"2024-03-16T09:13:37.126288+0000","last_active":"2024-03-16T09:14:31.994554+0000","last_peered":"2024-03-16T09:14:31.994554+0000","last_clean":"2024-03-16T09:14:31.994554+0000","last_became_active":"2024-03-16T09:13:37.125383+0000","last_became_peered":"2024-03-16T09:13:37.125383+0000","last_unstale":"2024-03-16T09:14:31.994554+0000","last_undegraded":"2024-03-16T09:14:31.994554+0000","last_fullsized":"2024-03-16T09:14:31.994554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:47:09.901508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997078+0000","last_change":"2024-03-16T09:13:37.125503+0000","last_active":"2024-03-16T09:14:31.997078+0000","last_peered":"2024-03-16T09:14:31.997078+0000","last_clean":"2024-03-16T09:14:31.997078+0000","last_became_active":"2024-03-16T09:13:37.125144+0000","last_became_peered":"2024-03-16T09:13:37.125144+0000","last_unstale":"2024-03-16T09:14:31.997078+0000","last_undegraded":"2024-03-16T09:14:31.997078+0000","last_fullsized":"2024-03-16T09:14:31.997078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:30:26.529080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994824+0000","last_change":"2024-03-16T09:13:37.114237+0000","last_active":"2024-03-16T09:14:31.994824+0000","last_peered":"2024-03-16T09:14:31.994824+0000","last_clean":"2024-03-16T09:14:31.994824+0000","last_became_active":"2024-03-16T09:13:37.113595+0000","last_became_peered":"2024-03-16T09:13:37.113595+0000","last_unstale":"2024-03-16T09:14:31.994824+0000","last_undegraded":"2024-03-16T09:14:31.994824+0000","last_fullsized":"2024-03-16T09:14:31.994824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:06:17.735301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.230671+0000","last_change":"2024-03-16T09:13:37.118843+0000","last_active":"2024-03-16T09:14:32.230671+0000","last_peered":"2024-03-16T09:14:32.230671+0000","last_clean":"2024-03-16T09:14:32.230671+0000","last_became_active":"2024-03-16T09:13:37.117086+0000","last_became_peered":"2024-03-16T09:13:37.117086+0000","last_unstale":"2024-03-16T09:14:32.230671+0000","last_undegraded":"2024-03-16T09:14:32.230671+0000","last_fullsized":"2024-03-16T09:14:32.230671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:23:12.929931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997583+0000","last_change":"2024-03-16T09:13:37.133427+0000","last_active":"2024-03-16T09:14:31.997583+0000","last_peered":"2024-03-16T09:14:31.997583+0000","last_clean":"2024-03-16T09:14:31.997583+0000","last_became_active":"2024-03-16T09:13:37.133032+0000","last_became_peered":"2024-03-16T09:13:37.133032+0000","last_unstale":"2024-03-16T09:14:31.997583+0000","last_undegraded":"2024-03-16T09:14:31.997583+0000","last_fullsized":"2024-03-16T09:14:31.997583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:49:28.179883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517199+0000","last_change":"2024-03-16T09:13:37.132132+0000","last_active":"2024-03-16T09:14:32.517199+0000","last_peered":"2024-03-16T09:14:32.517199+0000","last_clean":"2024-03-16T09:14:32.517199+0000","last_became_active":"2024-03-16T09:13:37.131801+0000","last_became_peered":"2024-03-16T09:13:37.131801+0000","last_unstale":"2024-03-16T09:14:32.517199+0000","last_undegraded":"2024-03-16T09:14:32.517199+0000","last_fullsized":"2024-03-16T09:14:32.517199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:45:03.891725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.001301+0000","last_change":"2024-03-16T09:13:37.116378+0000","last_active":"2024-03-16T09:14:32.001301+0000","last_peered":"2024-03-16T09:14:32.001301+0000","last_clean":"2024-03-16T09:14:32.001301+0000","last_became_active":"2024-03-16T09:13:37.116027+0000","last_became_peered":"2024-03-16T09:13:37.116027+0000","last_unstale":"2024-03-16T09:14:32.001301+0000","last_undegraded":"2024-03-16T09:14:32.001301+0000","last_fullsized":"2024-03-16T09:14:32.001301+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:14:38.681613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.230911+0000","last_change":"2024-03-16T09:13:37.121417+0000","last_active":"2024-03-16T09:14:32.230911+0000","last_peered":"2024-03-16T09:14:32.230911+0000","last_clean":"2024-03-16T09:14:32.230911+0000","last_became_active":"2024-03-16T09:13:37.121112+0000","last_became_peered":"2024-03-16T09:13:37.121112+0000","last_unstale":"2024-03-16T09:14:32.230911+0000","last_undegraded":"2024-03-16T09:14:32.230911+0000","last_fullsized":"2024-03-16T09:14:32.230911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:38:07.823604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997932+0000","last_change":"2024-03-16T09:13:37.121368+0000","last_active":"2024-03-16T09:14:31.997932+0000","last_peered":"2024-03-16T09:14:31.997932+0000","last_clean":"2024-03-16T09:14:31.997932+0000","last_became_active":"2024-03-16T09:13:37.121060+0000","last_became_peered":"2024-03-16T09:13:37.121060+0000","last_unstale":"2024-03-16T09:14:31.997932+0000","last_undegraded":"2024-03-16T09:14:31.997932+0000","last_fullsized":"2024-03-16T09:14:31.997932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:48:49.168285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997040+0000","last_change":"2024-03-16T09:13:37.127004+0000","last_active":"2024-03-16T09:14:31.997040+0000","last_peered":"2024-03-16T09:14:31.997040+0000","last_clean":"2024-03-16T09:14:31.997040+0000","last_became_active":"2024-03-16T09:13:37.126579+0000","last_became_peered":"2024-03-16T09:13:37.126579+0000","last_unstale":"2024-03-16T09:14:31.997040+0000","last_undegraded":"2024-03-16T09:14:31.997040+0000","last_fullsized":"2024-03-16T09:14:31.997040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:54:18.382514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385477+0000","last_change":"2024-03-16T09:13:37.104534+0000","last_active":"2024-03-16T09:14:32.385477+0000","last_peered":"2024-03-16T09:14:32.385477+0000","last_clean":"2024-03-16T09:14:32.385477+0000","last_became_active":"2024-03-16T09:13:37.104202+0000","last_became_peered":"2024-03-16T09:13:37.104202+0000","last_unstale":"2024-03-16T09:14:32.385477+0000","last_undegraded":"2024-03-16T09:14:32.385477+0000","last_fullsized":"2024-03-16T09:14:32.385477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:40:55.448238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994426+0000","last_change":"2024-03-16T09:13:37.117166+0000","last_active":"2024-03-16T09:14:31.994426+0000","last_peered":"2024-03-16T09:14:31.994426+0000","last_clean":"2024-03-16T09:14:31.994426+0000","last_became_active":"2024-03-16T09:13:37.116747+0000","last_became_peered":"2024-03-16T09:13:37.116747+0000","last_unstale":"2024-03-16T09:14:31.994426+0000","last_undegraded":"2024-03-16T09:14:31.994426+0000","last_fullsized":"2024-03-16T09:14:31.994426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:50:58.705683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.000383+0000","last_change":"2024-03-16T09:14:32.000383+0000","last_active":"2024-03-16T09:14:32.000383+0000","last_peered":"2024-03-16T09:14:32.000383+0000","last_clean":"2024-03-16T09:14:32.000383+0000","last_became_active":"2024-03-16T09:14:32.000027+0000","last_became_peered":"2024-03-16T09:14:32.000027+0000","last_unstale":"2024-03-16T09:14:32.000383+0000","last_undegraded":"2024-03-16T09:14:32.000383+0000","last_fullsized":"2024-03-16T09:14:32.000383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:04:03.478849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.000294+0000","last_change":"2024-03-16T09:13:37.120583+0000","last_active":"2024-03-16T09:14:32.000294+0000","last_peered":"2024-03-16T09:14:32.000294+0000","last_clean":"2024-03-16T09:14:32.000294+0000","last_became_active":"2024-03-16T09:13:37.120217+0000","last_became_peered":"2024-03-16T09:13:37.120217+0000","last_unstale":"2024-03-16T09:14:32.000294+0000","last_undegraded":"2024-03-16T09:14:32.000294+0000","last_fullsized":"2024-03-16T09:14:32.000294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:00:17.653736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996525+0000","last_change":"2024-03-16T09:13:37.121179+0000","last_active":"2024-03-16T09:14:31.996525+0000","last_peered":"2024-03-16T09:14:31.996525+0000","last_clean":"2024-03-16T09:14:31.996525+0000","last_became_active":"2024-03-16T09:13:37.120070+0000","last_became_peered":"2024-03-16T09:13:37.120070+0000","last_unstale":"2024-03-16T09:14:31.996525+0000","last_undegraded":"2024-03-16T09:14:31.996525+0000","last_fullsized":"2024-03-16T09:14:31.996525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:39:24.937068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996804+0000","last_change":"2024-03-16T09:13:37.110074+0000","last_active":"2024-03-16T09:14:31.996804+0000","last_peered":"2024-03-16T09:14:31.996804+0000","last_clean":"2024-03-16T09:14:31.996804+0000","last_became_active":"2024-03-16T09:13:37.109767+0000","last_became_peered":"2024-03-16T09:13:37.109767+0000","last_unstale":"2024-03-16T09:14:31.996804+0000","last_undegraded":"2024-03-16T09:14:31.996804+0000","last_fullsized":"2024-03-16T09:14:31.996804+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:20:37.408262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.999135+0000","last_change":"2024-03-16T09:13:37.130791+0000","last_active":"2024-03-16T09:14:31.999135+0000","last_peered":"2024-03-16T09:14:31.999135+0000","last_clean":"2024-03-16T09:14:31.999135+0000","last_became_active":"2024-03-16T09:13:37.130428+0000","last_became_peered":"2024-03-16T09:13:37.130428+0000","last_unstale":"2024-03-16T09:14:31.999135+0000","last_undegraded":"2024-03-16T09:14:31.999135+0000","last_fullsized":"2024-03-16T09:14:31.999135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:32:33.185589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.006081+0000","last_change":"2024-03-16T09:14:32.006081+0000","last_active":"2024-03-16T09:14:32.006081+0000","last_peered":"2024-03-16T09:14:32.006081+0000","last_clean":"2024-03-16T09:14:32.006081+0000","last_became_active":"2024-03-16T09:14:32.000165+0000","last_became_peered":"2024-03-16T09:14:32.000165+0000","last_unstale":"2024-03-16T09:14:32.006081+0000","last_undegraded":"2024-03-16T09:14:32.006081+0000","last_fullsized":"2024-03-16T09:14:32.006081+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:12:54.685988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517549+0000","last_change":"2024-03-16T09:13:37.124946+0000","last_active":"2024-03-16T09:14:32.517549+0000","last_peered":"2024-03-16T09:14:32.517549+0000","last_clean":"2024-03-16T09:14:32.517549+0000","last_became_active":"2024-03-16T09:13:37.124624+0000","last_became_peered":"2024-03-16T09:13:37.124624+0000","last_unstale":"2024-03-16T09:14:32.517549+0000","last_undegraded":"2024-03-16T09:14:32.517549+0000","last_fullsized":"2024-03-16T09:14:32.517549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:18:15.188823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517090+0000","last_change":"2024-03-16T09:13:36.116312+0000","last_active":"2024-03-16T09:14:32.517090+0000","last_peered":"2024-03-16T09:14:32.517090+0000","last_clean":"2024-03-16T09:14:32.517090+0000","last_became_active":"2024-03-16T09:13:36.115999+0000","last_became_peered":"2024-03-16T09:13:36.115999+0000","last_unstale":"2024-03-16T09:14:32.517090+0000","last_undegraded":"2024-03-16T09:14:32.517090+0000","last_fullsized":"2024-03-16T09:14:32.517090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:04:46.795291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994711+0000","last_change":"2024-03-16T09:13:37.126024+0000","last_active":"2024-03-16T09:14:31.994711+0000","last_peered":"2024-03-16T09:14:31.994711+0000","last_clean":"2024-03-16T09:14:31.994711+0000","last_became_active":"2024-03-16T09:13:37.125722+0000","last_became_peered":"2024-03-16T09:13:37.125722+0000","last_unstale":"2024-03-16T09:14:31.994711+0000","last_undegraded":"2024-03-16T09:14:31.994711+0000","last_fullsized":"2024-03-16T09:14:31.994711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:27:24.923022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.005879+0000","last_change":"2024-03-16T09:14:32.005295+0000","last_active":"2024-03-16T09:14:32.005879+0000","last_peered":"2024-03-16T09:14:32.005879+0000","last_clean":"2024-03-16T09:14:32.005879+0000","last_became_active":"2024-03-16T09:14:31.999981+0000","last_became_peered":"2024-03-16T09:14:31.999981+0000","last_unstale":"2024-03-16T09:14:32.005879+0000","last_undegraded":"2024-03-16T09:14:32.005879+0000","last_fullsized":"2024-03-16T09:14:32.005879+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:13:32.661689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.000732+0000","last_change":"2024-03-16T09:13:37.127593+0000","last_active":"2024-03-16T09:14:32.000732+0000","last_peered":"2024-03-16T09:14:32.000732+0000","last_clean":"2024-03-16T09:14:32.000732+0000","last_became_active":"2024-03-16T09:13:37.127255+0000","last_became_peered":"2024-03-16T09:13:37.127255+0000","last_unstale":"2024-03-16T09:14:32.000732+0000","last_undegraded":"2024-03-16T09:14:32.000732+0000","last_fullsized":"2024-03-16T09:14:32.000732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:07:51.263266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.995143+0000","last_change":"2024-03-16T09:13:36.117583+0000","last_active":"2024-03-16T09:14:31.995143+0000","last_peered":"2024-03-16T09:14:31.995143+0000","last_clean":"2024-03-16T09:14:31.995143+0000","last_became_active":"2024-03-16T09:13:36.117270+0000","last_became_peered":"2024-03-16T09:13:36.117270+0000","last_unstale":"2024-03-16T09:14:31.995143+0000","last_undegraded":"2024-03-16T09:14:31.995143+0000","last_fullsized":"2024-03-16T09:14:31.995143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:10:53.541872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248667+0000","last_change":"2024-03-16T09:13:37.119019+0000","last_active":"2024-03-16T09:14:32.248667+0000","last_peered":"2024-03-16T09:14:32.248667+0000","last_clean":"2024-03-16T09:14:32.248667+0000","last_became_active":"2024-03-16T09:13:37.118148+0000","last_became_peered":"2024-03-16T09:13:37.118148+0000","last_unstale":"2024-03-16T09:14:32.248667+0000","last_undegraded":"2024-03-16T09:14:32.248667+0000","last_fullsized":"2024-03-16T09:14:32.248667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:17:43.042779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:50.401230+0000","last_change":"2024-03-16T09:13:36.114415+0000","last_active":"2024-03-16T09:14:50.401230+0000","last_peered":"2024-03-16T09:14:50.401230+0000","last_clean":"2024-03-16T09:14:50.401230+0000","last_became_active":"2024-03-16T09:13:36.114056+0000","last_became_peered":"2024-03-16T09:13:36.114056+0000","last_unstale":"2024-03-16T09:14:50.401230+0000","last_undegraded":"2024-03-16T09:14:50.401230+0000","last_fullsized":"2024-03-16T09:14:50.401230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:50:28.322627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994972+0000","last_change":"2024-03-16T09:13:37.124997+0000","last_active":"2024-03-16T09:14:31.994972+0000","last_peered":"2024-03-16T09:14:31.994972+0000","last_clean":"2024-03-16T09:14:31.994972+0000","last_became_active":"2024-03-16T09:13:37.124664+0000","last_became_peered":"2024-03-16T09:13:37.124664+0000","last_unstale":"2024-03-16T09:14:31.994972+0000","last_undegraded":"2024-03-16T09:14:31.994972+0000","last_fullsized":"2024-03-16T09:14:31.994972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:24:30.104576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.230143+0000","last_change":"2024-03-16T09:13:36.121980+0000","last_active":"2024-03-16T09:14:32.230143+0000","last_peered":"2024-03-16T09:14:32.230143+0000","last_clean":"2024-03-16T09:14:32.230143+0000","last_became_active":"2024-03-16T09:13:36.121703+0000","last_became_peered":"2024-03-16T09:13:36.121703+0000","last_unstale":"2024-03-16T09:14:32.230143+0000","last_undegraded":"2024-03-16T09:14:32.230143+0000","last_fullsized":"2024-03-16T09:14:32.230143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:40:26.601688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248438+0000","last_change":"2024-03-16T09:13:37.123443+0000","last_active":"2024-03-16T09:14:32.248438+0000","last_peered":"2024-03-16T09:14:32.248438+0000","last_clean":"2024-03-16T09:14:32.248438+0000","last_became_active":"2024-03-16T09:13:37.121987+0000","last_became_peered":"2024-03-16T09:13:37.121987+0000","last_unstale":"2024-03-16T09:14:32.248438+0000","last_undegraded":"2024-03-16T09:14:32.248438+0000","last_fullsized":"2024-03-16T09:14:32.248438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:37:08.478552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385000+0000","last_change":"2024-03-16T09:13:36.106391+0000","last_active":"2024-03-16T09:14:32.385000+0000","last_peered":"2024-03-16T09:14:32.385000+0000","last_clean":"2024-03-16T09:14:32.385000+0000","last_became_active":"2024-03-16T09:13:36.106046+0000","last_became_peered":"2024-03-16T09:13:36.106046+0000","last_unstale":"2024-03-16T09:14:32.385000+0000","last_undegraded":"2024-03-16T09:14:32.385000+0000","last_fullsized":"2024-03-16T09:14:32.385000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:03:23.255519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517168+0000","last_change":"2024-03-16T09:13:37.114337+0000","last_active":"2024-03-16T09:14:32.517168+0000","last_peered":"2024-03-16T09:14:32.517168+0000","last_clean":"2024-03-16T09:14:32.517168+0000","last_became_active":"2024-03-16T09:13:37.113991+0000","last_became_peered":"2024-03-16T09:13:37.113991+0000","last_unstale":"2024-03-16T09:14:32.517168+0000","last_undegraded":"2024-03-16T09:14:32.517168+0000","last_fullsized":"2024-03-16T09:14:32.517168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:23:54.141479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248113+0000","last_change":"2024-03-16T09:13:36.113297+0000","last_active":"2024-03-16T09:14:32.248113+0000","last_peered":"2024-03-16T09:14:32.248113+0000","last_clean":"2024-03-16T09:14:32.248113+0000","last_became_active":"2024-03-16T09:13:36.112950+0000","last_became_peered":"2024-03-16T09:13:36.112950+0000","last_unstale":"2024-03-16T09:14:32.248113+0000","last_undegraded":"2024-03-16T09:14:32.248113+0000","last_fullsized":"2024-03-16T09:14:32.248113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:40:17.682450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996565+0000","last_change":"2024-03-16T09:13:37.112239+0000","last_active":"2024-03-16T09:14:31.996565+0000","last_peered":"2024-03-16T09:14:31.996565+0000","last_clean":"2024-03-16T09:14:31.996565+0000","last_became_active":"2024-03-16T09:13:37.111885+0000","last_became_peered":"2024-03-16T09:13:37.111885+0000","last_unstale":"2024-03-16T09:14:31.996565+0000","last_undegraded":"2024-03-16T09:14:31.996565+0000","last_fullsized":"2024-03-16T09:14:31.996565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:49.565846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:54.405264+0000","last_change":"2024-03-16T09:13:36.111341+0000","last_active":"2024-03-16T09:14:54.405264+0000","last_peered":"2024-03-16T09:14:54.405264+0000","last_clean":"2024-03-16T09:14:54.405264+0000","last_became_active":"2024-03-16T09:13:36.110558+0000","last_became_peered":"2024-03-16T09:13:36.110558+0000","last_unstale":"2024-03-16T09:14:54.405264+0000","last_undegraded":"2024-03-16T09:14:54.405264+0000","last_fullsized":"2024-03-16T09:14:54.405264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:56:44.252002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.000951+0000","last_change":"2024-03-16T09:13:37.126808+0000","last_active":"2024-03-16T09:14:32.000951+0000","last_peered":"2024-03-16T09:14:32.000951+0000","last_clean":"2024-03-16T09:14:32.000951+0000","last_became_active":"2024-03-16T09:13:37.126442+0000","last_became_peered":"2024-03-16T09:13:37.126442+0000","last_unstale":"2024-03-16T09:14:32.000951+0000","last_undegraded":"2024-03-16T09:14:32.000951+0000","last_fullsized":"2024-03-16T09:14:32.000951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:04:39.216835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.004330+0000","last_change":"2024-03-16T09:14:32.004330+0000","last_active":"2024-03-16T09:14:32.004330+0000","last_peered":"2024-03-16T09:14:32.004330+0000","last_clean":"2024-03-16T09:14:32.004330+0000","last_became_active":"2024-03-16T09:14:32.001741+0000","last_became_peered":"2024-03-16T09:14:32.001741+0000","last_unstale":"2024-03-16T09:14:32.004330+0000","last_undegraded":"2024-03-16T09:14:32.004330+0000","last_fullsized":"2024-03-16T09:14:32.004330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:08:19.002873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994663+0000","last_change":"2024-03-16T09:13:37.125833+0000","last_active":"2024-03-16T09:14:31.994663+0000","last_peered":"2024-03-16T09:14:31.994663+0000","last_clean":"2024-03-16T09:14:31.994663+0000","last_became_active":"2024-03-16T09:13:37.125508+0000","last_became_peered":"2024-03-16T09:13:37.125508+0000","last_unstale":"2024-03-16T09:14:31.994663+0000","last_undegraded":"2024-03-16T09:14:31.994663+0000","last_fullsized":"2024-03-16T09:14:31.994663+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:14:47.749715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248156+0000","last_change":"2024-03-16T09:13:36.112200+0000","last_active":"2024-03-16T09:14:32.248156+0000","last_peered":"2024-03-16T09:14:32.248156+0000","last_clean":"2024-03-16T09:14:32.248156+0000","last_became_active":"2024-03-16T09:13:36.111830+0000","last_became_peered":"2024-03-16T09:13:36.111830+0000","last_unstale":"2024-03-16T09:14:32.248156+0000","last_undegraded":"2024-03-16T09:14:32.248156+0000","last_fullsized":"2024-03-16T09:14:32.248156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:07:57.147960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.000844+0000","last_change":"2024-03-16T09:13:37.118434+0000","last_active":"2024-03-16T09:14:32.000844+0000","last_peered":"2024-03-16T09:14:32.000844+0000","last_clean":"2024-03-16T09:14:32.000844+0000","last_became_active":"2024-03-16T09:13:37.118069+0000","last_became_peered":"2024-03-16T09:13:37.118069+0000","last_unstale":"2024-03-16T09:14:32.000844+0000","last_undegraded":"2024-03-16T09:14:32.000844+0000","last_fullsized":"2024-03-16T09:14:32.000844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:02:56.559384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'27","reported_seq":205,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:55.406269+0000","last_change":"2024-03-16T09:13:36.114591+0000","last_active":"2024-03-16T09:14:55.406269+0000","last_peered":"2024-03-16T09:14:55.406269+0000","last_clean":"2024-03-16T09:14:55.406269+0000","last_became_active":"2024-03-16T09:13:36.114252+0000","last_became_peered":"2024-03-16T09:13:36.114252+0000","last_unstale":"2024-03-16T09:14:55.406269+0000","last_undegraded":"2024-03-16T09:14:55.406269+0000","last_fullsized":"2024-03-16T09:14:55.406269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:37:52.637394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59465,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":159,"num_read_kb":137,"num_write":28,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994943+0000","last_change":"2024-03-16T09:13:37.133102+0000","last_active":"2024-03-16T09:14:31.994943+0000","last_peered":"2024-03-16T09:14:31.994943+0000","last_clean":"2024-03-16T09:14:31.994943+0000","last_became_active":"2024-03-16T09:13:37.132728+0000","last_became_peered":"2024-03-16T09:13:37.132728+0000","last_unstale":"2024-03-16T09:14:31.994943+0000","last_undegraded":"2024-03-16T09:14:31.994943+0000","last_fullsized":"2024-03-16T09:14:31.994943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:54:27.511997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998770+0000","last_change":"2024-03-16T09:13:36.114195+0000","last_active":"2024-03-16T09:14:31.998770+0000","last_peered":"2024-03-16T09:14:31.998770+0000","last_clean":"2024-03-16T09:14:31.998770+0000","last_became_active":"2024-03-16T09:13:36.113829+0000","last_became_peered":"2024-03-16T09:13:36.113829+0000","last_unstale":"2024-03-16T09:14:31.998770+0000","last_undegraded":"2024-03-16T09:14:31.998770+0000","last_fullsized":"2024-03-16T09:14:31.998770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:28:40.509877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385216+0000","last_change":"2024-03-16T09:13:37.132974+0000","last_active":"2024-03-16T09:14:32.385216+0000","last_peered":"2024-03-16T09:14:32.385216+0000","last_clean":"2024-03-16T09:14:32.385216+0000","last_became_active":"2024-03-16T09:13:37.132474+0000","last_became_peered":"2024-03-16T09:13:37.132474+0000","last_unstale":"2024-03-16T09:14:32.385216+0000","last_undegraded":"2024-03-16T09:14:32.385216+0000","last_fullsized":"2024-03-16T09:14:32.385216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:58:04.480441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.247992+0000","last_change":"2024-03-16T09:13:36.115242+0000","last_active":"2024-03-16T09:14:32.247992+0000","last_peered":"2024-03-16T09:14:32.247992+0000","last_clean":"2024-03-16T09:14:32.247992+0000","last_became_active":"2024-03-16T09:13:36.113566+0000","last_became_peered":"2024-03-16T09:13:36.113566+0000","last_unstale":"2024-03-16T09:14:32.247992+0000","last_undegraded":"2024-03-16T09:14:32.247992+0000","last_fullsized":"2024-03-16T09:14:32.247992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:47:50.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.386530+0000","last_change":"2024-03-16T09:14:32.386530+0000","last_active":"2024-03-16T09:14:32.386530+0000","last_peered":"2024-03-16T09:14:32.386530+0000","last_clean":"2024-03-16T09:14:32.386530+0000","last_became_active":"2024-03-16T09:14:32.386088+0000","last_became_peered":"2024-03-16T09:14:32.386088+0000","last_unstale":"2024-03-16T09:14:32.386530+0000","last_undegraded":"2024-03-16T09:14:32.386530+0000","last_fullsized":"2024-03-16T09:14:32.386530+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:44:57.427154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":97,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:51.407466+0000","last_change":"2024-03-16T09:13:36.109585+0000","last_active":"2024-03-16T09:14:51.407466+0000","last_peered":"2024-03-16T09:14:51.407466+0000","last_clean":"2024-03-16T09:14:51.407466+0000","last_became_active":"2024-03-16T09:13:36.109267+0000","last_became_peered":"2024-03-16T09:13:36.109267+0000","last_unstale":"2024-03-16T09:14:51.407466+0000","last_undegraded":"2024-03-16T09:14:51.407466+0000","last_fullsized":"2024-03-16T09:14:51.407466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:26:55.700791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997586+0000","last_change":"2024-03-16T09:13:37.136465+0000","last_active":"2024-03-16T09:14:31.997586+0000","last_peered":"2024-03-16T09:14:31.997586+0000","last_clean":"2024-03-16T09:14:31.997586+0000","last_became_active":"2024-03-16T09:13:37.135933+0000","last_became_peered":"2024-03-16T09:13:37.135933+0000","last_unstale":"2024-03-16T09:14:31.997586+0000","last_undegraded":"2024-03-16T09:14:31.997586+0000","last_fullsized":"2024-03-16T09:14:31.997586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:08:28.490533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994732+0000","last_change":"2024-03-16T09:13:36.119428+0000","last_active":"2024-03-16T09:14:31.994732+0000","last_peered":"2024-03-16T09:14:31.994732+0000","last_clean":"2024-03-16T09:14:31.994732+0000","last_became_active":"2024-03-16T09:13:36.119083+0000","last_became_peered":"2024-03-16T09:13:36.119083+0000","last_unstale":"2024-03-16T09:14:31.994732+0000","last_undegraded":"2024-03-16T09:14:31.994732+0000","last_fullsized":"2024-03-16T09:14:31.994732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:58:56.065151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248311+0000","last_change":"2024-03-16T09:13:37.118082+0000","last_active":"2024-03-16T09:14:32.248311+0000","last_peered":"2024-03-16T09:14:32.248311+0000","last_clean":"2024-03-16T09:14:32.248311+0000","last_became_active":"2024-03-16T09:13:37.117671+0000","last_became_peered":"2024-03-16T09:13:37.117671+0000","last_unstale":"2024-03-16T09:14:32.248311+0000","last_undegraded":"2024-03-16T09:14:32.248311+0000","last_fullsized":"2024-03-16T09:14:32.248311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:14:05.352191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.995286+0000","last_change":"2024-03-16T09:13:36.121077+0000","last_active":"2024-03-16T09:14:31.995286+0000","last_peered":"2024-03-16T09:14:31.995286+0000","last_clean":"2024-03-16T09:14:31.995286+0000","last_became_active":"2024-03-16T09:13:36.120723+0000","last_became_peered":"2024-03-16T09:13:36.120723+0000","last_unstale":"2024-03-16T09:14:31.995286+0000","last_undegraded":"2024-03-16T09:14:31.995286+0000","last_fullsized":"2024-03-16T09:14:31.995286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:58:14.746215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996410+0000","last_change":"2024-03-16T09:13:37.116725+0000","last_active":"2024-03-16T09:14:31.996410+0000","last_peered":"2024-03-16T09:14:31.996410+0000","last_clean":"2024-03-16T09:14:31.996410+0000","last_became_active":"2024-03-16T09:13:37.116406+0000","last_became_peered":"2024-03-16T09:13:37.116406+0000","last_unstale":"2024-03-16T09:14:31.996410+0000","last_undegraded":"2024-03-16T09:14:31.996410+0000","last_fullsized":"2024-03-16T09:14:31.996410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:56:59.136649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996706+0000","last_change":"2024-03-16T09:13:36.110568+0000","last_active":"2024-03-16T09:14:31.996706+0000","last_peered":"2024-03-16T09:14:31.996706+0000","last_clean":"2024-03-16T09:14:31.996706+0000","last_became_active":"2024-03-16T09:13:36.110246+0000","last_became_peered":"2024-03-16T09:13:36.110246+0000","last_unstale":"2024-03-16T09:14:31.996706+0000","last_undegraded":"2024-03-16T09:14:31.996706+0000","last_fullsized":"2024-03-16T09:14:31.996706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:36:50.011941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.230834+0000","last_change":"2024-03-16T09:13:37.122296+0000","last_active":"2024-03-16T09:14:32.230834+0000","last_peered":"2024-03-16T09:14:32.230834+0000","last_clean":"2024-03-16T09:14:32.230834+0000","last_became_active":"2024-03-16T09:13:37.121968+0000","last_became_peered":"2024-03-16T09:13:37.121968+0000","last_unstale":"2024-03-16T09:14:32.230834+0000","last_undegraded":"2024-03-16T09:14:32.230834+0000","last_fullsized":"2024-03-16T09:14:32.230834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:10:43.449397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997895+0000","last_change":"2024-03-16T09:13:36.112262+0000","last_active":"2024-03-16T09:14:31.997895+0000","last_peered":"2024-03-16T09:14:31.997895+0000","last_clean":"2024-03-16T09:14:31.997895+0000","last_became_active":"2024-03-16T09:13:36.111916+0000","last_became_peered":"2024-03-16T09:13:36.111916+0000","last_unstale":"2024-03-16T09:14:31.997895+0000","last_undegraded":"2024-03-16T09:14:31.997895+0000","last_fullsized":"2024-03-16T09:14:31.997895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:27:41.163508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.230779+0000","last_change":"2024-03-16T09:13:37.119814+0000","last_active":"2024-03-16T09:14:32.230779+0000","last_peered":"2024-03-16T09:14:32.230779+0000","last_clean":"2024-03-16T09:14:32.230779+0000","last_became_active":"2024-03-16T09:13:37.119454+0000","last_became_peered":"2024-03-16T09:13:37.119454+0000","last_unstale":"2024-03-16T09:14:32.230779+0000","last_undegraded":"2024-03-16T09:14:32.230779+0000","last_fullsized":"2024-03-16T09:14:32.230779+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:57:25.988044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.000336+0000","last_change":"2024-03-16T09:14:31.999902+0000","last_active":"2024-03-16T09:14:32.000336+0000","last_peered":"2024-03-16T09:14:32.000336+0000","last_clean":"2024-03-16T09:14:32.000336+0000","last_became_active":"2024-03-16T09:14:31.999609+0000","last_became_peered":"2024-03-16T09:14:31.999609+0000","last_unstale":"2024-03-16T09:14:32.000336+0000","last_undegraded":"2024-03-16T09:14:32.000336+0000","last_fullsized":"2024-03-16T09:14:32.000336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:58:47.495746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":96,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:51.406986+0000","last_change":"2024-03-16T09:13:36.111300+0000","last_active":"2024-03-16T09:14:51.406986+0000","last_peered":"2024-03-16T09:14:51.406986+0000","last_clean":"2024-03-16T09:14:51.406986+0000","last_became_active":"2024-03-16T09:13:36.110966+0000","last_became_peered":"2024-03-16T09:13:36.110966+0000","last_unstale":"2024-03-16T09:14:51.406986+0000","last_undegraded":"2024-03-16T09:14:51.406986+0000","last_fullsized":"2024-03-16T09:14:51.406986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:18:33.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517362+0000","last_change":"2024-03-16T09:13:37.125791+0000","last_active":"2024-03-16T09:14:32.517362+0000","last_peered":"2024-03-16T09:14:32.517362+0000","last_clean":"2024-03-16T09:14:32.517362+0000","last_became_active":"2024-03-16T09:13:37.125448+0000","last_became_peered":"2024-03-16T09:13:37.125448+0000","last_unstale":"2024-03-16T09:14:32.517362+0000","last_undegraded":"2024-03-16T09:14:32.517362+0000","last_fullsized":"2024-03-16T09:14:32.517362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:51:30.767750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996257+0000","last_change":"2024-03-16T09:13:36.107462+0000","last_active":"2024-03-16T09:14:31.996257+0000","last_peered":"2024-03-16T09:14:31.996257+0000","last_clean":"2024-03-16T09:14:31.996257+0000","last_became_active":"2024-03-16T09:13:36.107115+0000","last_became_peered":"2024-03-16T09:13:36.107115+0000","last_unstale":"2024-03-16T09:14:31.996257+0000","last_undegraded":"2024-03-16T09:14:31.996257+0000","last_fullsized":"2024-03-16T09:14:31.996257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:37.796719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":341,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997675+0000","last_change":"2024-03-16T09:12:24.331159+0000","last_active":"2024-03-16T09:14:31.997675+0000","last_peered":"2024-03-16T09:14:31.997675+0000","last_clean":"2024-03-16T09:14:31.997675+0000","last_became_active":"2024-03-16T09:12:24.330554+0000","last_became_peered":"2024-03-16T09:12:24.330554+0000","last_unstale":"2024-03-16T09:14:31.997675+0000","last_undegraded":"2024-03-16T09:14:31.997675+0000","last_fullsized":"2024-03-16T09:14:31.997675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:08:57.697012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:08:57.697012+0000","last_clean_scrub_stamp":"2024-03-16T09:08:57.697012+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:43:02.986865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385248+0000","last_change":"2024-03-16T09:13:37.114299+0000","last_active":"2024-03-16T09:14:32.385248+0000","last_peered":"2024-03-16T09:14:32.385248+0000","last_clean":"2024-03-16T09:14:32.385248+0000","last_became_active":"2024-03-16T09:13:37.113981+0000","last_became_peered":"2024-03-16T09:13:37.113981+0000","last_unstale":"2024-03-16T09:14:32.385248+0000","last_undegraded":"2024-03-16T09:14:32.385248+0000","last_fullsized":"2024-03-16T09:14:32.385248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:17:15.882130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997873+0000","last_change":"2024-03-16T09:13:36.113962+0000","last_active":"2024-03-16T09:14:31.997873+0000","last_peered":"2024-03-16T09:14:31.997873+0000","last_clean":"2024-03-16T09:14:31.997873+0000","last_became_active":"2024-03-16T09:13:36.113616+0000","last_became_peered":"2024-03-16T09:13:36.113616+0000","last_unstale":"2024-03-16T09:14:31.997873+0000","last_undegraded":"2024-03-16T09:14:31.997873+0000","last_fullsized":"2024-03-16T09:14:31.997873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:15:15.004360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.995031+0000","last_change":"2024-03-16T09:13:36.121946+0000","last_active":"2024-03-16T09:14:31.995031+0000","last_peered":"2024-03-16T09:14:31.995031+0000","last_clean":"2024-03-16T09:14:31.995031+0000","last_became_active":"2024-03-16T09:13:36.121209+0000","last_became_peered":"2024-03-16T09:13:36.121209+0000","last_unstale":"2024-03-16T09:14:31.995031+0000","last_undegraded":"2024-03-16T09:14:31.995031+0000","last_fullsized":"2024-03-16T09:14:31.995031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:09:51.139468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385432+0000","last_change":"2024-03-16T09:13:37.128931+0000","last_active":"2024-03-16T09:14:32.385432+0000","last_peered":"2024-03-16T09:14:32.385432+0000","last_clean":"2024-03-16T09:14:32.385432+0000","last_became_active":"2024-03-16T09:13:37.128515+0000","last_became_peered":"2024-03-16T09:13:37.128515+0000","last_unstale":"2024-03-16T09:14:32.385432+0000","last_undegraded":"2024-03-16T09:14:32.385432+0000","last_fullsized":"2024-03-16T09:14:32.385432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:58:46.429144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996860+0000","last_change":"2024-03-16T09:13:36.118831+0000","last_active":"2024-03-16T09:14:31.996860+0000","last_peered":"2024-03-16T09:14:31.996860+0000","last_clean":"2024-03-16T09:14:31.996860+0000","last_became_active":"2024-03-16T09:13:36.118489+0000","last_became_peered":"2024-03-16T09:13:36.118489+0000","last_unstale":"2024-03-16T09:14:31.996860+0000","last_undegraded":"2024-03-16T09:14:31.996860+0000","last_fullsized":"2024-03-16T09:14:31.996860+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:55:34.089451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517648+0000","last_change":"2024-03-16T09:13:37.120581+0000","last_active":"2024-03-16T09:14:32.517648+0000","last_peered":"2024-03-16T09:14:32.517648+0000","last_clean":"2024-03-16T09:14:32.517648+0000","last_became_active":"2024-03-16T09:13:37.120266+0000","last_became_peered":"2024-03-16T09:13:37.120266+0000","last_unstale":"2024-03-16T09:14:32.517648+0000","last_undegraded":"2024-03-16T09:14:32.517648+0000","last_fullsized":"2024-03-16T09:14:32.517648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:16:10.784229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996414+0000","last_change":"2024-03-16T09:13:36.107125+0000","last_active":"2024-03-16T09:14:31.996414+0000","last_peered":"2024-03-16T09:14:31.996414+0000","last_clean":"2024-03-16T09:14:31.996414+0000","last_became_active":"2024-03-16T09:13:36.106795+0000","last_became_peered":"2024-03-16T09:13:36.106795+0000","last_unstale":"2024-03-16T09:14:31.996414+0000","last_undegraded":"2024-03-16T09:14:31.996414+0000","last_fullsized":"2024-03-16T09:14:31.996414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:29:10.130565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994815+0000","last_change":"2024-03-16T09:13:37.123013+0000","last_active":"2024-03-16T09:14:31.994815+0000","last_peered":"2024-03-16T09:14:31.994815+0000","last_clean":"2024-03-16T09:14:31.994815+0000","last_became_active":"2024-03-16T09:13:37.122642+0000","last_became_peered":"2024-03-16T09:13:37.122642+0000","last_unstale":"2024-03-16T09:14:31.994815+0000","last_undegraded":"2024-03-16T09:14:31.994815+0000","last_fullsized":"2024-03-16T09:14:31.994815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:35:05.297337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.000335+0000","last_change":"2024-03-16T09:13:36.110868+0000","last_active":"2024-03-16T09:14:32.000335+0000","last_peered":"2024-03-16T09:14:32.000335+0000","last_clean":"2024-03-16T09:14:32.000335+0000","last_became_active":"2024-03-16T09:13:36.110536+0000","last_became_peered":"2024-03-16T09:13:36.110536+0000","last_unstale":"2024-03-16T09:14:32.000335+0000","last_undegraded":"2024-03-16T09:14:32.000335+0000","last_fullsized":"2024-03-16T09:14:32.000335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:07:06.829458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994742+0000","last_change":"2024-03-16T09:13:37.116579+0000","last_active":"2024-03-16T09:14:31.994742+0000","last_peered":"2024-03-16T09:14:31.994742+0000","last_clean":"2024-03-16T09:14:31.994742+0000","last_became_active":"2024-03-16T09:13:37.116226+0000","last_became_peered":"2024-03-16T09:13:37.116226+0000","last_unstale":"2024-03-16T09:14:31.994742+0000","last_undegraded":"2024-03-16T09:14:31.994742+0000","last_fullsized":"2024-03-16T09:14:31.994742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:08:39.015162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517053+0000","last_change":"2024-03-16T09:13:36.107881+0000","last_active":"2024-03-16T09:14:32.517053+0000","last_peered":"2024-03-16T09:14:32.517053+0000","last_clean":"2024-03-16T09:14:32.517053+0000","last_became_active":"2024-03-16T09:13:36.107538+0000","last_became_peered":"2024-03-16T09:13:36.107538+0000","last_unstale":"2024-03-16T09:14:32.517053+0000","last_undegraded":"2024-03-16T09:14:32.517053+0000","last_fullsized":"2024-03-16T09:14:32.517053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:49:41.932089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996243+0000","last_change":"2024-03-16T09:13:37.118580+0000","last_active":"2024-03-16T09:14:31.996243+0000","last_peered":"2024-03-16T09:14:31.996243+0000","last_clean":"2024-03-16T09:14:31.996243+0000","last_became_active":"2024-03-16T09:13:37.118166+0000","last_became_peered":"2024-03-16T09:13:37.118166+0000","last_unstale":"2024-03-16T09:14:31.996243+0000","last_undegraded":"2024-03-16T09:14:31.996243+0000","last_fullsized":"2024-03-16T09:14:31.996243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:10:13.957439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998331+0000","last_change":"2024-03-16T09:13:36.122307+0000","last_active":"2024-03-16T09:14:31.998331+0000","last_peered":"2024-03-16T09:14:31.998331+0000","last_clean":"2024-03-16T09:14:31.998331+0000","last_became_active":"2024-03-16T09:13:36.121961+0000","last_became_peered":"2024-03-16T09:13:36.121961+0000","last_unstale":"2024-03-16T09:14:31.998331+0000","last_undegraded":"2024-03-16T09:14:31.998331+0000","last_fullsized":"2024-03-16T09:14:31.998331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:50:03.482300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517496+0000","last_change":"2024-03-16T09:13:37.131918+0000","last_active":"2024-03-16T09:14:32.517496+0000","last_peered":"2024-03-16T09:14:32.517496+0000","last_clean":"2024-03-16T09:14:32.517496+0000","last_became_active":"2024-03-16T09:13:37.131560+0000","last_became_peered":"2024-03-16T09:13:37.131560+0000","last_unstale":"2024-03-16T09:14:32.517496+0000","last_undegraded":"2024-03-16T09:14:32.517496+0000","last_fullsized":"2024-03-16T09:14:32.517496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:57.315125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994873+0000","last_change":"2024-03-16T09:13:36.120192+0000","last_active":"2024-03-16T09:14:31.994873+0000","last_peered":"2024-03-16T09:14:31.994873+0000","last_clean":"2024-03-16T09:14:31.994873+0000","last_became_active":"2024-03-16T09:13:36.119063+0000","last_became_peered":"2024-03-16T09:13:36.119063+0000","last_unstale":"2024-03-16T09:14:31.994873+0000","last_undegraded":"2024-03-16T09:14:31.994873+0000","last_fullsized":"2024-03-16T09:14:31.994873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:18:56.616624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998065+0000","last_change":"2024-03-16T09:13:37.129597+0000","last_active":"2024-03-16T09:14:31.998065+0000","last_peered":"2024-03-16T09:14:31.998065+0000","last_clean":"2024-03-16T09:14:31.998065+0000","last_became_active":"2024-03-16T09:13:37.129248+0000","last_became_peered":"2024-03-16T09:13:37.129248+0000","last_unstale":"2024-03-16T09:14:31.998065+0000","last_undegraded":"2024-03-16T09:14:31.998065+0000","last_fullsized":"2024-03-16T09:14:31.998065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:41:11.118192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.010274+0000","last_change":"2024-03-16T09:14:32.009577+0000","last_active":"2024-03-16T09:14:32.010274+0000","last_peered":"2024-03-16T09:14:32.010274+0000","last_clean":"2024-03-16T09:14:32.010274+0000","last_became_active":"2024-03-16T09:14:32.000521+0000","last_became_peered":"2024-03-16T09:14:32.000521+0000","last_unstale":"2024-03-16T09:14:32.010274+0000","last_undegraded":"2024-03-16T09:14:32.010274+0000","last_fullsized":"2024-03-16T09:14:32.010274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:29:07.221252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998028+0000","last_change":"2024-03-16T09:13:37.120317+0000","last_active":"2024-03-16T09:14:31.998028+0000","last_peered":"2024-03-16T09:14:31.998028+0000","last_clean":"2024-03-16T09:14:31.998028+0000","last_became_active":"2024-03-16T09:13:37.119974+0000","last_became_peered":"2024-03-16T09:13:37.119974+0000","last_unstale":"2024-03-16T09:14:31.998028+0000","last_undegraded":"2024-03-16T09:14:31.998028+0000","last_fullsized":"2024-03-16T09:14:31.998028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:25:34.774377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248531+0000","last_change":"2024-03-16T09:13:36.112395+0000","last_active":"2024-03-16T09:14:32.248531+0000","last_peered":"2024-03-16T09:14:32.248531+0000","last_clean":"2024-03-16T09:14:32.248531+0000","last_became_active":"2024-03-16T09:13:36.112036+0000","last_became_peered":"2024-03-16T09:13:36.112036+0000","last_unstale":"2024-03-16T09:14:32.248531+0000","last_undegraded":"2024-03-16T09:14:32.248531+0000","last_fullsized":"2024-03-16T09:14:32.248531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:41:02.895820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998157+0000","last_change":"2024-03-16T09:13:37.133845+0000","last_active":"2024-03-16T09:14:31.998157+0000","last_peered":"2024-03-16T09:14:31.998157+0000","last_clean":"2024-03-16T09:14:31.998157+0000","last_became_active":"2024-03-16T09:13:37.132833+0000","last_became_peered":"2024-03-16T09:13:37.132833+0000","last_unstale":"2024-03-16T09:14:31.998157+0000","last_undegraded":"2024-03-16T09:14:31.998157+0000","last_fullsized":"2024-03-16T09:14:31.998157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:12:18.049210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385012+0000","last_change":"2024-03-16T09:13:36.114830+0000","last_active":"2024-03-16T09:14:32.385012+0000","last_peered":"2024-03-16T09:14:32.385012+0000","last_clean":"2024-03-16T09:14:32.385012+0000","last_became_active":"2024-03-16T09:13:36.113898+0000","last_became_peered":"2024-03-16T09:13:36.113898+0000","last_unstale":"2024-03-16T09:14:32.385012+0000","last_undegraded":"2024-03-16T09:14:32.385012+0000","last_fullsized":"2024-03-16T09:14:32.385012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:27:38.910819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998466+0000","last_change":"2024-03-16T09:13:37.115991+0000","last_active":"2024-03-16T09:14:31.998466+0000","last_peered":"2024-03-16T09:14:31.998466+0000","last_clean":"2024-03-16T09:14:31.998466+0000","last_became_active":"2024-03-16T09:13:37.115671+0000","last_became_peered":"2024-03-16T09:13:37.115671+0000","last_unstale":"2024-03-16T09:14:31.998466+0000","last_undegraded":"2024-03-16T09:14:31.998466+0000","last_fullsized":"2024-03-16T09:14:31.998466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:05:59.703675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996265+0000","last_change":"2024-03-16T09:13:37.115122+0000","last_active":"2024-03-16T09:14:31.996265+0000","last_peered":"2024-03-16T09:14:31.996265+0000","last_clean":"2024-03-16T09:14:31.996265+0000","last_became_active":"2024-03-16T09:13:37.114779+0000","last_became_peered":"2024-03-16T09:13:37.114779+0000","last_unstale":"2024-03-16T09:14:31.996265+0000","last_undegraded":"2024-03-16T09:14:31.996265+0000","last_fullsized":"2024-03-16T09:14:31.996265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:50:02.051104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.999014+0000","last_change":"2024-03-16T09:13:37.116542+0000","last_active":"2024-03-16T09:14:31.999014+0000","last_peered":"2024-03-16T09:14:31.999014+0000","last_clean":"2024-03-16T09:14:31.999014+0000","last_became_active":"2024-03-16T09:13:37.116224+0000","last_became_peered":"2024-03-16T09:13:37.116224+0000","last_unstale":"2024-03-16T09:14:31.999014+0000","last_undegraded":"2024-03-16T09:14:31.999014+0000","last_fullsized":"2024-03-16T09:14:31.999014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:32:40.106979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998208+0000","last_change":"2024-03-16T09:13:37.128750+0000","last_active":"2024-03-16T09:14:31.998208+0000","last_peered":"2024-03-16T09:14:31.998208+0000","last_clean":"2024-03-16T09:14:31.998208+0000","last_became_active":"2024-03-16T09:13:37.128411+0000","last_became_peered":"2024-03-16T09:13:37.128411+0000","last_unstale":"2024-03-16T09:14:31.998208+0000","last_undegraded":"2024-03-16T09:14:31.998208+0000","last_fullsized":"2024-03-16T09:14:31.998208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:01:02.119578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994531+0000","last_change":"2024-03-16T09:13:37.124024+0000","last_active":"2024-03-16T09:14:31.994531+0000","last_peered":"2024-03-16T09:14:31.994531+0000","last_clean":"2024-03-16T09:14:31.994531+0000","last_became_active":"2024-03-16T09:13:37.123721+0000","last_became_peered":"2024-03-16T09:13:37.123721+0000","last_unstale":"2024-03-16T09:14:31.994531+0000","last_undegraded":"2024-03-16T09:14:31.994531+0000","last_fullsized":"2024-03-16T09:14:31.994531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:29:28.079163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385428+0000","last_change":"2024-03-16T09:13:37.116603+0000","last_active":"2024-03-16T09:14:32.385428+0000","last_peered":"2024-03-16T09:14:32.385428+0000","last_clean":"2024-03-16T09:14:32.385428+0000","last_became_active":"2024-03-16T09:13:37.116279+0000","last_became_peered":"2024-03-16T09:13:37.116279+0000","last_unstale":"2024-03-16T09:14:32.385428+0000","last_undegraded":"2024-03-16T09:14:32.385428+0000","last_fullsized":"2024-03-16T09:14:32.385428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:25:13.001817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997029+0000","last_change":"2024-03-16T09:13:37.116863+0000","last_active":"2024-03-16T09:14:31.997029+0000","last_peered":"2024-03-16T09:14:31.997029+0000","last_clean":"2024-03-16T09:14:31.997029+0000","last_became_active":"2024-03-16T09:13:37.116543+0000","last_became_peered":"2024-03-16T09:13:37.116543+0000","last_unstale":"2024-03-16T09:14:31.997029+0000","last_undegraded":"2024-03-16T09:14:31.997029+0000","last_fullsized":"2024-03-16T09:14:31.997029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:06:35.501566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998909+0000","last_change":"2024-03-16T09:13:37.120325+0000","last_active":"2024-03-16T09:14:31.998909+0000","last_peered":"2024-03-16T09:14:31.998909+0000","last_clean":"2024-03-16T09:14:31.998909+0000","last_became_active":"2024-03-16T09:13:37.119985+0000","last_became_peered":"2024-03-16T09:13:37.119985+0000","last_unstale":"2024-03-16T09:14:31.998909+0000","last_undegraded":"2024-03-16T09:14:31.998909+0000","last_fullsized":"2024-03-16T09:14:31.998909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:35.419225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517644+0000","last_change":"2024-03-16T09:13:37.119604+0000","last_active":"2024-03-16T09:14:32.517644+0000","last_peered":"2024-03-16T09:14:32.517644+0000","last_clean":"2024-03-16T09:14:32.517644+0000","last_became_active":"2024-03-16T09:13:37.119019+0000","last_became_peered":"2024-03-16T09:13:37.119019+0000","last_unstale":"2024-03-16T09:14:32.517644+0000","last_undegraded":"2024-03-16T09:14:32.517644+0000","last_fullsized":"2024-03-16T09:14:32.517644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:42:57.902370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248334+0000","last_change":"2024-03-16T09:13:37.123845+0000","last_active":"2024-03-16T09:14:32.248334+0000","last_peered":"2024-03-16T09:14:32.248334+0000","last_clean":"2024-03-16T09:14:32.248334+0000","last_became_active":"2024-03-16T09:13:37.121753+0000","last_became_peered":"2024-03-16T09:13:37.121753+0000","last_unstale":"2024-03-16T09:14:32.248334+0000","last_undegraded":"2024-03-16T09:14:32.248334+0000","last_fullsized":"2024-03-16T09:14:32.248334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:23:01.289378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517187+0000","last_change":"2024-03-16T09:13:37.118182+0000","last_active":"2024-03-16T09:14:32.517187+0000","last_peered":"2024-03-16T09:14:32.517187+0000","last_clean":"2024-03-16T09:14:32.517187+0000","last_became_active":"2024-03-16T09:13:37.117820+0000","last_became_peered":"2024-03-16T09:13:37.117820+0000","last_unstale":"2024-03-16T09:14:32.517187+0000","last_undegraded":"2024-03-16T09:14:32.517187+0000","last_fullsized":"2024-03-16T09:14:32.517187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:27:31.393266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.000632+0000","last_change":"2024-03-16T09:13:37.121348+0000","last_active":"2024-03-16T09:14:32.000632+0000","last_peered":"2024-03-16T09:14:32.000632+0000","last_clean":"2024-03-16T09:14:32.000632+0000","last_became_active":"2024-03-16T09:13:37.120826+0000","last_became_peered":"2024-03-16T09:13:37.120826+0000","last_unstale":"2024-03-16T09:14:32.000632+0000","last_undegraded":"2024-03-16T09:14:32.000632+0000","last_fullsized":"2024-03-16T09:14:32.000632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:36:33.456230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385469+0000","last_change":"2024-03-16T09:13:37.131889+0000","last_active":"2024-03-16T09:14:32.385469+0000","last_peered":"2024-03-16T09:14:32.385469+0000","last_clean":"2024-03-16T09:14:32.385469+0000","last_became_active":"2024-03-16T09:13:37.131564+0000","last_became_peered":"2024-03-16T09:13:37.131564+0000","last_unstale":"2024-03-16T09:14:32.385469+0000","last_undegraded":"2024-03-16T09:14:32.385469+0000","last_fullsized":"2024-03-16T09:14:32.385469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:43:34.255378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998395+0000","last_change":"2024-03-16T09:14:31.998395+0000","last_active":"2024-03-16T09:14:31.998395+0000","last_peered":"2024-03-16T09:14:31.998395+0000","last_clean":"2024-03-16T09:14:31.998395+0000","last_became_active":"2024-03-16T09:14:31.998063+0000","last_became_peered":"2024-03-16T09:14:31.998063+0000","last_unstale":"2024-03-16T09:14:31.998395+0000","last_undegraded":"2024-03-16T09:14:31.998395+0000","last_fullsized":"2024-03-16T09:14:31.998395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:50:57.125765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280795,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63075,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":470,"num_read_kb":303,"num_write":154,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1164,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":189225,"data_compressed":5547,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383709,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":456779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645346,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":1107143,"data_compressed":37563,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808872,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":456353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332146,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":1047310,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":455822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":515080,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":457093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":515848,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":456133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":456841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757307,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":1047576,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":59465,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":60233,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":61043,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T09:14:58.310 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-16T09:14:58.310 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-16T09:14:58.310 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-16T09:14:58.311 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph health --format=json 2024-03-16T09:14:58.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:57 smithi194 ceph-mon[36729]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:14:58.553 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:14:59.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:58 smithi159 ceph-mon[36739]: from='client.24808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:14:59.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:58 smithi112 ceph-mon[32307]: from='client.24808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:14:59.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:58 smithi194 ceph-mon[36729]: from='client.24808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:15:00.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:14:59 smithi159 ceph-mon[36739]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T09:15:00.239 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:14:59 smithi112 ceph-mon[32307]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T09:15:00.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.237+0000 7f40f4681700 1 -- 172.21.15.112:0/3894810405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec100c00 msgr2=0x7f40ec101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:00.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.237+0000 7f40f4681700 1 --2- 172.21.15.112:0/3894810405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec100c00 0x7f40ec101080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00aaa0 tx=0x7f40dc00adb0 comp rx=0 tx=0).stop 2024-03-16T09:15:00.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.237+0000 7f40f4681700 1 -- 172.21.15.112:0/3894810405 shutdown_connections 2024-03-16T09:15:00.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.237+0000 7f40f4681700 1 --2- 172.21.15.112:0/3894810405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40ec1015c0 0x7f40ec10d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:00.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.237+0000 7f40f4681700 1 --2- 172.21.15.112:0/3894810405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40ec1069e0 0x7f40ec106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:00.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.237+0000 7f40f4681700 1 --2- 172.21.15.112:0/3894810405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec100c00 0x7f40ec101080 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:00.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.237+0000 7f40f4681700 1 -- 172.21.15.112:0/3894810405 >> 172.21.15.112:0/3894810405 conn(0x7f40ec076500 msgr2=0x7f40ec076910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:00.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.237+0000 7f40f4681700 1 -- 172.21.15.112:0/3894810405 shutdown_connections 2024-03-16T09:15:00.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.237+0000 7f40f4681700 1 -- 172.21.15.112:0/3894810405 wait complete. 2024-03-16T09:15:00.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.237+0000 7f40f4681700 1 Processor -- start 2024-03-16T09:15:00.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.237+0000 7f40f4681700 1 -- start start 2024-03-16T09:15:00.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40f4681700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40ec100c00 0x7f40ec19c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:00.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40f4681700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40ec1015c0 0x7f40ec19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:00.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40f4681700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec1069e0 0x7f40ec196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:00.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40f4681700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec110500 con 0x7f40ec1069e0 2024-03-16T09:15:00.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40f4681700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec110380 con 0x7f40ec100c00 2024-03-16T09:15:00.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40f4681700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec110680 con 0x7f40ec1015c0 2024-03-16T09:15:00.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40f1c1c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40ec1015c0 0x7f40ec19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:00.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40f2c1e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec1069e0 0x7f40ec196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:00.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40f1c1c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40ec1015c0 0x7f40ec19cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:56242/0 (socket says 172.21.15.112:56242) 2024-03-16T09:15:00.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40f1c1c700 1 -- 172.21.15.112:0/1437430213 learned_addr learned my addr 172.21.15.112:0/1437430213 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:00.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40f241d700 1 --2- 172.21.15.112:0/1437430213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40ec100c00 0x7f40ec19c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:00.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40f1c1c700 1 -- 172.21.15.112:0/1437430213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40ec100c00 msgr2=0x7f40ec19c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:00.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40f1c1c700 1 --2- 172.21.15.112:0/1437430213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40ec100c00 0x7f40ec19c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:00.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40f1c1c700 1 -- 172.21.15.112:0/1437430213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec1069e0 msgr2=0x7f40ec196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:00.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40f1c1c700 1 --2- 172.21.15.112:0/1437430213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec1069e0 0x7f40ec196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:00.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40f1c1c700 1 -- 172.21.15.112:0/1437430213 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ec196f40 con 0x7f40ec1015c0 2024-03-16T09:15:00.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40f1c1c700 1 --2- 172.21.15.112:0/1437430213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40ec1015c0 0x7f40ec19cad0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f40dc000c00 tx=0x7f40dc003930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:00.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40e37fe700 1 -- 172.21.15.112:0/1437430213 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc023020 con 0x7f40ec1015c0 2024-03-16T09:15:00.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40f4681700 1 -- 172.21.15.112:0/1437430213 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40ec1971d0 con 0x7f40ec1015c0 2024-03-16T09:15:00.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40e37fe700 1 -- 172.21.15.112:0/1437430213 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f40dc00e450 con 0x7f40ec1015c0 2024-03-16T09:15:00.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.238+0000 7f40e37fe700 1 -- 172.21.15.112:0/1437430213 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc01b040 con 0x7f40ec1015c0 2024-03-16T09:15:00.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.239+0000 7f40f4681700 1 -- 172.21.15.112:0/1437430213 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40ec1a3510 con 0x7f40ec1015c0 2024-03-16T09:15:00.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.239+0000 7f40f4681700 1 -- 172.21.15.112:0/1437430213 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40ec197530 con 0x7f40ec1015c0 2024-03-16T09:15:00.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.240+0000 7f40e37fe700 1 -- 172.21.15.112:0/1437430213 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f40dc026020 con 0x7f40ec1015c0 2024-03-16T09:15:00.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.241+0000 7f40e37fe700 1 --2- 172.21.15.112:0/1437430213 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f40d80a2920 0x7f40d80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:00.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.241+0000 7f40f241d700 1 --2- 172.21.15.112:0/1437430213 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f40d80a2920 0x7f40d80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:00.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.241+0000 7f40e37fe700 1 -- 172.21.15.112:0/1437430213 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f40dc0cffa0 con 0x7f40ec1015c0 2024-03-16T09:15:00.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.241+0000 7f40f241d700 1 --2- 172.21.15.112:0/1437430213 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f40d80a2920 0x7f40d80a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f40e80044f0 tx=0x7f40e8014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:00.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.246+0000 7f40e37fe700 1 -- 172.21.15.112:0/1437430213 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f40dc02d020 con 0x7f40ec1015c0 2024-03-16T09:15:00.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:14:59 smithi194 ceph-mon[36729]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T09:15:00.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.483+0000 7f40f4681700 1 -- 172.21.15.112:0/1437430213 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f40ec197b90 con 0x7f40ec1015c0 2024-03-16T09:15:00.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.484+0000 7f40e37fe700 1 -- 172.21.15.112:0/1437430213 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f40dc09cf80 con 0x7f40ec1015c0 2024-03-16T09:15:00.486 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:00.486 INFO:teuthology.orchestra.run.smithi112.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-16T09:15:00.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.486+0000 7f40f4681700 1 -- 172.21.15.112:0/1437430213 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f40d80a2920 msgr2=0x7f40d80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:00.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.486+0000 7f40f4681700 1 --2- 172.21.15.112:0/1437430213 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f40d80a2920 0x7f40d80a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f40e80044f0 tx=0x7f40e8014040 comp rx=0 tx=0).stop 2024-03-16T09:15:00.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.486+0000 7f40f4681700 1 -- 172.21.15.112:0/1437430213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40ec1015c0 msgr2=0x7f40ec19cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:00.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.486+0000 7f40f4681700 1 --2- 172.21.15.112:0/1437430213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40ec1015c0 0x7f40ec19cad0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f40dc000c00 tx=0x7f40dc003930 comp rx=0 tx=0).stop 2024-03-16T09:15:00.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.486+0000 7f40f4681700 1 -- 172.21.15.112:0/1437430213 shutdown_connections 2024-03-16T09:15:00.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.486+0000 7f40f4681700 1 --2- 172.21.15.112:0/1437430213 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f40d80a2920 0x7f40d80a4de0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:00.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.486+0000 7f40f4681700 1 --2- 172.21.15.112:0/1437430213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40ec100c00 0x7f40ec19c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:00.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.486+0000 7f40f4681700 1 --2- 172.21.15.112:0/1437430213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40ec1069e0 0x7f40ec196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:00.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.486+0000 7f40f4681700 1 --2- 172.21.15.112:0/1437430213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f40ec1015c0 0x7f40ec19cad0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:00.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.486+0000 7f40f4681700 1 -- 172.21.15.112:0/1437430213 >> 172.21.15.112:0/1437430213 conn(0x7f40ec076500 msgr2=0x7f40ec0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:00.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.486+0000 7f40f4681700 1 -- 172.21.15.112:0/1437430213 shutdown_connections 2024-03-16T09:15:00.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:00.486+0000 7f40f4681700 1 -- 172.21.15.112:0/1437430213 wait complete. 2024-03-16T09:15:01.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:00 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1437430213' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-16T09:15:01.155 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-16T09:15:01.156 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:15:01.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:00 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1437430213' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-16T09:15:01.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.257+0000 7f3f00fd3640 1 -- 172.21.15.112:0/1407776074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3efc101800 msgr2=0x7f3efc101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:01.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.257+0000 7f3f00fd3640 1 --2- 172.21.15.112:0/1407776074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3efc101800 0x7f3efc101c20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009900 tx=0x7f3eec02bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:01.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.257+0000 7f3f00fd3640 1 -- 172.21.15.112:0/1407776074 shutdown_connections 2024-03-16T09:15:01.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.257+0000 7f3f00fd3640 1 --2- 172.21.15.112:0/1407776074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3efc1033b0 0x7f3efc109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.257+0000 7f3f00fd3640 1 --2- 172.21.15.112:0/1407776074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3efc1029f0 0x7f3efc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.257+0000 7f3f00fd3640 1 --2- 172.21.15.112:0/1407776074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3efc101800 0x7f3efc101c20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.257+0000 7f3f00fd3640 1 -- 172.21.15.112:0/1407776074 >> 172.21.15.112:0/1407776074 conn(0x7f3efc0fcd90 msgr2=0x7f3efc0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:01.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.257+0000 7f3f00fd3640 1 -- 172.21.15.112:0/1407776074 shutdown_connections 2024-03-16T09:15:01.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.257+0000 7f3f00fd3640 1 -- 172.21.15.112:0/1407776074 wait complete. 2024-03-16T09:15:01.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.258+0000 7f3f00fd3640 1 Processor -- start 2024-03-16T09:15:01.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.258+0000 7f3f00fd3640 1 -- start start 2024-03-16T09:15:01.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.258+0000 7f3f00fd3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3efc101800 0x7f3efc075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:01.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.258+0000 7f3f00fd3640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3efc1029f0 0x7f3efc073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:01.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.258+0000 7f3f00fd3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3efc1033b0 0x7f3efc074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:01.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.258+0000 7f3f00fd3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc10d560 con 0x7f3efc1033b0 2024-03-16T09:15:01.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.258+0000 7f3f00fd3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc10d3e0 con 0x7f3efc101800 2024-03-16T09:15:01.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.258+0000 7f3f00fd3640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3efc10d6e0 con 0x7f3efc1029f0 2024-03-16T09:15:01.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.258+0000 7f3efad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3efc1033b0 0x7f3efc074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:01.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.258+0000 7f3efad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3efc1033b0 0x7f3efc074110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45692/0 (socket says 172.21.15.112:45692) 2024-03-16T09:15:01.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.258+0000 7f3efad9d640 1 -- 172.21.15.112:0/1420711606 learned_addr learned my addr 172.21.15.112:0/1420711606 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:01.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.258+0000 7f3efa59c640 1 --2- 172.21.15.112:0/1420711606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3efc101800 0x7f3efc075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:01.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.258+0000 7f3ef3fff640 1 --2- 172.21.15.112:0/1420711606 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3efc1029f0 0x7f3efc073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:01.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.258+0000 7f3efad9d640 1 -- 172.21.15.112:0/1420711606 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3efc1029f0 msgr2=0x7f3efc073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:01.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.258+0000 7f3efad9d640 1 --2- 172.21.15.112:0/1420711606 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3efc1029f0 0x7f3efc073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.258+0000 7f3efad9d640 1 -- 172.21.15.112:0/1420711606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3efc101800 msgr2=0x7f3efc075590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:01.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.258+0000 7f3efad9d640 1 --2- 172.21.15.112:0/1420711606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3efc101800 0x7f3efc075590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.258+0000 7f3efad9d640 1 -- 172.21.15.112:0/1420711606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3efc0749a0 con 0x7f3efc1033b0 2024-03-16T09:15:01.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.258+0000 7f3efad9d640 1 --2- 172.21.15.112:0/1420711606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3efc1033b0 0x7f3efc074110 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400eb10 tx=0x7f3ef400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:01.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.259+0000 7f3ef37fe640 1 -- 172.21.15.112:0/1420711606 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4004b20 con 0x7f3efc1033b0 2024-03-16T09:15:01.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.259+0000 7f3f00fd3640 1 -- 172.21.15.112:0/1420711606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3efc1a1d20 con 0x7f3efc1033b0 2024-03-16T09:15:01.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.259+0000 7f3ef37fe640 1 -- 172.21.15.112:0/1420711606 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3ef4004cc0 con 0x7f3efc1033b0 2024-03-16T09:15:01.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.259+0000 7f3ef37fe640 1 -- 172.21.15.112:0/1420711606 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40107f0 con 0x7f3efc1033b0 2024-03-16T09:15:01.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.259+0000 7f3f00fd3640 1 -- 172.21.15.112:0/1420711606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3efc1a21c0 con 0x7f3efc1033b0 2024-03-16T09:15:01.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.260+0000 7f3f00fd3640 1 -- 172.21.15.112:0/1420711606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3efc108b20 con 0x7f3efc1033b0 2024-03-16T09:15:01.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.261+0000 7f3ef37fe640 1 -- 172.21.15.112:0/1420711606 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f3ef4005020 con 0x7f3efc1033b0 2024-03-16T09:15:01.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.261+0000 7f3ef37fe640 1 --2- 172.21.15.112:0/1420711606 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3ee80a2890 0x7f3ee80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:01.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.261+0000 7f3efa59c640 1 --2- 172.21.15.112:0/1420711606 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3ee80a2890 0x7f3ee80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:01.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.261+0000 7f3ef37fe640 1 -- 172.21.15.112:0/1420711606 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f3ef40cf1d0 con 0x7f3efc1033b0 2024-03-16T09:15:01.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.262+0000 7f3efa59c640 1 --2- 172.21.15.112:0/1420711606 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3ee80a2890 0x7f3ee80a4d50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3eec000c00 tx=0x7f3eec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:01.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.266+0000 7f3ef37fe640 1 -- 172.21.15.112:0/1420711606 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3ef409c120 con 0x7f3efc1033b0 2024-03-16T09:15:01.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:00 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1437430213' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-16T09:15:01.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.444+0000 7f3f00fd3640 1 -- 172.21.15.112:0/1420711606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3efc02d360 con 0x7f3efc1033b0 2024-03-16T09:15:01.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.445+0000 7f3ef37fe640 1 -- 172.21.15.112:0/1420711606 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22725 (secure 0 0 0) 0x7f3ef409ba20 con 0x7f3efc1033b0 2024-03-16T09:15:01.447 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:01.447 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":87,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:14:31.991998+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T09:13:35.040628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T09:13:35.689419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:15:01.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.447+0000 7f3f00fd3640 1 -- 172.21.15.112:0/1420711606 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3ee80a2890 msgr2=0x7f3ee80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:01.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.447+0000 7f3f00fd3640 1 --2- 172.21.15.112:0/1420711606 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3ee80a2890 0x7f3ee80a4d50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3eec000c00 tx=0x7f3eec005fd0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.447+0000 7f3f00fd3640 1 -- 172.21.15.112:0/1420711606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3efc1033b0 msgr2=0x7f3efc074110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:01.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.447+0000 7f3f00fd3640 1 --2- 172.21.15.112:0/1420711606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3efc1033b0 0x7f3efc074110 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400eb10 tx=0x7f3ef400c2d0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.447+0000 7f3f00fd3640 1 -- 172.21.15.112:0/1420711606 shutdown_connections 2024-03-16T09:15:01.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.447+0000 7f3f00fd3640 1 --2- 172.21.15.112:0/1420711606 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3ee80a2890 0x7f3ee80a4d50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.447+0000 7f3f00fd3640 1 --2- 172.21.15.112:0/1420711606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3efc1033b0 0x7f3efc074110 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.447+0000 7f3f00fd3640 1 --2- 172.21.15.112:0/1420711606 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3efc1029f0 0x7f3efc073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.447+0000 7f3f00fd3640 1 --2- 172.21.15.112:0/1420711606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3efc101800 0x7f3efc075590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.447+0000 7f3f00fd3640 1 -- 172.21.15.112:0/1420711606 >> 172.21.15.112:0/1420711606 conn(0x7f3efc0fcd90 msgr2=0x7f3efc105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:01.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.447+0000 7f3f00fd3640 1 -- 172.21.15.112:0/1420711606 shutdown_connections 2024-03-16T09:15:01.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.447+0000 7f3f00fd3640 1 -- 172.21.15.112:0/1420711606 wait complete. 2024-03-16T09:15:01.461 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T09:08:57.135312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T09:13:35.040628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T09:13:35.689419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T09:15:01.461 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T09:15:01.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.564+0000 7f443c80c640 1 -- 172.21.15.112:0/2319903296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4438109680 msgr2=0x7f443810bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:01.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.564+0000 7f443c80c640 1 --2- 172.21.15.112:0/2319903296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4438109680 0x7f443810bb20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4430009ca0 tx=0x7f443002c180 comp rx=0 tx=0).stop 2024-03-16T09:15:01.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.564+0000 7f443c80c640 1 -- 172.21.15.112:0/2319903296 shutdown_connections 2024-03-16T09:15:01.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.564+0000 7f443c80c640 1 --2- 172.21.15.112:0/2319903296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4438109680 0x7f443810bb20 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.564+0000 7f443c80c640 1 --2- 172.21.15.112:0/2319903296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44380741e0 0x7f4438074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.564+0000 7f443c80c640 1 --2- 172.21.15.112:0/2319903296 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f44380757c0 0x7f4438073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.564+0000 7f443c80c640 1 -- 172.21.15.112:0/2319903296 >> 172.21.15.112:0/2319903296 conn(0x7f44380fcd80 msgr2=0x7f44380ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.564+0000 7f443c80c640 1 -- 172.21.15.112:0/2319903296 shutdown_connections 2024-03-16T09:15:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.564+0000 7f443c80c640 1 -- 172.21.15.112:0/2319903296 wait complete. 2024-03-16T09:15:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f443c80c640 1 Processor -- start 2024-03-16T09:15:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f443c80c640 1 -- start start 2024-03-16T09:15:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f443c80c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44380741e0 0x7f4438199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f443c80c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44380757c0 0x7f443819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f443c80c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4438109680 0x7f44381a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f443c80c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f443810f7a0 con 0x7f44380741e0 2024-03-16T09:15:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f443c80c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f443810f620 con 0x7f44380757c0 2024-03-16T09:15:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f443c80c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f443810f920 con 0x7f4438109680 2024-03-16T09:15:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f443659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44380741e0 0x7f4438199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:01.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f443659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44380741e0 0x7f4438199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38058/0 (socket says 172.21.15.112:38058) 2024-03-16T09:15:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f443659c640 1 -- 172.21.15.112:0/738595281 learned_addr learned my addr 172.21.15.112:0/738595281 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f4435d9b640 1 --2- 172.21.15.112:0/738595281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44380757c0 0x7f443819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f4436d9d640 1 --2- 172.21.15.112:0/738595281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4438109680 0x7f44381a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f443659c640 1 -- 172.21.15.112:0/738595281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4438109680 msgr2=0x7f44381a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f443659c640 1 --2- 172.21.15.112:0/738595281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4438109680 0x7f44381a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f443659c640 1 -- 172.21.15.112:0/738595281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44380757c0 msgr2=0x7f443819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f443659c640 1 --2- 172.21.15.112:0/738595281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44380757c0 0x7f443819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f443659c640 1 -- 172.21.15.112:0/738595281 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44381a19c0 con 0x7f44380741e0 2024-03-16T09:15:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f4435d9b640 1 --2- 172.21.15.112:0/738595281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44380757c0 0x7f443819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.565+0000 7f443659c640 1 --2- 172.21.15.112:0/738595281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44380741e0 0x7f4438199db0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f442800e9f0 tx=0x7f442800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.566+0000 7f44277fe640 1 -- 172.21.15.112:0/738595281 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442800cd80 con 0x7f44380741e0 2024-03-16T09:15:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.566+0000 7f44277fe640 1 -- 172.21.15.112:0/738595281 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4428005520 con 0x7f44380741e0 2024-03-16T09:15:01.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.566+0000 7f443c80c640 1 -- 172.21.15.112:0/738595281 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44381a1cb0 con 0x7f44380741e0 2024-03-16T09:15:01.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.566+0000 7f44277fe640 1 -- 172.21.15.112:0/738595281 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428010690 con 0x7f44380741e0 2024-03-16T09:15:01.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.566+0000 7f443c80c640 1 -- 172.21.15.112:0/738595281 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44381a21b0 con 0x7f44380741e0 2024-03-16T09:15:01.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.567+0000 7f443c80c640 1 -- 172.21.15.112:0/738595281 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4438108440 con 0x7f44380741e0 2024-03-16T09:15:01.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.568+0000 7f44277fe640 1 -- 172.21.15.112:0/738595281 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f44280046c0 con 0x7f44380741e0 2024-03-16T09:15:01.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.568+0000 7f44277fe640 1 --2- 172.21.15.112:0/738595281 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f44200a2890 0x7f44200a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:01.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.568+0000 7f4435d9b640 1 --2- 172.21.15.112:0/738595281 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f44200a2890 0x7f44200a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:01.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.568+0000 7f44277fe640 1 -- 172.21.15.112:0/738595281 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f44280cf090 con 0x7f44380741e0 2024-03-16T09:15:01.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.569+0000 7f4435d9b640 1 --2- 172.21.15.112:0/738595281 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f44200a2890 0x7f44200a4d50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f442c009fc0 tx=0x7f442c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:01.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.572+0000 7f44277fe640 1 -- 172.21.15.112:0/738595281 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f442809bfe0 con 0x7f44380741e0 2024-03-16T09:15:01.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.748+0000 7f443c80c640 1 -- 172.21.15.112:0/738595281 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f443802d3d0 con 0x7f44380741e0 2024-03-16T09:15:01.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.749+0000 7f44277fe640 1 -- 172.21.15.112:0/738595281 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f442809b8e0 con 0x7f44380741e0 2024-03-16T09:15:01.751 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-03-16T09:15:01.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.751+0000 7f443c80c640 1 -- 172.21.15.112:0/738595281 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f44200a2890 msgr2=0x7f44200a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:01.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.751+0000 7f443c80c640 1 --2- 172.21.15.112:0/738595281 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f44200a2890 0x7f44200a4d50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f442c009fc0 tx=0x7f442c005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:01.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.751+0000 7f443c80c640 1 -- 172.21.15.112:0/738595281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44380741e0 msgr2=0x7f4438199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:01.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.751+0000 7f443c80c640 1 --2- 172.21.15.112:0/738595281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44380741e0 0x7f4438199db0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f442800e9f0 tx=0x7f442800edf0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.751+0000 7f443c80c640 1 -- 172.21.15.112:0/738595281 shutdown_connections 2024-03-16T09:15:01.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.751+0000 7f443c80c640 1 --2- 172.21.15.112:0/738595281 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f44200a2890 0x7f44200a4d50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.751+0000 7f443c80c640 1 --2- 172.21.15.112:0/738595281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4438109680 0x7f44381a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.751+0000 7f443c80c640 1 --2- 172.21.15.112:0/738595281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44380757c0 0x7f443819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.751+0000 7f443c80c640 1 --2- 172.21.15.112:0/738595281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44380741e0 0x7f4438199db0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.751+0000 7f443c80c640 1 -- 172.21.15.112:0/738595281 >> 172.21.15.112:0/738595281 conn(0x7f44380fcd80 msgr2=0x7f4438105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:01.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.751+0000 7f443c80c640 1 -- 172.21.15.112:0/738595281 shutdown_connections 2024-03-16T09:15:01.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.751+0000 7f443c80c640 1 -- 172.21.15.112:0/738595281 wait complete. 2024-03-16T09:15:01.764 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T09:15:01.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.866+0000 7f3baece8640 1 -- 172.21.15.112:0/3863186615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ba81033a0 msgr2=0x7f3ba8109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:01.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.866+0000 7f3baece8640 1 --2- 172.21.15.112:0/3863186615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ba81033a0 0x7f3ba8109c30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009ca0 tx=0x7f3ba002c0f0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.866+0000 7f3baece8640 1 -- 172.21.15.112:0/3863186615 shutdown_connections 2024-03-16T09:15:01.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.866+0000 7f3baece8640 1 --2- 172.21.15.112:0/3863186615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ba81033a0 0x7f3ba8109c30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.866+0000 7f3baece8640 1 --2- 172.21.15.112:0/3863186615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ba81029e0 0x7f3ba8102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.866+0000 7f3baece8640 1 --2- 172.21.15.112:0/3863186615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ba81017f0 0x7f3ba8101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.866+0000 7f3baece8640 1 -- 172.21.15.112:0/3863186615 >> 172.21.15.112:0/3863186615 conn(0x7f3ba80fcd80 msgr2=0x7f3ba80ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:01.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.866+0000 7f3baece8640 1 -- 172.21.15.112:0/3863186615 shutdown_connections 2024-03-16T09:15:01.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.866+0000 7f3baece8640 1 -- 172.21.15.112:0/3863186615 wait complete. 2024-03-16T09:15:01.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.867+0000 7f3baece8640 1 Processor -- start 2024-03-16T09:15:01.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.867+0000 7f3baece8640 1 -- start start 2024-03-16T09:15:01.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.867+0000 7f3baece8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ba81017f0 0x7f3ba8075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:01.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.867+0000 7f3baece8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ba81029e0 0x7f3ba8073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.867+0000 7f3baece8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ba81033a0 0x7f3ba8074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.867+0000 7f3baece8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba810d5a0 con 0x7f3ba81017f0 2024-03-16T09:15:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.867+0000 7f3baece8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba810d420 con 0x7f3ba81033a0 2024-03-16T09:15:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.867+0000 7f3baece8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba810d720 con 0x7f3ba81029e0 2024-03-16T09:15:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.867+0000 7f3baca84640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ba81017f0 0x7f3ba8075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.867+0000 7f3baca84640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ba81017f0 0x7f3ba8075590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38072/0 (socket says 172.21.15.112:38072) 2024-03-16T09:15:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.867+0000 7f3baca84640 1 -- 172.21.15.112:0/2943258554 learned_addr learned my addr 172.21.15.112:0/2943258554 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.867+0000 7f3ba7fff640 1 --2- 172.21.15.112:0/2943258554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ba81029e0 0x7f3ba8073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.867+0000 7f3bad285640 1 --2- 172.21.15.112:0/2943258554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ba81033a0 0x7f3ba8074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.867+0000 7f3baca84640 1 -- 172.21.15.112:0/2943258554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ba81029e0 msgr2=0x7f3ba8073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.867+0000 7f3baca84640 1 --2- 172.21.15.112:0/2943258554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ba81029e0 0x7f3ba8073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.867+0000 7f3baca84640 1 -- 172.21.15.112:0/2943258554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ba81033a0 msgr2=0x7f3ba8074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.867+0000 7f3baca84640 1 --2- 172.21.15.112:0/2943258554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ba81033a0 0x7f3ba8074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:01.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.867+0000 7f3baca84640 1 -- 172.21.15.112:0/2943258554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba80749a0 con 0x7f3ba81017f0 2024-03-16T09:15:01.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.868+0000 7f3baca84640 1 --2- 172.21.15.112:0/2943258554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ba81017f0 0x7f3ba8075590 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800e9c0 tx=0x7f3b9800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:01.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.868+0000 7f3ba5ffb640 1 -- 172.21.15.112:0/2943258554 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800cd80 con 0x7f3ba81017f0 2024-03-16T09:15:01.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.868+0000 7f3ba5ffb640 1 -- 172.21.15.112:0/2943258554 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3b980035f0 con 0x7f3ba81017f0 2024-03-16T09:15:01.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.868+0000 7f3baece8640 1 -- 172.21.15.112:0/2943258554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba81a1d20 con 0x7f3ba81017f0 2024-03-16T09:15:01.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.868+0000 7f3ba5ffb640 1 -- 172.21.15.112:0/2943258554 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98010670 con 0x7f3ba81017f0 2024-03-16T09:15:01.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.868+0000 7f3baece8640 1 -- 172.21.15.112:0/2943258554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba81a21c0 con 0x7f3ba81017f0 2024-03-16T09:15:01.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.869+0000 7f3baece8640 1 -- 172.21.15.112:0/2943258554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ba8108ab0 con 0x7f3ba81017f0 2024-03-16T09:15:01.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.870+0000 7f3ba5ffb640 1 -- 172.21.15.112:0/2943258554 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137342+0+0 (secure 0 0 0) 0x7f3b980037a0 con 0x7f3ba81017f0 2024-03-16T09:15:01.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.870+0000 7f3ba5ffb640 1 --2- 172.21.15.112:0/2943258554 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3b900a2910 0x7f3b900a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:01.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.871+0000 7f3ba7fff640 1 --2- 172.21.15.112:0/2943258554 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3b900a2910 0x7f3b900a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:01.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.871+0000 7f3ba5ffb640 1 -- 172.21.15.112:0/2943258554 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f3b980cef10 con 0x7f3ba81017f0 2024-03-16T09:15:01.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.871+0000 7f3ba7fff640 1 --2- 172.21.15.112:0/2943258554 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3b900a2910 0x7f3b900a4dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0095f0 tx=0x7f3b9c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:01.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.875+0000 7f3ba5ffb640 1 -- 172.21.15.112:0/2943258554 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3b9809be60 con 0x7f3ba81017f0 2024-03-16T09:15:01.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:01.979+0000 7f3ba5ffb640 1 -- 172.21.15.112:0/2943258554 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f3b98097dd0 con 0x7f3ba81017f0 2024-03-16T09:15:02.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.057+0000 7f3baece8640 1 -- 172.21.15.112:0/2943258554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3ba802d410 con 0x7f3ba81017f0 2024-03-16T09:15:02.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.058+0000 7f3ba5ffb640 1 -- 172.21.15.112:0/2943258554 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f3b98046130 con 0x7f3ba81017f0 2024-03-16T09:15:02.060 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-03-16T09:15:02.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.060+0000 7f3baece8640 1 -- 172.21.15.112:0/2943258554 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3b900a2910 msgr2=0x7f3b900a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:02.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.060+0000 7f3baece8640 1 --2- 172.21.15.112:0/2943258554 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3b900a2910 0x7f3b900a4dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0095f0 tx=0x7f3b9c005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:02.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.060+0000 7f3baece8640 1 -- 172.21.15.112:0/2943258554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ba81017f0 msgr2=0x7f3ba8075590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:02.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.060+0000 7f3baece8640 1 --2- 172.21.15.112:0/2943258554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ba81017f0 0x7f3ba8075590 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800e9c0 tx=0x7f3b9800ee80 comp rx=0 tx=0).stop 2024-03-16T09:15:02.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.060+0000 7f3baece8640 1 -- 172.21.15.112:0/2943258554 shutdown_connections 2024-03-16T09:15:02.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.060+0000 7f3baece8640 1 --2- 172.21.15.112:0/2943258554 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3b900a2910 0x7f3b900a4dd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.060+0000 7f3baece8640 1 --2- 172.21.15.112:0/2943258554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ba81033a0 0x7f3ba8074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.060+0000 7f3baece8640 1 --2- 172.21.15.112:0/2943258554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ba81029e0 0x7f3ba8073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.060+0000 7f3baece8640 1 --2- 172.21.15.112:0/2943258554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ba81017f0 0x7f3ba8075590 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.060+0000 7f3baece8640 1 -- 172.21.15.112:0/2943258554 >> 172.21.15.112:0/2943258554 conn(0x7f3ba80fcd80 msgr2=0x7f3ba8105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:02.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.060+0000 7f3baece8640 1 -- 172.21.15.112:0/2943258554 shutdown_connections 2024-03-16T09:15:02.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.060+0000 7f3baece8640 1 -- 172.21.15.112:0/2943258554 wait complete. 2024-03-16T09:15:02.072 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:01 smithi112 ceph-mon[32307]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T09:15:02.072 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:01 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1420711606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:02.072 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:01 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/738595281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:02.073 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T09:15:02.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.175+0000 7f88e6659640 1 -- 172.21.15.112:0/3637666844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88e01033a0 msgr2=0x7f88e0109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:02.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.175+0000 7f88e6659640 1 --2- 172.21.15.112:0/3637666844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88e01033a0 0x7f88e0109c30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009ca0 tx=0x7f88d802c0f0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.175+0000 7f88e6659640 1 -- 172.21.15.112:0/3637666844 shutdown_connections 2024-03-16T09:15:02.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.175+0000 7f88e6659640 1 --2- 172.21.15.112:0/3637666844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88e01033a0 0x7f88e0109c30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.175+0000 7f88e6659640 1 --2- 172.21.15.112:0/3637666844 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88e01029e0 0x7f88e0102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.175+0000 7f88e6659640 1 --2- 172.21.15.112:0/3637666844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88e01017f0 0x7f88e0101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.175+0000 7f88e6659640 1 -- 172.21.15.112:0/3637666844 >> 172.21.15.112:0/3637666844 conn(0x7f88e00fcd80 msgr2=0x7f88e00ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:02.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.175+0000 7f88e6659640 1 -- 172.21.15.112:0/3637666844 shutdown_connections 2024-03-16T09:15:02.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.175+0000 7f88e6659640 1 -- 172.21.15.112:0/3637666844 wait complete. 2024-03-16T09:15:02.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.176+0000 7f88e6659640 1 Processor -- start 2024-03-16T09:15:02.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.176+0000 7f88e6659640 1 -- start start 2024-03-16T09:15:02.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.176+0000 7f88e6659640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88e01017f0 0x7f88e0197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:02.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.176+0000 7f88e6659640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88e01029e0 0x7f88e01980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:02.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.176+0000 7f88e6659640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88e01033a0 0x7f88e019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:02.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.176+0000 7f88e6659640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f88e010d5a0 con 0x7f88e01033a0 2024-03-16T09:15:02.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.176+0000 7f88e6659640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f88e010d420 con 0x7f88e01029e0 2024-03-16T09:15:02.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.176+0000 7f88e6659640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f88e010d720 con 0x7f88e01017f0 2024-03-16T09:15:02.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.176+0000 7f88e4bf6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88e01033a0 0x7f88e019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:02.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.176+0000 7f88e4bf6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88e01033a0 0x7f88e019f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38092/0 (socket says 172.21.15.112:38092) 2024-03-16T09:15:02.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.176+0000 7f88e4bf6640 1 -- 172.21.15.112:0/3600016654 learned_addr learned my addr 172.21.15.112:0/3600016654 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:02.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.176+0000 7f88dffff640 1 --2- 172.21.15.112:0/3600016654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88e01017f0 0x7f88e0197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:02.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.176+0000 7f88df7fe640 1 --2- 172.21.15.112:0/3600016654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88e01029e0 0x7f88e01980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:02.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.176+0000 7f88e4bf6640 1 -- 172.21.15.112:0/3600016654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88e01017f0 msgr2=0x7f88e0197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:02.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.176+0000 7f88e4bf6640 1 --2- 172.21.15.112:0/3600016654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88e01017f0 0x7f88e0197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.176+0000 7f88e4bf6640 1 -- 172.21.15.112:0/3600016654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88e01029e0 msgr2=0x7f88e01980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:02.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.176+0000 7f88e4bf6640 1 --2- 172.21.15.112:0/3600016654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88e01029e0 0x7f88e01980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.176+0000 7f88e4bf6640 1 -- 172.21.15.112:0/3600016654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e019f7d0 con 0x7f88e01033a0 2024-03-16T09:15:02.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.176+0000 7f88e4bf6640 1 --2- 172.21.15.112:0/3600016654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88e01033a0 0x7f88e019f160 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f88d802c700 tx=0x7f88d80346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:02.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.177+0000 7f88dd7fa640 1 -- 172.21.15.112:0/3600016654 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8039040 con 0x7f88e01033a0 2024-03-16T09:15:02.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.177+0000 7f88e6659640 1 -- 172.21.15.112:0/3600016654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e019fa60 con 0x7f88e01033a0 2024-03-16T09:15:02.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.177+0000 7f88dd7fa640 1 -- 172.21.15.112:0/3600016654 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f88d8044070 con 0x7f88e01033a0 2024-03-16T09:15:02.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.177+0000 7f88dd7fa640 1 -- 172.21.15.112:0/3600016654 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d80043c0 con 0x7f88e01033a0 2024-03-16T09:15:02.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.177+0000 7f88e6659640 1 -- 172.21.15.112:0/3600016654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e019ffa0 con 0x7f88e01033a0 2024-03-16T09:15:02.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.178+0000 7f88e6659640 1 -- 172.21.15.112:0/3600016654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e0108b10 con 0x7f88e01033a0 2024-03-16T09:15:02.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.179+0000 7f88dd7fa640 1 -- 172.21.15.112:0/3600016654 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f88d8004560 con 0x7f88e01033a0 2024-03-16T09:15:02.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.179+0000 7f88dd7fa640 1 --2- 172.21.15.112:0/3600016654 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88c80a2800 0x7f88c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:02.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.179+0000 7f88dffff640 1 --2- 172.21.15.112:0/3600016654 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88c80a2800 0x7f88c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:02.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.179+0000 7f88dd7fa640 1 -- 172.21.15.112:0/3600016654 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f88d80f1690 con 0x7f88e01033a0 2024-03-16T09:15:02.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.180+0000 7f88dffff640 1 --2- 172.21.15.112:0/3600016654 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88c80a2800 0x7f88c80a4cc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f88e0102840 tx=0x7f88d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:02.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.183+0000 7f88dd7fa640 1 -- 172.21.15.112:0/3600016654 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f88d80be860 con 0x7f88e01033a0 2024-03-16T09:15:02.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:01 smithi159 ceph-mon[36739]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T09:15:02.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:01 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1420711606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:02.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:01 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/738595281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:02.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.356+0000 7f88e6659640 1 -- 172.21.15.112:0/3600016654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f88e002d1b0 con 0x7f88e01033a0 2024-03-16T09:15:02.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.356+0000 7f88dd7fa640 1 -- 172.21.15.112:0/3600016654 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f88d80be160 con 0x7f88e01033a0 2024-03-16T09:15:02.358 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-03-16T09:15:02.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.358+0000 7f88e6659640 1 -- 172.21.15.112:0/3600016654 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88c80a2800 msgr2=0x7f88c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:02.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.358+0000 7f88e6659640 1 --2- 172.21.15.112:0/3600016654 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88c80a2800 0x7f88c80a4cc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f88e0102840 tx=0x7f88d0005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:02.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.358+0000 7f88e6659640 1 -- 172.21.15.112:0/3600016654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88e01033a0 msgr2=0x7f88e019f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:02.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.358+0000 7f88e6659640 1 --2- 172.21.15.112:0/3600016654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88e01033a0 0x7f88e019f160 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f88d802c700 tx=0x7f88d80346d0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.358+0000 7f88e6659640 1 -- 172.21.15.112:0/3600016654 shutdown_connections 2024-03-16T09:15:02.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.358+0000 7f88e6659640 1 --2- 172.21.15.112:0/3600016654 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88c80a2800 0x7f88c80a4cc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.358+0000 7f88e6659640 1 --2- 172.21.15.112:0/3600016654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88e01033a0 0x7f88e019f160 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.358+0000 7f88e6659640 1 --2- 172.21.15.112:0/3600016654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88e01029e0 0x7f88e01980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.358+0000 7f88e6659640 1 --2- 172.21.15.112:0/3600016654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88e01017f0 0x7f88e0197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.358+0000 7f88e6659640 1 -- 172.21.15.112:0/3600016654 >> 172.21.15.112:0/3600016654 conn(0x7f88e00fcd80 msgr2=0x7f88e0105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:02.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.358+0000 7f88e6659640 1 -- 172.21.15.112:0/3600016654 shutdown_connections 2024-03-16T09:15:02.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.358+0000 7f88e6659640 1 -- 172.21.15.112:0/3600016654 wait complete. 2024-03-16T09:15:02.371 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:15:02.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:01 smithi194 ceph-mon[36729]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T09:15:02.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:01 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1420711606' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:02.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:01 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/738595281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.471+0000 7f88ddd90640 1 -- 172.21.15.112:0/353792644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d8101a00 msgr2=0x7f88d8108320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.471+0000 7f88ddd90640 1 --2- 172.21.15.112:0/353792644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d8101a00 0x7f88d8108320 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009ca0 tx=0x7f88d002c0f0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.471+0000 7f88ddd90640 1 -- 172.21.15.112:0/353792644 shutdown_connections 2024-03-16T09:15:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.471+0000 7f88ddd90640 1 --2- 172.21.15.112:0/353792644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d8101a00 0x7f88d8108320 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.471+0000 7f88ddd90640 1 --2- 172.21.15.112:0/353792644 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88d8101040 0x7f88d81014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.471+0000 7f88ddd90640 1 --2- 172.21.15.112:0/353792644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d80ffe60 0x7f88d8100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.471+0000 7f88ddd90640 1 -- 172.21.15.112:0/353792644 >> 172.21.15.112:0/353792644 conn(0x7f88d80fb420 msgr2=0x7f88d80fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.471+0000 7f88ddd90640 1 -- 172.21.15.112:0/353792644 shutdown_connections 2024-03-16T09:15:02.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.471+0000 7f88ddd90640 1 -- 172.21.15.112:0/353792644 wait complete. 2024-03-16T09:15:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.471+0000 7f88ddd90640 1 Processor -- start 2024-03-16T09:15:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88ddd90640 1 -- start start 2024-03-16T09:15:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88ddd90640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88d80ffe60 0x7f88d8196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88ddd90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d8101040 0x7f88d8197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88ddd90640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d8101a00 0x7f88d819e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88ddd90640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f88d810c510 con 0x7f88d8101040 2024-03-16T09:15:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88ddd90640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f88d810c390 con 0x7f88d8101a00 2024-03-16T09:15:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88ddd90640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f88d810c690 con 0x7f88d80ffe60 2024-03-16T09:15:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88d6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d8101040 0x7f88d8197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88d77fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88d80ffe60 0x7f88d8196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88d6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d8101040 0x7f88d8197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38108/0 (socket says 172.21.15.112:38108) 2024-03-16T09:15:02.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88d6ffd640 1 -- 172.21.15.112:0/437656025 learned_addr learned my addr 172.21.15.112:0/437656025 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88d77fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88d80ffe60 0x7f88d8196ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:38674/0 (socket says 172.21.15.112:38674) 2024-03-16T09:15:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88d7fff640 1 --2- 172.21.15.112:0/437656025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d8101a00 0x7f88d819e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88d6ffd640 1 -- 172.21.15.112:0/437656025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88d80ffe60 msgr2=0x7f88d8196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88d6ffd640 1 --2- 172.21.15.112:0/437656025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88d80ffe60 0x7f88d8196ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88d6ffd640 1 -- 172.21.15.112:0/437656025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d8101a00 msgr2=0x7f88d819e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88d6ffd640 1 --2- 172.21.15.112:0/437656025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d8101a00 0x7f88d819e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88d6ffd640 1 -- 172.21.15.112:0/437656025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d819e700 con 0x7f88d8101040 2024-03-16T09:15:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88d77fe640 1 --2- 172.21.15.112:0/437656025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88d80ffe60 0x7f88d8196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88d6ffd640 1 --2- 172.21.15.112:0/437656025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d8101040 0x7f88d8197020 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00d950 tx=0x7f88cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88d4ff9640 1 -- 172.21.15.112:0/437656025 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc017030 con 0x7f88d8101040 2024-03-16T09:15:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.472+0000 7f88ddd90640 1 -- 172.21.15.112:0/437656025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d819e9f0 con 0x7f88d8101040 2024-03-16T09:15:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.473+0000 7f88d4ff9640 1 -- 172.21.15.112:0/437656025 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f88cc004020 con 0x7f88d8101040 2024-03-16T09:15:02.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.473+0000 7f88d4ff9640 1 -- 172.21.15.112:0/437656025 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc004a00 con 0x7f88d8101040 2024-03-16T09:15:02.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.473+0000 7f88ddd90640 1 -- 172.21.15.112:0/437656025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d819ef90 con 0x7f88d8101040 2024-03-16T09:15:02.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.474+0000 7f88ddd90640 1 -- 172.21.15.112:0/437656025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d81071a0 con 0x7f88d8101040 2024-03-16T09:15:02.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.475+0000 7f88d4ff9640 1 -- 172.21.15.112:0/437656025 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f88cc0041d0 con 0x7f88d8101040 2024-03-16T09:15:02.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.475+0000 7f88d4ff9640 1 --2- 172.21.15.112:0/437656025 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88c00a2800 0x7f88c00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:02.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.475+0000 7f88d77fe640 1 --2- 172.21.15.112:0/437656025 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88c00a2800 0x7f88c00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:02.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.475+0000 7f88d4ff9640 1 -- 172.21.15.112:0/437656025 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f88cc0ce3e0 con 0x7f88d8101040 2024-03-16T09:15:02.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.475+0000 7f88d77fe640 1 --2- 172.21.15.112:0/437656025 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88c00a2800 0x7f88c00a4cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f88d8100bb0 tx=0x7f88c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:02.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.480+0000 7f88d4ff9640 1 -- 172.21.15.112:0/437656025 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f88cc09b490 con 0x7f88d8101040 2024-03-16T09:15:02.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.691+0000 7f88ddd90640 1 -- 172.21.15.112:0/437656025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88d80623e0 con 0x7f88d8101040 2024-03-16T09:15:02.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.692+0000 7f88d4ff9640 1 -- 172.21.15.112:0/437656025 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11881 (secure 0 0 0) 0x7f88cc09ad90 con 0x7f88d8101040 2024-03-16T09:15:02.694 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:02.694 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3939748854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3939748854},{"type":"v1","addr":"172.21.15.112:6839","nonce":3939748854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3118523258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3118523258},{"type":"v1","addr":"172.21.15.112:6841","nonce":3118523258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T09:13:36.450216+0000","modified":"2024-03-16T09:13:40.527486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/1682076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":1682076},{"type":"v1","addr":"172.21.15.194:6837","nonce":1682076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.194:6839/2528943309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2528943309},{"type":"v1","addr":"172.21.15.194:6839","nonce":2528943309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/272895965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":272895965},{"type":"v1","addr":"172.21.15.159:6839","nonce":272895965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24452,24466,24494]},"id":1}]} 2024-03-16T09:15:02.694 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2024-03-16T09:15:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.694+0000 7f88ddd90640 1 -- 172.21.15.112:0/437656025 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88c00a2800 msgr2=0x7f88c00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.694+0000 7f88ddd90640 1 --2- 172.21.15.112:0/437656025 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88c00a2800 0x7f88c00a4cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f88d8100bb0 tx=0x7f88c8005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.694+0000 7f88ddd90640 1 -- 172.21.15.112:0/437656025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d8101040 msgr2=0x7f88d8197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.694+0000 7f88ddd90640 1 --2- 172.21.15.112:0/437656025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d8101040 0x7f88d8197020 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00d950 tx=0x7f88cc00de10 comp rx=0 tx=0).stop 2024-03-16T09:15:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.694+0000 7f88ddd90640 1 -- 172.21.15.112:0/437656025 shutdown_connections 2024-03-16T09:15:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.694+0000 7f88ddd90640 1 --2- 172.21.15.112:0/437656025 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88c00a2800 0x7f88c00a4cc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.694+0000 7f88ddd90640 1 --2- 172.21.15.112:0/437656025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88d8101a00 0x7f88d819e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.694+0000 7f88ddd90640 1 --2- 172.21.15.112:0/437656025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88d8101040 0x7f88d8197020 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.694+0000 7f88ddd90640 1 --2- 172.21.15.112:0/437656025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f88d80ffe60 0x7f88d8196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.694+0000 7f88ddd90640 1 -- 172.21.15.112:0/437656025 >> 172.21.15.112:0/437656025 conn(0x7f88d80fb420 msgr2=0x7f88d8104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.694+0000 7f88ddd90640 1 -- 172.21.15.112:0/437656025 shutdown_connections 2024-03-16T09:15:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.694+0000 7f88ddd90640 1 -- 172.21.15.112:0/437656025 wait complete. 2024-03-16T09:15:02.707 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:15:02.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.807+0000 7f15c3d3b640 1 -- 172.21.15.112:0/2269380557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15bc107560 msgr2=0x7f15bc1099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:02.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.807+0000 7f15c3d3b640 1 --2- 172.21.15.112:0/2269380557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15bc107560 0x7f15bc1099b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009ca0 tx=0x7f15b402c0f0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.807+0000 7f15c3d3b640 1 -- 172.21.15.112:0/2269380557 shutdown_connections 2024-03-16T09:15:02.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.807+0000 7f15c3d3b640 1 --2- 172.21.15.112:0/2269380557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15bc107560 0x7f15bc1099b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.807+0000 7f15c3d3b640 1 --2- 172.21.15.112:0/2269380557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15bc104c30 0x7f15bc107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.807+0000 7f15c3d3b640 1 --2- 172.21.15.112:0/2269380557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15bc102300 0x7f15bc1046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.807+0000 7f15c3d3b640 1 -- 172.21.15.112:0/2269380557 >> 172.21.15.112:0/2269380557 conn(0x7f15bc0fbd10 msgr2=0x7f15bc0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:02.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.807+0000 7f15c3d3b640 1 -- 172.21.15.112:0/2269380557 shutdown_connections 2024-03-16T09:15:02.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.807+0000 7f15c3d3b640 1 -- 172.21.15.112:0/2269380557 wait complete. 2024-03-16T09:15:02.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c3d3b640 1 Processor -- start 2024-03-16T09:15:02.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c3d3b640 1 -- start start 2024-03-16T09:15:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c3d3b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15bc102300 0x7f15bc197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c3d3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15bc104c30 0x7f15bc1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c3d3b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15bc107560 0x7f15bc19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c3d3b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc10d5e0 con 0x7f15bc104c30 2024-03-16T09:15:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c3d3b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc10d460 con 0x7f15bc102300 2024-03-16T09:15:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c3d3b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc10d760 con 0x7f15bc107560 2024-03-16T09:15:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c12d6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15bc104c30 0x7f15bc1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c12d6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15bc104c30 0x7f15bc1980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38136/0 (socket says 172.21.15.112:38136) 2024-03-16T09:15:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c12d6640 1 -- 172.21.15.112:0/2956342936 learned_addr learned my addr 172.21.15.112:0/2956342936 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c1ad7640 1 --2- 172.21.15.112:0/2956342936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15bc102300 0x7f15bc197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c22d8640 1 --2- 172.21.15.112:0/2956342936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15bc107560 0x7f15bc19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c12d6640 1 -- 172.21.15.112:0/2956342936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15bc107560 msgr2=0x7f15bc19f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c12d6640 1 --2- 172.21.15.112:0/2956342936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15bc107560 0x7f15bc19f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c12d6640 1 -- 172.21.15.112:0/2956342936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15bc102300 msgr2=0x7f15bc197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c12d6640 1 --2- 172.21.15.112:0/2956342936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15bc102300 0x7f15bc197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c12d6640 1 -- 172.21.15.112:0/2956342936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15bc19f790 con 0x7f15bc104c30 2024-03-16T09:15:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c12d6640 1 --2- 172.21.15.112:0/2956342936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15bc104c30 0x7f15bc1980b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00e9c0 tx=0x7f15ac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15b2ffd640 1 -- 172.21.15.112:0/2956342936 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac00cd80 con 0x7f15bc104c30 2024-03-16T09:15:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15b2ffd640 1 -- 172.21.15.112:0/2956342936 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f15ac005520 con 0x7f15bc104c30 2024-03-16T09:15:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c3d3b640 1 -- 172.21.15.112:0/2956342936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15bc19fa80 con 0x7f15bc104c30 2024-03-16T09:15:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15b2ffd640 1 -- 172.21.15.112:0/2956342936 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac010690 con 0x7f15bc104c30 2024-03-16T09:15:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.808+0000 7f15c3d3b640 1 -- 172.21.15.112:0/2956342936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15bc1a0020 con 0x7f15bc104c30 2024-03-16T09:15:02.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.809+0000 7f15c3d3b640 1 -- 172.21.15.112:0/2956342936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15bc0fd800 con 0x7f15bc104c30 2024-03-16T09:15:02.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.810+0000 7f15b2ffd640 1 -- 172.21.15.112:0/2956342936 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f15ac0046c0 con 0x7f15bc104c30 2024-03-16T09:15:02.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.811+0000 7f15b2ffd640 1 --2- 172.21.15.112:0/2956342936 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f15a80a2880 0x7f15a80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:02.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.811+0000 7f15c1ad7640 1 --2- 172.21.15.112:0/2956342936 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f15a80a2880 0x7f15a80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:02.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.811+0000 7f15b2ffd640 1 -- 172.21.15.112:0/2956342936 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f15ac0ced00 con 0x7f15bc104c30 2024-03-16T09:15:02.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.811+0000 7f15c1ad7640 1 --2- 172.21.15.112:0/2956342936 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f15a80a2880 0x7f15a80a4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f15b8009bd0 tx=0x7f15b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:02.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.815+0000 7f15b2ffd640 1 -- 172.21.15.112:0/2956342936 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f15ac09bdb0 con 0x7f15bc104c30 2024-03-16T09:15:02.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.997+0000 7f15c3d3b640 1 -- 172.21.15.112:0/2956342936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15bc062410 con 0x7f15bc104c30 2024-03-16T09:15:03.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:02.998+0000 7f15b2ffd640 1 -- 172.21.15.112:0/2956342936 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22725 (secure 0 0 0) 0x7f15ac0bb030 con 0x7f15bc104c30 2024-03-16T09:15:03.001 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:03.001 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":87,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:14:31.991998+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T09:13:35.040628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T09:13:35.689419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:15:03.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.000+0000 7f15c3d3b640 1 -- 172.21.15.112:0/2956342936 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f15a80a2880 msgr2=0x7f15a80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:03.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.000+0000 7f15c3d3b640 1 --2- 172.21.15.112:0/2956342936 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f15a80a2880 0x7f15a80a4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f15b8009bd0 tx=0x7f15b8005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:03.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.000+0000 7f15c3d3b640 1 -- 172.21.15.112:0/2956342936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15bc104c30 msgr2=0x7f15bc1980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:03.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.000+0000 7f15c3d3b640 1 --2- 172.21.15.112:0/2956342936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15bc104c30 0x7f15bc1980b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00e9c0 tx=0x7f15ac00ee80 comp rx=0 tx=0).stop 2024-03-16T09:15:03.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.001+0000 7f15c3d3b640 1 -- 172.21.15.112:0/2956342936 shutdown_connections 2024-03-16T09:15:03.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.001+0000 7f15c3d3b640 1 --2- 172.21.15.112:0/2956342936 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f15a80a2880 0x7f15a80a4d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.001+0000 7f15c3d3b640 1 --2- 172.21.15.112:0/2956342936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15bc107560 0x7f15bc19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.001+0000 7f15c3d3b640 1 --2- 172.21.15.112:0/2956342936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15bc104c30 0x7f15bc1980b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.001+0000 7f15c3d3b640 1 --2- 172.21.15.112:0/2956342936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15bc102300 0x7f15bc197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.001+0000 7f15c3d3b640 1 -- 172.21.15.112:0/2956342936 >> 172.21.15.112:0/2956342936 conn(0x7f15bc0fbd10 msgr2=0x7f15bc102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:03.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.001+0000 7f15c3d3b640 1 -- 172.21.15.112:0/2956342936 shutdown_connections 2024-03-16T09:15:03.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.001+0000 7f15c3d3b640 1 -- 172.21.15.112:0/2956342936 wait complete. 2024-03-16T09:15:03.014 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T09:08:57.135312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T09:13:35.040628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T09:13:35.689419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T09:15:03.014 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T09:15:03.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.117+0000 7efe34434640 1 -- 172.21.15.112:0/1931633577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe2c101330 msgr2=0x7efe2c108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:03.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.117+0000 7efe34434640 1 --2- 172.21.15.112:0/1931633577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe2c101330 0x7efe2c108bd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efe24009ca0 tx=0x7efe2402c0f0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.118+0000 7efe34434640 1 -- 172.21.15.112:0/1931633577 shutdown_connections 2024-03-16T09:15:03.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.118+0000 7efe34434640 1 --2- 172.21.15.112:0/1931633577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe2c101330 0x7efe2c108bd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.118+0000 7efe34434640 1 --2- 172.21.15.112:0/1931633577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe2c100970 0x7efe2c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.118+0000 7efe34434640 1 --2- 172.21.15.112:0/1931633577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe2c101d20 0x7efe2c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.118+0000 7efe34434640 1 -- 172.21.15.112:0/1931633577 >> 172.21.15.112:0/1931633577 conn(0x7efe2c0fb3e0 msgr2=0x7efe2c0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:03.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.118+0000 7efe34434640 1 -- 172.21.15.112:0/1931633577 shutdown_connections 2024-03-16T09:15:03.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.118+0000 7efe34434640 1 -- 172.21.15.112:0/1931633577 wait complete. 2024-03-16T09:15:03.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.118+0000 7efe34434640 1 Processor -- start 2024-03-16T09:15:03.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.118+0000 7efe34434640 1 -- start start 2024-03-16T09:15:03.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.118+0000 7efe34434640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe2c100970 0x7efe2c10b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:03.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.118+0000 7efe34434640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe2c101330 0x7efe2c109700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:03.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.118+0000 7efe34434640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe2c101d20 0x7efe2c109c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:03.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.118+0000 7efe34434640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c10d620 con 0x7efe2c101d20 2024-03-16T09:15:03.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.118+0000 7efe34434640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c10d4a0 con 0x7efe2c101330 2024-03-16T09:15:03.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.118+0000 7efe34434640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c10d7a0 con 0x7efe2c100970 2024-03-16T09:15:03.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.118+0000 7efe329d1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe2c101d20 0x7efe2c109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:03.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.118+0000 7efe329d1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe2c101d20 0x7efe2c109c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38158/0 (socket says 172.21.15.112:38158) 2024-03-16T09:15:03.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.118+0000 7efe329d1640 1 -- 172.21.15.112:0/3818650255 learned_addr learned my addr 172.21.15.112:0/3818650255 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:03.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.118+0000 7efe319cf640 1 --2- 172.21.15.112:0/3818650255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe2c101330 0x7efe2c109700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:03.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.119+0000 7efe321d0640 1 --2- 172.21.15.112:0/3818650255 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe2c100970 0x7efe2c10b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:03.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.119+0000 7efe329d1640 1 -- 172.21.15.112:0/3818650255 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe2c100970 msgr2=0x7efe2c10b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:03.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.119+0000 7efe329d1640 1 --2- 172.21.15.112:0/3818650255 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe2c100970 0x7efe2c10b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.119+0000 7efe329d1640 1 -- 172.21.15.112:0/3818650255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe2c101330 msgr2=0x7efe2c109700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:03.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.119+0000 7efe329d1640 1 --2- 172.21.15.112:0/3818650255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe2c101330 0x7efe2c109700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.119+0000 7efe329d1640 1 -- 172.21.15.112:0/3818650255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2c10a4d0 con 0x7efe2c101d20 2024-03-16T09:15:03.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.119+0000 7efe329d1640 1 --2- 172.21.15.112:0/3818650255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe2c101d20 0x7efe2c109c40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efe24037040 tx=0x7efe24035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:03.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.119+0000 7efe237fe640 1 -- 172.21.15.112:0/3818650255 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24039040 con 0x7efe2c101d20 2024-03-16T09:15:03.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.119+0000 7efe34434640 1 -- 172.21.15.112:0/3818650255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe2c1a1c50 con 0x7efe2c101d20 2024-03-16T09:15:03.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.119+0000 7efe237fe640 1 -- 172.21.15.112:0/3818650255 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efe24044070 con 0x7efe2c101d20 2024-03-16T09:15:03.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.119+0000 7efe237fe640 1 -- 172.21.15.112:0/3818650255 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24004430 con 0x7efe2c101d20 2024-03-16T09:15:03.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.119+0000 7efe34434640 1 -- 172.21.15.112:0/3818650255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe2c1a21a0 con 0x7efe2c101d20 2024-03-16T09:15:03.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.120+0000 7efe34434640 1 -- 172.21.15.112:0/3818650255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe2c1079e0 con 0x7efe2c101d20 2024-03-16T09:15:03.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.121+0000 7efe237fe640 1 -- 172.21.15.112:0/3818650255 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7efe240045d0 con 0x7efe2c101d20 2024-03-16T09:15:03.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.121+0000 7efe237fe640 1 --2- 172.21.15.112:0/3818650255 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efe180a2880 0x7efe180a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:03.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.122+0000 7efe321d0640 1 --2- 172.21.15.112:0/3818650255 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efe180a2880 0x7efe180a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:03.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.122+0000 7efe237fe640 1 -- 172.21.15.112:0/3818650255 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7efe240f10b0 con 0x7efe2c101d20 2024-03-16T09:15:03.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.122+0000 7efe321d0640 1 --2- 172.21.15.112:0/3818650255 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efe180a2880 0x7efe180a4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efe2c069bd0 tx=0x7efe28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:03.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.126+0000 7efe237fe640 1 -- 172.21.15.112:0/3818650255 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efe240be100 con 0x7efe2c101d20 2024-03-16T09:15:03.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:02 smithi159 ceph-mon[36739]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-16T09:15:03.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:02 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2943258554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:03.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:02 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3600016654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:03.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:02 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/437656025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:03.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.300+0000 7efe34434640 1 -- 172.21.15.112:0/3818650255 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efe2c0623e0 con 0x7efe2c101d20 2024-03-16T09:15:03.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.300+0000 7efe237fe640 1 -- 172.21.15.112:0/3818650255 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7efe240bda00 con 0x7efe2c101d20 2024-03-16T09:15:03.303 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-03-16T09:15:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.302+0000 7efe34434640 1 -- 172.21.15.112:0/3818650255 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efe180a2880 msgr2=0x7efe180a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.302+0000 7efe34434640 1 --2- 172.21.15.112:0/3818650255 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efe180a2880 0x7efe180a4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efe2c069bd0 tx=0x7efe28005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.303+0000 7efe34434640 1 -- 172.21.15.112:0/3818650255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe2c101d20 msgr2=0x7efe2c109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.303+0000 7efe34434640 1 --2- 172.21.15.112:0/3818650255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe2c101d20 0x7efe2c109c40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efe24037040 tx=0x7efe24035db0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.303+0000 7efe34434640 1 -- 172.21.15.112:0/3818650255 shutdown_connections 2024-03-16T09:15:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.303+0000 7efe34434640 1 --2- 172.21.15.112:0/3818650255 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efe180a2880 0x7efe180a4d40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.303+0000 7efe34434640 1 --2- 172.21.15.112:0/3818650255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efe2c101d20 0x7efe2c109c40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.303+0000 7efe34434640 1 --2- 172.21.15.112:0/3818650255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe2c101330 0x7efe2c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.303+0000 7efe34434640 1 --2- 172.21.15.112:0/3818650255 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efe2c100970 0x7efe2c10b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.303+0000 7efe34434640 1 -- 172.21.15.112:0/3818650255 >> 172.21.15.112:0/3818650255 conn(0x7efe2c0fb3e0 msgr2=0x7efe2c104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:03.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.303+0000 7efe34434640 1 -- 172.21.15.112:0/3818650255 shutdown_connections 2024-03-16T09:15:03.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.303+0000 7efe34434640 1 -- 172.21.15.112:0/3818650255 wait complete. 2024-03-16T09:15:03.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:02 smithi112 ceph-mon[32307]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-16T09:15:03.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:02 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2943258554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:03.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:02 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3600016654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:03.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:02 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/437656025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:03.316 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T09:15:03.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:02 smithi194 ceph-mon[36729]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-16T09:15:03.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:02 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2943258554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:03.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:02 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3600016654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:03.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:02 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/437656025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:03.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.418+0000 7f8e18913640 1 -- 172.21.15.112:0/137779380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e141018f0 msgr2=0x7f8e14101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:03.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.418+0000 7f8e18913640 1 --2- 172.21.15.112:0/137779380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e141018f0 0x7f8e14101d70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009900 tx=0x7f8dfc02bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:03.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.418+0000 7f8e18913640 1 -- 172.21.15.112:0/137779380 shutdown_connections 2024-03-16T09:15:03.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.418+0000 7f8e18913640 1 --2- 172.21.15.112:0/137779380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e141022b0 0x7f8e14108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.418+0000 7f8e18913640 1 --2- 172.21.15.112:0/137779380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e141018f0 0x7f8e14101d70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.418+0000 7f8e18913640 1 --2- 172.21.15.112:0/137779380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e14100700 0x7f8e14100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.418+0000 7f8e18913640 1 -- 172.21.15.112:0/137779380 >> 172.21.15.112:0/137779380 conn(0x7f8e140fbc90 msgr2=0x7f8e140fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.418+0000 7f8e18913640 1 -- 172.21.15.112:0/137779380 shutdown_connections 2024-03-16T09:15:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.418+0000 7f8e18913640 1 -- 172.21.15.112:0/137779380 wait complete. 2024-03-16T09:15:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.419+0000 7f8e18913640 1 Processor -- start 2024-03-16T09:15:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.419+0000 7f8e18913640 1 -- start start 2024-03-16T09:15:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.419+0000 7f8e18913640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e14100700 0x7f8e14197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.419+0000 7f8e18913640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e141018f0 0x7f8e14198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.419+0000 7f8e18913640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e141022b0 0x7f8e1419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.419+0000 7f8e18913640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1410c460 con 0x7f8e141022b0 2024-03-16T09:15:03.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.419+0000 7f8e18913640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1410c2e0 con 0x7f8e141018f0 2024-03-16T09:15:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.419+0000 7f8e18913640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1410c5e0 con 0x7f8e14100700 2024-03-16T09:15:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.419+0000 7f8e12d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e141022b0 0x7f8e1419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.419+0000 7f8e12d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e141022b0 0x7f8e1419f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38166/0 (socket says 172.21.15.112:38166) 2024-03-16T09:15:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.419+0000 7f8e12d9d640 1 -- 172.21.15.112:0/2803585598 learned_addr learned my addr 172.21.15.112:0/2803585598 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.419+0000 7f8e0bfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e141018f0 0x7f8e14198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.419+0000 7f8e1259c640 1 --2- 172.21.15.112:0/2803585598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e14100700 0x7f8e14197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.419+0000 7f8e12d9d640 1 -- 172.21.15.112:0/2803585598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e14100700 msgr2=0x7f8e14197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.419+0000 7f8e12d9d640 1 --2- 172.21.15.112:0/2803585598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e14100700 0x7f8e14197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.419+0000 7f8e12d9d640 1 -- 172.21.15.112:0/2803585598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e141018f0 msgr2=0x7f8e14198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.419+0000 7f8e12d9d640 1 --2- 172.21.15.112:0/2803585598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e141018f0 0x7f8e14198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.419+0000 7f8e12d9d640 1 -- 172.21.15.112:0/2803585598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1419f810 con 0x7f8e141022b0 2024-03-16T09:15:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.419+0000 7f8e0bfff640 1 --2- 172.21.15.112:0/2803585598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e141018f0 0x7f8e14198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:03.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.420+0000 7f8e12d9d640 1 --2- 172.21.15.112:0/2803585598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e141022b0 0x7f8e1419f1a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00efd0 tx=0x7f8e0c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:03.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.420+0000 7f8e0b7fe640 1 -- 172.21.15.112:0/2803585598 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c003b70 con 0x7f8e141022b0 2024-03-16T09:15:03.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.420+0000 7f8e18913640 1 -- 172.21.15.112:0/2803585598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e1419fb00 con 0x7f8e141022b0 2024-03-16T09:15:03.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.420+0000 7f8e0b7fe640 1 -- 172.21.15.112:0/2803585598 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8e0c003d10 con 0x7f8e141022b0 2024-03-16T09:15:03.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.420+0000 7f8e0b7fe640 1 -- 172.21.15.112:0/2803585598 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c010a90 con 0x7f8e141022b0 2024-03-16T09:15:03.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.420+0000 7f8e18913640 1 -- 172.21.15.112:0/2803585598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e141a0020 con 0x7f8e141022b0 2024-03-16T09:15:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.421+0000 7f8e18913640 1 -- 172.21.15.112:0/2803585598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e141a0250 con 0x7f8e141022b0 2024-03-16T09:15:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.422+0000 7f8e0b7fe640 1 -- 172.21.15.112:0/2803585598 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f8e0c004000 con 0x7f8e141022b0 2024-03-16T09:15:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.422+0000 7f8e0b7fe640 1 --2- 172.21.15.112:0/2803585598 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8e000a2800 0x7f8e000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.422+0000 7f8e1259c640 1 --2- 172.21.15.112:0/2803585598 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8e000a2800 0x7f8e000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.422+0000 7f8e0b7fe640 1 -- 172.21.15.112:0/2803585598 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8e0c0ceec0 con 0x7f8e141022b0 2024-03-16T09:15:03.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.423+0000 7f8e1259c640 1 --2- 172.21.15.112:0/2803585598 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8e000a2800 0x7f8e000a4cc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8e14101750 tx=0x7f8e04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:03.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.427+0000 7f8e0b7fe640 1 -- 172.21.15.112:0/2803585598 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8e0c09bf70 con 0x7f8e141022b0 2024-03-16T09:15:03.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.602+0000 7f8e18913640 1 -- 172.21.15.112:0/2803585598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8e141a0420 con 0x7f8e141022b0 2024-03-16T09:15:03.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.602+0000 7f8e0b7fe640 1 -- 172.21.15.112:0/2803585598 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f8e0c09b870 con 0x7f8e141022b0 2024-03-16T09:15:03.604 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-03-16T09:15:03.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.604+0000 7f8e18913640 1 -- 172.21.15.112:0/2803585598 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8e000a2800 msgr2=0x7f8e000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:03.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.604+0000 7f8e18913640 1 --2- 172.21.15.112:0/2803585598 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8e000a2800 0x7f8e000a4cc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8e14101750 tx=0x7f8e04005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:03.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.604+0000 7f8e18913640 1 -- 172.21.15.112:0/2803585598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e141022b0 msgr2=0x7f8e1419f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:03.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.604+0000 7f8e18913640 1 --2- 172.21.15.112:0/2803585598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e141022b0 0x7f8e1419f1a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00efd0 tx=0x7f8e0c00c6a0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.604+0000 7f8e18913640 1 -- 172.21.15.112:0/2803585598 shutdown_connections 2024-03-16T09:15:03.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.604+0000 7f8e18913640 1 --2- 172.21.15.112:0/2803585598 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8e000a2800 0x7f8e000a4cc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.605+0000 7f8e18913640 1 --2- 172.21.15.112:0/2803585598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e141022b0 0x7f8e1419f1a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.605+0000 7f8e18913640 1 --2- 172.21.15.112:0/2803585598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e141018f0 0x7f8e14198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.605+0000 7f8e18913640 1 --2- 172.21.15.112:0/2803585598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e14100700 0x7f8e14197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.605+0000 7f8e18913640 1 -- 172.21.15.112:0/2803585598 >> 172.21.15.112:0/2803585598 conn(0x7f8e140fbc90 msgr2=0x7f8e14069df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:03.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.605+0000 7f8e18913640 1 -- 172.21.15.112:0/2803585598 shutdown_connections 2024-03-16T09:15:03.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.605+0000 7f8e18913640 1 -- 172.21.15.112:0/2803585598 wait complete. 2024-03-16T09:15:03.618 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T09:15:03.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.720+0000 7f6f853c4640 1 -- 172.21.15.112:0/4239052664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f801033a0 msgr2=0x7f6f80109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:03.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.720+0000 7f6f853c4640 1 --2- 172.21.15.112:0/4239052664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f801033a0 0x7f6f80109c30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009900 tx=0x7f6f7802bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:03.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.720+0000 7f6f853c4640 1 -- 172.21.15.112:0/4239052664 shutdown_connections 2024-03-16T09:15:03.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.720+0000 7f6f853c4640 1 --2- 172.21.15.112:0/4239052664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f801033a0 0x7f6f80109c30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.720+0000 7f6f853c4640 1 --2- 172.21.15.112:0/4239052664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f801029e0 0x7f6f80102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.720+0000 7f6f853c4640 1 --2- 172.21.15.112:0/4239052664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f801017f0 0x7f6f80101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.720+0000 7f6f853c4640 1 -- 172.21.15.112:0/4239052664 >> 172.21.15.112:0/4239052664 conn(0x7f6f800fcd80 msgr2=0x7f6f800ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:03.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.720+0000 7f6f853c4640 1 -- 172.21.15.112:0/4239052664 shutdown_connections 2024-03-16T09:15:03.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.720+0000 7f6f853c4640 1 -- 172.21.15.112:0/4239052664 wait complete. 2024-03-16T09:15:03.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.721+0000 7f6f853c4640 1 Processor -- start 2024-03-16T09:15:03.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.721+0000 7f6f853c4640 1 -- start start 2024-03-16T09:15:03.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.721+0000 7f6f853c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f801017f0 0x7f6f80197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:03.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.721+0000 7f6f853c4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f801029e0 0x7f6f80198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:03.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.721+0000 7f6f853c4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f801033a0 0x7f6f8019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:03.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.721+0000 7f6f853c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8010d550 con 0x7f6f801017f0 2024-03-16T09:15:03.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.721+0000 7f6f853c4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8010d3d0 con 0x7f6f801033a0 2024-03-16T09:15:03.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.721+0000 7f6f853c4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8010d6d0 con 0x7f6f801029e0 2024-03-16T09:15:03.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.721+0000 7f6f7effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f801017f0 0x7f6f80197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.721+0000 7f6f7effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f801017f0 0x7f6f80197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38174/0 (socket says 172.21.15.112:38174) 2024-03-16T09:15:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.721+0000 7f6f7effd640 1 -- 172.21.15.112:0/1919310891 learned_addr learned my addr 172.21.15.112:0/1919310891 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.721+0000 7f6f7e7fc640 1 --2- 172.21.15.112:0/1919310891 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f801029e0 0x7f6f80198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.721+0000 7f6f7f7fe640 1 --2- 172.21.15.112:0/1919310891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f801033a0 0x7f6f8019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.721+0000 7f6f7effd640 1 -- 172.21.15.112:0/1919310891 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f801029e0 msgr2=0x7f6f80198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.721+0000 7f6f7effd640 1 --2- 172.21.15.112:0/1919310891 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f801029e0 0x7f6f80198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.721+0000 7f6f7effd640 1 -- 172.21.15.112:0/1919310891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f801033a0 msgr2=0x7f6f8019f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.721+0000 7f6f7effd640 1 --2- 172.21.15.112:0/1919310891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f801033a0 0x7f6f8019f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.721+0000 7f6f7effd640 1 -- 172.21.15.112:0/1919310891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8019f760 con 0x7f6f801017f0 2024-03-16T09:15:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.721+0000 7f6f7effd640 1 --2- 172.21.15.112:0/1919310891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f801017f0 0x7f6f80197b40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000ebc0 tx=0x7f6f7000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.722+0000 7f6f67fff640 1 -- 172.21.15.112:0/1919310891 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70003b70 con 0x7f6f801017f0 2024-03-16T09:15:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.722+0000 7f6f853c4640 1 -- 172.21.15.112:0/1919310891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8019fa50 con 0x7f6f801017f0 2024-03-16T09:15:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.722+0000 7f6f67fff640 1 -- 172.21.15.112:0/1919310891 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6f70003d10 con 0x7f6f801017f0 2024-03-16T09:15:03.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.722+0000 7f6f67fff640 1 -- 172.21.15.112:0/1919310891 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70019980 con 0x7f6f801017f0 2024-03-16T09:15:03.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.722+0000 7f6f853c4640 1 -- 172.21.15.112:0/1919310891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f8019fff0 con 0x7f6f801017f0 2024-03-16T09:15:03.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.723+0000 7f6f853c4640 1 -- 172.21.15.112:0/1919310891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f80108b10 con 0x7f6f801017f0 2024-03-16T09:15:03.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.724+0000 7f6f67fff640 1 -- 172.21.15.112:0/1919310891 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f6f70019b20 con 0x7f6f801017f0 2024-03-16T09:15:03.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.724+0000 7f6f67fff640 1 --2- 172.21.15.112:0/1919310891 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6f680a2800 0x7f6f680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:03.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.725+0000 7f6f7e7fc640 1 --2- 172.21.15.112:0/1919310891 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6f680a2800 0x7f6f680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:03.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.725+0000 7f6f67fff640 1 -- 172.21.15.112:0/1919310891 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f6f700cf2e0 con 0x7f6f801017f0 2024-03-16T09:15:03.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.725+0000 7f6f7e7fc640 1 --2- 172.21.15.112:0/1919310891 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6f680a2800 0x7f6f680a4cc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6f80199050 tx=0x7f6f74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:03.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.729+0000 7f6f67fff640 1 -- 172.21.15.112:0/1919310891 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6f7009c390 con 0x7f6f801017f0 2024-03-16T09:15:03.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.905+0000 7f6f853c4640 1 -- 172.21.15.112:0/1919310891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6f8002d1b0 con 0x7f6f801017f0 2024-03-16T09:15:03.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.906+0000 7f6f67fff640 1 -- 172.21.15.112:0/1919310891 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f6f8002d1b0 con 0x7f6f801017f0 2024-03-16T09:15:03.908 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-03-16T09:15:03.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.908+0000 7f6f853c4640 1 -- 172.21.15.112:0/1919310891 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6f680a2800 msgr2=0x7f6f680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:03.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.908+0000 7f6f853c4640 1 --2- 172.21.15.112:0/1919310891 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6f680a2800 0x7f6f680a4cc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6f80199050 tx=0x7f6f74005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:03.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.908+0000 7f6f853c4640 1 -- 172.21.15.112:0/1919310891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f801017f0 msgr2=0x7f6f80197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:03.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.908+0000 7f6f853c4640 1 --2- 172.21.15.112:0/1919310891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f801017f0 0x7f6f80197b40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000ebc0 tx=0x7f6f7000c6a0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.908+0000 7f6f853c4640 1 -- 172.21.15.112:0/1919310891 shutdown_connections 2024-03-16T09:15:03.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.908+0000 7f6f853c4640 1 --2- 172.21.15.112:0/1919310891 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6f680a2800 0x7f6f680a4cc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.908+0000 7f6f853c4640 1 --2- 172.21.15.112:0/1919310891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f801033a0 0x7f6f8019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.908+0000 7f6f853c4640 1 --2- 172.21.15.112:0/1919310891 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f801029e0 0x7f6f80198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.908+0000 7f6f853c4640 1 --2- 172.21.15.112:0/1919310891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f801017f0 0x7f6f80197b40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:03.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.908+0000 7f6f853c4640 1 -- 172.21.15.112:0/1919310891 >> 172.21.15.112:0/1919310891 conn(0x7f6f800fcd80 msgr2=0x7f6f80105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:03.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.908+0000 7f6f853c4640 1 -- 172.21.15.112:0/1919310891 shutdown_connections 2024-03-16T09:15:03.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:03.908+0000 7f6f853c4640 1 -- 172.21.15.112:0/1919310891 wait complete. 2024-03-16T09:15:03.921 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:15:04.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.023+0000 7fe471953640 1 -- 172.21.15.112:0/2460570530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe46c1022d0 msgr2=0x7fe46c108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:04.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.023+0000 7fe471953640 1 --2- 172.21.15.112:0/2460570530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe46c1022d0 0x7fe46c108b60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe464009ca0 tx=0x7fe46402c0f0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.023+0000 7fe471953640 1 -- 172.21.15.112:0/2460570530 shutdown_connections 2024-03-16T09:15:04.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.023+0000 7fe471953640 1 --2- 172.21.15.112:0/2460570530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe46c1022d0 0x7fe46c108b60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.023+0000 7fe471953640 1 --2- 172.21.15.112:0/2460570530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe46c101910 0x7fe46c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.023+0000 7fe471953640 1 --2- 172.21.15.112:0/2460570530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe46c100720 0x7fe46c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.023+0000 7fe471953640 1 -- 172.21.15.112:0/2460570530 >> 172.21.15.112:0/2460570530 conn(0x7fe46c0fbc90 msgr2=0x7fe46c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:04.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.023+0000 7fe471953640 1 -- 172.21.15.112:0/2460570530 shutdown_connections 2024-03-16T09:15:04.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.023+0000 7fe471953640 1 -- 172.21.15.112:0/2460570530 wait complete. 2024-03-16T09:15:04.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.023+0000 7fe471953640 1 Processor -- start 2024-03-16T09:15:04.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe471953640 1 -- start start 2024-03-16T09:15:04.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe471953640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe46c100720 0x7fe46c199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:04.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe471953640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe46c101910 0x7fe46c19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:04.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe471953640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe46c1022d0 0x7fe46c1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:04.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe471953640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c10c4d0 con 0x7fe46c100720 2024-03-16T09:15:04.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe471953640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c10c350 con 0x7fe46c1022d0 2024-03-16T09:15:04.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe471953640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe46c10c650 con 0x7fe46c101910 2024-03-16T09:15:04.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe46affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe46c100720 0x7fe46c199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:04.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe46affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe46c100720 0x7fe46c199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38190/0 (socket says 172.21.15.112:38190) 2024-03-16T09:15:04.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe46affd640 1 -- 172.21.15.112:0/3801419382 learned_addr learned my addr 172.21.15.112:0/3801419382 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:04.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe46a7fc640 1 --2- 172.21.15.112:0/3801419382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe46c101910 0x7fe46c19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:04.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe46b7fe640 1 --2- 172.21.15.112:0/3801419382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe46c1022d0 0x7fe46c1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:04.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe46affd640 1 -- 172.21.15.112:0/3801419382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe46c101910 msgr2=0x7fe46c19a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:04.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe46affd640 1 --2- 172.21.15.112:0/3801419382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe46c101910 0x7fe46c19a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe46affd640 1 -- 172.21.15.112:0/3801419382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe46c1022d0 msgr2=0x7fe46c1a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:04.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe46affd640 1 --2- 172.21.15.112:0/3801419382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe46c1022d0 0x7fe46c1a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe46affd640 1 -- 172.21.15.112:0/3801419382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46c1a1a30 con 0x7fe46c100720 2024-03-16T09:15:04.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe46affd640 1 --2- 172.21.15.112:0/3801419382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe46c100720 0x7fe46c199e10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00e9c0 tx=0x7fe45c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:04.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe470951640 1 -- 172.21.15.112:0/3801419382 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c00cd80 con 0x7fe46c100720 2024-03-16T09:15:04.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe471953640 1 -- 172.21.15.112:0/3801419382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe46c1a1d20 con 0x7fe46c100720 2024-03-16T09:15:04.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe470951640 1 -- 172.21.15.112:0/3801419382 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe45c0035f0 con 0x7fe46c100720 2024-03-16T09:15:04.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.024+0000 7fe470951640 1 -- 172.21.15.112:0/3801419382 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c010690 con 0x7fe46c100720 2024-03-16T09:15:04.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.025+0000 7fe471953640 1 -- 172.21.15.112:0/3801419382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe46c1a2200 con 0x7fe46c100720 2024-03-16T09:15:04.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.025+0000 7fe471953640 1 -- 172.21.15.112:0/3801419382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe46c1a2490 con 0x7fe46c100720 2024-03-16T09:15:04.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.027+0000 7fe470951640 1 -- 172.21.15.112:0/3801419382 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7fe45c0037a0 con 0x7fe46c100720 2024-03-16T09:15:04.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.027+0000 7fe470951640 1 --2- 172.21.15.112:0/3801419382 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe4540a2800 0x7fe4540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:04.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.027+0000 7fe46a7fc640 1 --2- 172.21.15.112:0/3801419382 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe4540a2800 0x7fe4540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:04.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.027+0000 7fe470951640 1 -- 172.21.15.112:0/3801419382 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fe45c0ceeb0 con 0x7fe46c100720 2024-03-16T09:15:04.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.027+0000 7fe46a7fc640 1 --2- 172.21.15.112:0/3801419382 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe4540a2800 0x7fe4540a4cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe46c19b320 tx=0x7fe460005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:04.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.031+0000 7fe470951640 1 -- 172.21.15.112:0/3801419382 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe45c09bff0 con 0x7fe46c100720 2024-03-16T09:15:04.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:03 smithi159 ceph-mon[36739]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-16T09:15:04.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:03 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2956342936' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:04.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:03 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3818650255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:04.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:03 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2803585598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:04.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:03 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1919310891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:04.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.246+0000 7fe471953640 1 -- 172.21.15.112:0/3801419382 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe46c1a2660 con 0x7fe46c100720 2024-03-16T09:15:04.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.246+0000 7fe470951640 1 -- 172.21.15.112:0/3801419382 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11881 (secure 0 0 0) 0x7fe45c09b8f0 con 0x7fe46c100720 2024-03-16T09:15:04.248 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:04.248 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3939748854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3939748854},{"type":"v1","addr":"172.21.15.112:6839","nonce":3939748854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3118523258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3118523258},{"type":"v1","addr":"172.21.15.112:6841","nonce":3118523258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T09:13:36.450216+0000","modified":"2024-03-16T09:13:40.527486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/1682076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":1682076},{"type":"v1","addr":"172.21.15.194:6837","nonce":1682076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.194:6839/2528943309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2528943309},{"type":"v1","addr":"172.21.15.194:6839","nonce":2528943309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/272895965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":272895965},{"type":"v1","addr":"172.21.15.159:6839","nonce":272895965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24452,24466,24494]},"id":1}]} 2024-03-16T09:15:04.249 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2024-03-16T09:15:04.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.248+0000 7fe471953640 1 -- 172.21.15.112:0/3801419382 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe4540a2800 msgr2=0x7fe4540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:04.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.248+0000 7fe471953640 1 --2- 172.21.15.112:0/3801419382 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe4540a2800 0x7fe4540a4cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe46c19b320 tx=0x7fe460005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:04.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.248+0000 7fe471953640 1 -- 172.21.15.112:0/3801419382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe46c100720 msgr2=0x7fe46c199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:04.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.248+0000 7fe471953640 1 --2- 172.21.15.112:0/3801419382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe46c100720 0x7fe46c199e10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00e9c0 tx=0x7fe45c00ee80 comp rx=0 tx=0).stop 2024-03-16T09:15:04.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.248+0000 7fe471953640 1 -- 172.21.15.112:0/3801419382 shutdown_connections 2024-03-16T09:15:04.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.248+0000 7fe471953640 1 --2- 172.21.15.112:0/3801419382 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe4540a2800 0x7fe4540a4cc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.248+0000 7fe471953640 1 --2- 172.21.15.112:0/3801419382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe46c1022d0 0x7fe46c1a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.248+0000 7fe471953640 1 --2- 172.21.15.112:0/3801419382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe46c101910 0x7fe46c19a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.248+0000 7fe471953640 1 --2- 172.21.15.112:0/3801419382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe46c100720 0x7fe46c199e10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.248+0000 7fe471953640 1 -- 172.21.15.112:0/3801419382 >> 172.21.15.112:0/3801419382 conn(0x7fe46c0fbc90 msgr2=0x7fe46c069ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:04.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.248+0000 7fe471953640 1 -- 172.21.15.112:0/3801419382 shutdown_connections 2024-03-16T09:15:04.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.249+0000 7fe471953640 1 -- 172.21.15.112:0/3801419382 wait complete. 2024-03-16T09:15:04.260 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:03 smithi112 ceph-mon[32307]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-16T09:15:04.260 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:03 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2956342936' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:04.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:03 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3818650255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:04.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:03 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2803585598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:04.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:03 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1919310891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:04.262 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:15:04.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.360+0000 7f6f0f597640 1 -- 172.21.15.112:0/2720696616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08102280 msgr2=0x7f6f08104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:04.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.360+0000 7f6f0f597640 1 --2- 172.21.15.112:0/2720696616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08102280 0x7f6f08104670 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009900 tx=0x7f6f0002bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:04.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.360+0000 7f6f0f597640 1 -- 172.21.15.112:0/2720696616 shutdown_connections 2024-03-16T09:15:04.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.360+0000 7f6f0f597640 1 --2- 172.21.15.112:0/2720696616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f081074e0 0x7f6f08109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.360+0000 7f6f0f597640 1 --2- 172.21.15.112:0/2720696616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f08104bb0 0x7f6f08106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.360+0000 7f6f0f597640 1 --2- 172.21.15.112:0/2720696616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08102280 0x7f6f08104670 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.360+0000 7f6f0f597640 1 -- 172.21.15.112:0/2720696616 >> 172.21.15.112:0/2720696616 conn(0x7f6f080fbc70 msgr2=0x7f6f080fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:04.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.360+0000 7f6f0f597640 1 -- 172.21.15.112:0/2720696616 shutdown_connections 2024-03-16T09:15:04.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.361+0000 7f6f0f597640 1 -- 172.21.15.112:0/2720696616 wait complete. 2024-03-16T09:15:04.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.361+0000 7f6f0f597640 1 Processor -- start 2024-03-16T09:15:04.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.361+0000 7f6f0f597640 1 -- start start 2024-03-16T09:15:04.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.361+0000 7f6f0f597640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f08102280 0x7f6f08197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:04.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.361+0000 7f6f0f597640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08104bb0 0x7f6f08198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:04.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.361+0000 7f6f0f597640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f081074e0 0x7f6f0819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:04.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.361+0000 7f6f0f597640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0810d580 con 0x7f6f08104bb0 2024-03-16T09:15:04.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.361+0000 7f6f0f597640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0810d400 con 0x7f6f081074e0 2024-03-16T09:15:04.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.361+0000 7f6f0f597640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0810d700 con 0x7f6f08102280 2024-03-16T09:15:04.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.361+0000 7f6f0cb32640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08104bb0 0x7f6f08198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:04.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.361+0000 7f6f0cb32640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08104bb0 0x7f6f08198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38212/0 (socket says 172.21.15.112:38212) 2024-03-16T09:15:04.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.361+0000 7f6f0cb32640 1 -- 172.21.15.112:0/3471205349 learned_addr learned my addr 172.21.15.112:0/3471205349 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:04.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.361+0000 7f6f0d333640 1 --2- 172.21.15.112:0/3471205349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f08102280 0x7f6f08197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:04.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.361+0000 7f6f0db34640 1 --2- 172.21.15.112:0/3471205349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f081074e0 0x7f6f0819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:04.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.362+0000 7f6f0cb32640 1 -- 172.21.15.112:0/3471205349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f08102280 msgr2=0x7f6f08197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:04.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.362+0000 7f6f0cb32640 1 --2- 172.21.15.112:0/3471205349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f08102280 0x7f6f08197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.362+0000 7f6f0cb32640 1 -- 172.21.15.112:0/3471205349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f081074e0 msgr2=0x7f6f0819f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:04.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.362+0000 7f6f0cb32640 1 --2- 172.21.15.112:0/3471205349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f081074e0 0x7f6f0819f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.362+0000 7f6f0cb32640 1 -- 172.21.15.112:0/3471205349 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0819f830 con 0x7f6f08104bb0 2024-03-16T09:15:04.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.362+0000 7f6f0cb32640 1 --2- 172.21.15.112:0/3471205349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08104bb0 0x7f6f08198160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800e9f0 tx=0x7f6ef800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:04.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.362+0000 7f6efe7fc640 1 -- 172.21.15.112:0/3471205349 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef800ccc0 con 0x7f6f08104bb0 2024-03-16T09:15:04.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.362+0000 7f6efe7fc640 1 -- 172.21.15.112:0/3471205349 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6ef8005520 con 0x7f6f08104bb0 2024-03-16T09:15:04.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.362+0000 7f6f0f597640 1 -- 172.21.15.112:0/3471205349 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0819fb20 con 0x7f6f08104bb0 2024-03-16T09:15:04.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.362+0000 7f6efe7fc640 1 -- 172.21.15.112:0/3471205349 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8010690 con 0x7f6f08104bb0 2024-03-16T09:15:04.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.362+0000 7f6f0f597640 1 -- 172.21.15.112:0/3471205349 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f081a0020 con 0x7f6f08104bb0 2024-03-16T09:15:04.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.363+0000 7f6f0f597640 1 -- 172.21.15.112:0/3471205349 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f080fd760 con 0x7f6f08104bb0 2024-03-16T09:15:04.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.364+0000 7f6efe7fc640 1 -- 172.21.15.112:0/3471205349 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f6ef8004f70 con 0x7f6f08104bb0 2024-03-16T09:15:04.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.364+0000 7f6efe7fc640 1 --2- 172.21.15.112:0/3471205349 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ef40a2800 0x7f6ef40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:04.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.364+0000 7f6f0d333640 1 --2- 172.21.15.112:0/3471205349 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ef40a2800 0x7f6ef40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:04.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.364+0000 7f6efe7fc640 1 -- 172.21.15.112:0/3471205349 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f6ef80cee80 con 0x7f6f08104bb0 2024-03-16T09:15:04.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.365+0000 7f6f0d333640 1 --2- 172.21.15.112:0/3471205349 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ef40a2800 0x7f6ef40a4cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6f081990f0 tx=0x7f6f00005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:04.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.369+0000 7f6efe7fc640 1 -- 172.21.15.112:0/3471205349 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6ef809bf30 con 0x7f6f08104bb0 2024-03-16T09:15:04.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:03 smithi194 ceph-mon[36729]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-16T09:15:04.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:03 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2956342936' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:04.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:03 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3818650255' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:04.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:03 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2803585598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:04.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:03 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1919310891' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:04.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.543+0000 7f6f0f597640 1 -- 172.21.15.112:0/3471205349 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f080623e0 con 0x7f6f08104bb0 2024-03-16T09:15:04.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.545+0000 7f6efe7fc640 1 -- 172.21.15.112:0/3471205349 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22725 (secure 0 0 0) 0x7f6ef809b830 con 0x7f6f08104bb0 2024-03-16T09:15:04.547 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:04.547 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":87,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:14:31.991998+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T09:13:35.040628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T09:13:35.689419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:15:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.547+0000 7f6f0f597640 1 -- 172.21.15.112:0/3471205349 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ef40a2800 msgr2=0x7f6ef40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.547+0000 7f6f0f597640 1 --2- 172.21.15.112:0/3471205349 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ef40a2800 0x7f6ef40a4cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6f081990f0 tx=0x7f6f00005fd0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.547+0000 7f6f0f597640 1 -- 172.21.15.112:0/3471205349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08104bb0 msgr2=0x7f6f08198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.547+0000 7f6f0f597640 1 --2- 172.21.15.112:0/3471205349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08104bb0 0x7f6f08198160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800e9f0 tx=0x7f6ef800eeb0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.547+0000 7f6f0f597640 1 -- 172.21.15.112:0/3471205349 shutdown_connections 2024-03-16T09:15:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.547+0000 7f6f0f597640 1 --2- 172.21.15.112:0/3471205349 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ef40a2800 0x7f6ef40a4cc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.547+0000 7f6f0f597640 1 --2- 172.21.15.112:0/3471205349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f081074e0 0x7f6f0819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.547+0000 7f6f0f597640 1 --2- 172.21.15.112:0/3471205349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f08104bb0 0x7f6f08198160 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.547+0000 7f6f0f597640 1 --2- 172.21.15.112:0/3471205349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6f08102280 0x7f6f08197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.547+0000 7f6f0f597640 1 -- 172.21.15.112:0/3471205349 >> 172.21.15.112:0/3471205349 conn(0x7f6f080fbc70 msgr2=0x7f6f08102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.547+0000 7f6f0f597640 1 -- 172.21.15.112:0/3471205349 shutdown_connections 2024-03-16T09:15:04.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.547+0000 7f6f0f597640 1 -- 172.21.15.112:0/3471205349 wait complete. 2024-03-16T09:15:04.560 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:15:04.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.661+0000 7fb378af7640 1 -- 172.21.15.112:0/383273202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb374100700 msgr2=0x7fb374100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:04.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.661+0000 7fb378af7640 1 --2- 172.21.15.112:0/383273202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb374100700 0x7fb374100b20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb364009900 tx=0x7fb36402bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:04.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.661+0000 7fb378af7640 1 -- 172.21.15.112:0/383273202 shutdown_connections 2024-03-16T09:15:04.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.661+0000 7fb378af7640 1 --2- 172.21.15.112:0/383273202 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3741022b0 0x7fb374108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.661+0000 7fb378af7640 1 --2- 172.21.15.112:0/383273202 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3741018f0 0x7fb374101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.661+0000 7fb378af7640 1 --2- 172.21.15.112:0/383273202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb374100700 0x7fb374100b20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.661+0000 7fb378af7640 1 -- 172.21.15.112:0/383273202 >> 172.21.15.112:0/383273202 conn(0x7fb3740fbc70 msgr2=0x7fb3740fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:04.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.661+0000 7fb378af7640 1 -- 172.21.15.112:0/383273202 shutdown_connections 2024-03-16T09:15:04.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.661+0000 7fb378af7640 1 -- 172.21.15.112:0/383273202 wait complete. 2024-03-16T09:15:04.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb378af7640 1 Processor -- start 2024-03-16T09:15:04.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb378af7640 1 -- start start 2024-03-16T09:15:04.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb378af7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb374100700 0x7fb3740694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:04.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb378af7640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3741018f0 0x7fb37419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:04.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb378af7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3741022b0 0x7fb37419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:04.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb378af7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb37410c460 con 0x7fb3741022b0 2024-03-16T09:15:04.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb378af7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb37410c2e0 con 0x7fb374100700 2024-03-16T09:15:04.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb378af7640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb37410c5e0 con 0x7fb3741018f0 2024-03-16T09:15:04.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb372d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3741022b0 0x7fb37419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:04.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb372d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3741022b0 0x7fb37419ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38218/0 (socket says 172.21.15.112:38218) 2024-03-16T09:15:04.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb372d9d640 1 -- 172.21.15.112:0/1615552221 learned_addr learned my addr 172.21.15.112:0/1615552221 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:04.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb37259c640 1 --2- 172.21.15.112:0/1615552221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb374100700 0x7fb3740694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:04.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb371d9b640 1 --2- 172.21.15.112:0/1615552221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3741018f0 0x7fb37419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:04.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb372d9d640 1 -- 172.21.15.112:0/1615552221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3741018f0 msgr2=0x7fb37419c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:04.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb372d9d640 1 --2- 172.21.15.112:0/1615552221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3741018f0 0x7fb37419c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb372d9d640 1 -- 172.21.15.112:0/1615552221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb374100700 msgr2=0x7fb3740694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:04.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb372d9d640 1 --2- 172.21.15.112:0/1615552221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb374100700 0x7fb3740694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb372d9d640 1 -- 172.21.15.112:0/1615552221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37419f830 con 0x7fb3741022b0 2024-03-16T09:15:04.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb37259c640 1 --2- 172.21.15.112:0/1615552221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb374100700 0x7fb3740694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:04.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb372d9d640 1 --2- 172.21.15.112:0/1615552221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3741022b0 0x7fb37419ef70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb36c00eb10 tx=0x7fb36c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:04.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb3637fe640 1 -- 172.21.15.112:0/1615552221 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c004b20 con 0x7fb3741022b0 2024-03-16T09:15:04.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb3637fe640 1 -- 172.21.15.112:0/1615552221 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb36c004cc0 con 0x7fb3741022b0 2024-03-16T09:15:04.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb378af7640 1 -- 172.21.15.112:0/1615552221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37419fb20 con 0x7fb3741022b0 2024-03-16T09:15:04.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb3637fe640 1 -- 172.21.15.112:0/1615552221 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c0107f0 con 0x7fb3741022b0 2024-03-16T09:15:04.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.662+0000 7fb378af7640 1 -- 172.21.15.112:0/1615552221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3741a0020 con 0x7fb3741022b0 2024-03-16T09:15:04.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.663+0000 7fb378af7640 1 -- 172.21.15.112:0/1615552221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb374107a20 con 0x7fb3741022b0 2024-03-16T09:15:04.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.664+0000 7fb3637fe640 1 -- 172.21.15.112:0/1615552221 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7fb36c005020 con 0x7fb3741022b0 2024-03-16T09:15:04.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.665+0000 7fb3637fe640 1 --2- 172.21.15.112:0/1615552221 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb35c0a2800 0x7fb35c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:04.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.665+0000 7fb37259c640 1 --2- 172.21.15.112:0/1615552221 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb35c0a2800 0x7fb35c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:04.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.665+0000 7fb3637fe640 1 -- 172.21.15.112:0/1615552221 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb36c0cf000 con 0x7fb3741022b0 2024-03-16T09:15:04.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.665+0000 7fb37259c640 1 --2- 172.21.15.112:0/1615552221 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb35c0a2800 0x7fb35c0a4cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb364000c00 tx=0x7fb364005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:04.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.669+0000 7fb3637fe640 1 -- 172.21.15.112:0/1615552221 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb36c09c0b0 con 0x7fb3741022b0 2024-03-16T09:15:04.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.883+0000 7fb378af7640 1 -- 172.21.15.112:0/1615552221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb37402d1b0 con 0x7fb3741022b0 2024-03-16T09:15:04.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.884+0000 7fb3637fe640 1 -- 172.21.15.112:0/1615552221 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11881 (secure 0 0 0) 0x7fb36c09b9b0 con 0x7fb3741022b0 2024-03-16T09:15:04.886 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:04.886 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3939748854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3939748854},{"type":"v1","addr":"172.21.15.112:6839","nonce":3939748854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3118523258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3118523258},{"type":"v1","addr":"172.21.15.112:6841","nonce":3118523258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T09:13:36.450216+0000","modified":"2024-03-16T09:13:40.527486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/1682076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":1682076},{"type":"v1","addr":"172.21.15.194:6837","nonce":1682076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.194:6839/2528943309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2528943309},{"type":"v1","addr":"172.21.15.194:6839","nonce":2528943309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/272895965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":272895965},{"type":"v1","addr":"172.21.15.159:6839","nonce":272895965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24452,24466,24494]},"id":1}]} 2024-03-16T09:15:04.886 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2024-03-16T09:15:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.886+0000 7fb378af7640 1 -- 172.21.15.112:0/1615552221 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb35c0a2800 msgr2=0x7fb35c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.886+0000 7fb378af7640 1 --2- 172.21.15.112:0/1615552221 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb35c0a2800 0x7fb35c0a4cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb364000c00 tx=0x7fb364005fd0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.886+0000 7fb378af7640 1 -- 172.21.15.112:0/1615552221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3741022b0 msgr2=0x7fb37419ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:04.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.886+0000 7fb378af7640 1 --2- 172.21.15.112:0/1615552221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3741022b0 0x7fb37419ef70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb36c00eb10 tx=0x7fb36c00c2d0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.886+0000 7fb378af7640 1 -- 172.21.15.112:0/1615552221 shutdown_connections 2024-03-16T09:15:04.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.886+0000 7fb378af7640 1 --2- 172.21.15.112:0/1615552221 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb35c0a2800 0x7fb35c0a4cc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.886+0000 7fb378af7640 1 --2- 172.21.15.112:0/1615552221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3741022b0 0x7fb37419ef70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.886+0000 7fb378af7640 1 --2- 172.21.15.112:0/1615552221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3741018f0 0x7fb37419c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.886+0000 7fb378af7640 1 --2- 172.21.15.112:0/1615552221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb374100700 0x7fb3740694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:04.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.886+0000 7fb378af7640 1 -- 172.21.15.112:0/1615552221 >> 172.21.15.112:0/1615552221 conn(0x7fb3740fbc70 msgr2=0x7fb374104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:04.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.886+0000 7fb378af7640 1 -- 172.21.15.112:0/1615552221 shutdown_connections 2024-03-16T09:15:04.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:04.886+0000 7fb378af7640 1 -- 172.21.15.112:0/1615552221 wait complete. 2024-03-16T09:15:04.899 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-16T09:13:36.450216+0000', 'modified': '2024-03-16T09:13:40.527486+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24494, 'mds_1': 24466, 'mds_2': 24452}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24413': {'gid': 24413, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6835/567707106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 567707106}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 567707106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6835/2682964784', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6834', 'nonce': 2682964784}, {'type': 'v1', 'addr': '172.21.15.194:6835', 'nonce': 2682964784}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6837/1682076', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6836', 'nonce': 1682076}, {'type': 'v1', 'addr': '172.21.15.194:6837', 'nonce': 1682076}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.159:6837/4023942203', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 4023942203}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 4023942203}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.194:6839/2528943309', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6838', 'nonce': 2528943309}, {'type': 'v1', 'addr': '172.21.15.194:6839', 'nonce': 2528943309}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.159:6839/272895965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 272895965}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 272895965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24452, 24466, 24494]} 2024-03-16T09:15:04.899 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-16T09:15:04.899 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-16T09:15:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.000+0000 7fdc92034640 1 -- 172.21.15.112:0/1165287162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c1018e0 msgr2=0x7fdc8c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.000+0000 7fdc92034640 1 --2- 172.21.15.112:0/1165287162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c1018e0 0x7fdc8c101d60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009900 tx=0x7fdc7402bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.001+0000 7fdc92034640 1 -- 172.21.15.112:0/1165287162 shutdown_connections 2024-03-16T09:15:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.001+0000 7fdc92034640 1 --2- 172.21.15.112:0/1165287162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc8c1022a0 0x7fdc8c108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.001+0000 7fdc92034640 1 --2- 172.21.15.112:0/1165287162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c1018e0 0x7fdc8c101d60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.001+0000 7fdc92034640 1 --2- 172.21.15.112:0/1165287162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc8c1006f0 0x7fdc8c100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.001+0000 7fdc92034640 1 -- 172.21.15.112:0/1165287162 >> 172.21.15.112:0/1165287162 conn(0x7fdc8c0fbc80 msgr2=0x7fdc8c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.001+0000 7fdc92034640 1 -- 172.21.15.112:0/1165287162 shutdown_connections 2024-03-16T09:15:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.001+0000 7fdc92034640 1 -- 172.21.15.112:0/1165287162 wait complete. 2024-03-16T09:15:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.001+0000 7fdc92034640 1 Processor -- start 2024-03-16T09:15:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.001+0000 7fdc92034640 1 -- start start 2024-03-16T09:15:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.001+0000 7fdc92034640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc8c1006f0 0x7fdc8c06a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.001+0000 7fdc92034640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c1018e0 0x7fdc8c19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.001+0000 7fdc92034640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc8c1022a0 0x7fdc8c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.001+0000 7fdc92034640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c10c450 con 0x7fdc8c1018e0 2024-03-16T09:15:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.001+0000 7fdc92034640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c10c2d0 con 0x7fdc8c1022a0 2024-03-16T09:15:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.001+0000 7fdc92034640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c10c5d0 con 0x7fdc8c1006f0 2024-03-16T09:15:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.002+0000 7fdc8affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c1018e0 0x7fdc8c19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.002+0000 7fdc8affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c1018e0 0x7fdc8c19c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38240/0 (socket says 172.21.15.112:38240) 2024-03-16T09:15:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.002+0000 7fdc8b7fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc8c1006f0 0x7fdc8c06a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.002+0000 7fdc8affd640 1 -- 172.21.15.112:0/2011532170 learned_addr learned my addr 172.21.15.112:0/2011532170 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.002+0000 7fdc8bfff640 1 --2- 172.21.15.112:0/2011532170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc8c1022a0 0x7fdc8c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.002+0000 7fdc8affd640 1 -- 172.21.15.112:0/2011532170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc8c1006f0 msgr2=0x7fdc8c06a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.002+0000 7fdc8affd640 1 --2- 172.21.15.112:0/2011532170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc8c1006f0 0x7fdc8c06a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.002+0000 7fdc8affd640 1 -- 172.21.15.112:0/2011532170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc8c1022a0 msgr2=0x7fdc8c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.002+0000 7fdc8affd640 1 --2- 172.21.15.112:0/2011532170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc8c1022a0 0x7fdc8c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.002+0000 7fdc8affd640 1 -- 172.21.15.112:0/2011532170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8c19f830 con 0x7fdc8c1018e0 2024-03-16T09:15:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.002+0000 7fdc8b7fe640 1 --2- 172.21.15.112:0/2011532170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc8c1006f0 0x7fdc8c06a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.002+0000 7fdc8bfff640 1 --2- 172.21.15.112:0/2011532170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc8c1022a0 0x7fdc8c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.002+0000 7fdc8affd640 1 --2- 172.21.15.112:0/2011532170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c1018e0 0x7fdc8c19c930 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009230 tx=0x7fdc74004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.002+0000 7fdc88ff9640 1 -- 172.21.15.112:0/2011532170 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74043020 con 0x7fdc8c1018e0 2024-03-16T09:15:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.002+0000 7fdc92034640 1 -- 172.21.15.112:0/2011532170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8c19fac0 con 0x7fdc8c1018e0 2024-03-16T09:15:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.002+0000 7fdc88ff9640 1 -- 172.21.15.112:0/2011532170 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdc74004250 con 0x7fdc8c1018e0 2024-03-16T09:15:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.002+0000 7fdc88ff9640 1 -- 172.21.15.112:0/2011532170 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74003b50 con 0x7fdc8c1018e0 2024-03-16T09:15:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.002+0000 7fdc92034640 1 -- 172.21.15.112:0/2011532170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc8c1a0060 con 0x7fdc8c1018e0 2024-03-16T09:15:05.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.004+0000 7fdc88ff9640 1 -- 172.21.15.112:0/2011532170 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7fdc74046020 con 0x7fdc8c1018e0 2024-03-16T09:15:05.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc88ff9640 1 --2- 172.21.15.112:0/2011532170 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdc780a2800 0x7fdc780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc94c1b740 1 Processor -- start 2024-03-16T09:15:05.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc94c1b740 1 -- start start 2024-03-16T09:15:05.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc8b7fe640 1 --2- 172.21.15.112:0/2011532170 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdc780a2800 0x7fdc780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc88ff9640 1 -- 172.21.15.112:0/2011532170 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdc740f0ae0 con 0x7fdc8c1018e0 2024-03-16T09:15:05.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc94c1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112f4fe0 0x5625112f5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc94c1b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112f59d0 0x5625112fa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc94c1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112fab40 0x5625112fcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc94c1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562511254c80 con 0x5625112fab40 2024-03-16T09:15:05.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc94c1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5625111fdaa0 con 0x5625112f4fe0 2024-03-16T09:15:05.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc94c1b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562511250d20 con 0x5625112f59d0 2024-03-16T09:15:05.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc8bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112fab40 0x5625112fcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc8bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112fab40 0x5625112fcfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38250/0 (socket says 172.21.15.112:38250) 2024-03-16T09:15:05.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc8bfff640 1 -- 172.21.15.112:0/4212070377 learned_addr learned my addr 172.21.15.112:0/4212070377 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:05.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc8affd640 1 --2- 172.21.15.112:0/4212070377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112f59d0 0x5625112fa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc8b7fe640 1 --2- 172.21.15.112:0/4212070377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112f4fe0 0x5625112f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc8b7fe640 1 --2- 172.21.15.112:0/2011532170 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdc780a2800 0x7fdc780a4cc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c101740 tx=0x7fdc7c00fb20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:05.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc8bfff640 1 -- 172.21.15.112:0/4212070377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112f59d0 msgr2=0x5625112fa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc8bfff640 1 --2- 172.21.15.112:0/4212070377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112f59d0 0x5625112fa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc8bfff640 1 -- 172.21.15.112:0/4212070377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112f4fe0 msgr2=0x5625112f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc8bfff640 1 --2- 172.21.15.112:0/4212070377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112f4fe0 0x5625112f5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc8bfff640 1 -- 172.21.15.112:0/4212070377 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56251121fff0 con 0x5625112fab40 2024-03-16T09:15:05.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc8affd640 1 --2- 172.21.15.112:0/4212070377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112f59d0 0x5625112fa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:05.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.005+0000 7fdc8b7fe640 1 --2- 172.21.15.112:0/4212070377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112f4fe0 0x5625112f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:05.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.006+0000 7fdc8bfff640 1 --2- 172.21.15.112:0/4212070377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112fab40 0x5625112fcfd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdc84011540 tx=0x7fdc84011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2ab32908078d7c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:05.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.006+0000 7fdc81ffb640 1 -- 172.21.15.112:0/4212070377 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8400f5c0 con 0x5625112fab40 2024-03-16T09:15:05.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.006+0000 7fdc81ffb640 1 -- 172.21.15.112:0/4212070377 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdc8400fc60 con 0x5625112fab40 2024-03-16T09:15:05.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.006+0000 7fdc81ffb640 1 -- 172.21.15.112:0/4212070377 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc84019d60 con 0x5625112fab40 2024-03-16T09:15:05.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.006+0000 7fdc94c1b740 1 -- 172.21.15.112:0/4212070377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112fab40 msgr2=0x5625112fcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.006+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/4212070377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112fab40 0x5625112fcfd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdc84011540 tx=0x7fdc84011a00 comp rx=0 tx=0).stop 2024-03-16T09:15:05.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.006+0000 7fdc94c1b740 1 -- 172.21.15.112:0/4212070377 shutdown_connections 2024-03-16T09:15:05.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.006+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/4212070377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112fab40 0x5625112fcfd0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.006+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/4212070377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112f59d0 0x5625112fa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.006+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/4212070377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112f4fe0 0x5625112f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.006+0000 7fdc94c1b740 1 -- 172.21.15.112:0/4212070377 >> 172.21.15.112:0/4212070377 conn(0x5625112ec140 msgr2=0x5625112ec540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:05.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.006+0000 7fdc94c1b740 1 -- 172.21.15.112:0/4212070377 shutdown_connections 2024-03-16T09:15:05.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.006+0000 7fdc94c1b740 1 -- 172.21.15.112:0/4212070377 wait complete. 2024-03-16T09:15:05.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.007+0000 7fdc94c1b740 1 -- start start 2024-03-16T09:15:05.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.007+0000 7fdc94c1b740 1 Processor -- start 2024-03-16T09:15:05.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.007+0000 7fdc94c1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f4fe0 0x5625112ff160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.007+0000 7fdc94c1b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112f59d0 0x5625112ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.007+0000 7fdc94c1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112fab40 0x5625113066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.007+0000 7fdc94c1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5625112fdaf0 con 0x5625112f4fe0 2024-03-16T09:15:05.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.007+0000 7fdc94c1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562511210ef0 con 0x5625112fab40 2024-03-16T09:15:05.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.007+0000 7fdc94c1b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5625112fdc70 con 0x5625112f59d0 2024-03-16T09:15:05.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.007+0000 7fdc94c1b740 10 client.0 _ll_register_callbacks cb 0x56251120bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:15:05.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.007+0000 7fdc8b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f4fe0 0x5625112ff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.007+0000 7fdc8b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f4fe0 0x5625112ff160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38266/0 (socket says 172.21.15.112:38266) 2024-03-16T09:15:05.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.007+0000 7fdc8b7fe640 1 -- 172.21.15.112:0/2644744049 learned_addr learned my addr 172.21.15.112:0/2644744049 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:05.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.007+0000 7fdc8bfff640 1 --2- 172.21.15.112:0/2644744049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112fab40 0x5625113066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.007+0000 7fdc8affd640 1 --2- 172.21.15.112:0/2644744049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112f59d0 0x5625112ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc8b7fe640 1 -- 172.21.15.112:0/2644744049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112f59d0 msgr2=0x5625112ff6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc8b7fe640 1 --2- 172.21.15.112:0/2644744049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112f59d0 0x5625112ff6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc8b7fe640 1 -- 172.21.15.112:0/2644744049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112fab40 msgr2=0x5625113066e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc8b7fe640 1 --2- 172.21.15.112:0/2644744049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112fab40 0x5625113066e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc8b7fe640 1 -- 172.21.15.112:0/2644744049 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56251121da30 con 0x5625112f4fe0 2024-03-16T09:15:05.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc8bfff640 1 --2- 172.21.15.112:0/2644744049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112fab40 0x5625113066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:05.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc8affd640 1 --2- 172.21.15.112:0/2644744049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112f59d0 0x5625112ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:05.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc8b7fe640 1 --2- 172.21.15.112:0/2644744049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f4fe0 0x5625112ff160 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c003e40 tx=0x7fdc7c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b90226a794732ae server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:05.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc6affd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:15:05.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc6affd640 1 -- 172.21.15.112:0/2644744049 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c017040 con 0x5625112f4fe0 2024-03-16T09:15:05.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2644744049 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc8400fc60 con 0x5625112f4fe0 2024-03-16T09:15:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc6affd640 1 -- 172.21.15.112:0/2644744049 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdc7c004d20 con 0x5625112f4fe0 2024-03-16T09:15:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc6affd640 1 -- 172.21.15.112:0/2644744049 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdc7c004810 con 0x5625112f4fe0 2024-03-16T09:15:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc94c1b740 10 client.15036 fetch_fsmap learned FSMap version 21 2024-03-16T09:15:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2644744049 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562510f692d0 con 0x5625112f4fe0 2024-03-16T09:15:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc6affd640 1 -- 172.21.15.112:0/2644744049 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdc8400fc60 con 0x5625112f4fe0 2024-03-16T09:15:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc6affd640 1 -- 172.21.15.112:0/2644744049 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdc7c025030 con 0x5625112f4fe0 2024-03-16T09:15:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc94c1b740 10 client.15036 fetch_fsmap finished waiting for FSMap version 21 2024-03-16T09:15:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc94c1b740 10 client.15036 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-16T09:15:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc94c1b740 20 client.15036 populate_metadata read hostname 'smithi112' 2024-03-16T09:15:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/2644744049 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x56251130c110 0x56251130d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc94c1b740 4 client.15036 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:15:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.008+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2644744049 --> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625112621c0 con 0x56251130c110 2024-03-16T09:15:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.009+0000 7fdc8affd640 1 --2- 172.21.15.112:0/2644744049 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x56251130c110 0x56251130d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.009+0000 7fdc8affd640 1 --2- 172.21.15.112:0/2644744049 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x56251130c110 0x56251130d580 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b3af79d23c12b3da server_cookie=bfa4e62a4775198e in_seq=0 out_seq=0 2024-03-16T09:15:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.009+0000 7fdc6affd640 10 client.15036 ms_handle_connect on v2:172.21.15.159:6836/4023942203 2024-03-16T09:15:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.010+0000 7fdc6affd640 1 -- 172.21.15.112:0/2644744049 <== mds.2 v2:172.21.15.159:6836/4023942203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5625112621c0 con 0x56251130c110 2024-03-16T09:15:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.010+0000 7fdc6affd640 10 client.15036 handle_command_reply: tid=0 2024-03-16T09:15:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.010+0000 7fdc94c1b740 1 client.15036 shutdown 2024-03-16T09:15:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.010+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2644744049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f4fe0 msgr2=0x5625112ff160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.010+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/2644744049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f4fe0 0x5625112ff160 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c003e40 tx=0x7fdc7c004ab0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.010+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2644744049 shutdown_connections 2024-03-16T09:15:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.010+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/2644744049 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x56251130c110 0x56251130d580 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.010+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/2644744049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112fab40 0x5625113066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.010+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/2644744049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112f59d0 0x5625112ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.010+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/2644744049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f4fe0 0x5625112ff160 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.010+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2644744049 >> 172.21.15.112:0/2644744049 conn(0x5625112ec140 msgr2=0x5625112eb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.010+0000 7fdc6affd640 0 client.15036 ms_handle_reset on v2:172.21.15.159:6836/4023942203 2024-03-16T09:15:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.010+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2644744049 shutdown_connections 2024-03-16T09:15:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.010+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2644744049 wait complete. 2024-03-16T09:15:05.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.010+0000 7fdc94c1b740 20 client.15036 trim_cache size 0 max 16384 2024-03-16T09:15:05.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc94c1b740 1 Processor -- start 2024-03-16T09:15:05.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc94c1b740 1 -- start start 2024-03-16T09:15:05.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc94c1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112f4fe0 0x56251130aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc94c1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f59d0 0x5625113066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc94c1b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112fab40 0x56251130af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc94c1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56251130b8c0 con 0x5625112f59d0 2024-03-16T09:15:05.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc94c1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56251130bf90 con 0x5625112f4fe0 2024-03-16T09:15:05.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc94c1b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56251130c660 con 0x5625112fab40 2024-03-16T09:15:05.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc8affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f59d0 0x5625113066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc8affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f59d0 0x5625113066a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38278/0 (socket says 172.21.15.112:38278) 2024-03-16T09:15:05.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc8affd640 1 -- 172.21.15.112:0/2632352 learned_addr learned my addr 172.21.15.112:0/2632352 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:05.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc8b7fe640 1 --2- 172.21.15.112:0/2632352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112f4fe0 0x56251130aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc8bfff640 1 --2- 172.21.15.112:0/2632352 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112fab40 0x56251130af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc8affd640 1 -- 172.21.15.112:0/2632352 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112fab40 msgr2=0x56251130af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc8affd640 1 --2- 172.21.15.112:0/2632352 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112fab40 0x56251130af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc8affd640 1 -- 172.21.15.112:0/2632352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112f4fe0 msgr2=0x56251130aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc8affd640 1 --2- 172.21.15.112:0/2632352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112f4fe0 0x56251130aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc8affd640 1 -- 172.21.15.112:0/2632352 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625110336b0 con 0x5625112f59d0 2024-03-16T09:15:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc8b7fe640 1 --2- 172.21.15.112:0/2632352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112f4fe0 0x56251130aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc8affd640 1 --2- 172.21.15.112:0/2632352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f59d0 0x5625113066a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdc740b98f0 tx=0x7fdc740f0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8736da9027b533dd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc817fa640 1 -- 172.21.15.112:0/2632352 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc740c5c30 con 0x5625112f59d0 2024-03-16T09:15:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc817fa640 1 -- 172.21.15.112:0/2632352 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdc740bb530 con 0x5625112f59d0 2024-03-16T09:15:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.028+0000 7fdc817fa640 1 -- 172.21.15.112:0/2632352 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc740c57a0 con 0x5625112f59d0 2024-03-16T09:15:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.029+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2632352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f59d0 msgr2=0x5625113066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.029+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/2632352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f59d0 0x5625113066a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdc740b98f0 tx=0x7fdc740f0040 comp rx=0 tx=0).stop 2024-03-16T09:15:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.029+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2632352 shutdown_connections 2024-03-16T09:15:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.029+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/2632352 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112fab40 0x56251130af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.029+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/2632352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f59d0 0x5625113066a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.029+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/2632352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112f4fe0 0x56251130aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.029+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2632352 >> 172.21.15.112:0/2632352 conn(0x5625112ec140 msgr2=0x5625112fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:05.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.029+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2632352 shutdown_connections 2024-03-16T09:15:05.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.029+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2632352 wait complete. 2024-03-16T09:15:05.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.029+0000 7fdc94c1b740 1 -- start start 2024-03-16T09:15:05.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc94c1b740 1 Processor -- start 2024-03-16T09:15:05.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc94c1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f4fe0 0x5625112f9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc94c1b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112f59d0 0x5625113066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc94c1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112fab40 0x5625112f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc94c1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56251130d400 con 0x5625112f4fe0 2024-03-16T09:15:05.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc94c1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56251130cd30 con 0x5625112fab40 2024-03-16T09:15:05.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc94c1b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56251130dad0 con 0x5625112f59d0 2024-03-16T09:15:05.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc94c1b740 10 client.0 _ll_register_callbacks cb 0x7fdc7c011a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:15:05.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc8b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f4fe0 0x5625112f9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc8b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f4fe0 0x5625112f9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38292/0 (socket says 172.21.15.112:38292) 2024-03-16T09:15:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc8b7fe640 1 -- 172.21.15.112:0/2916758578 learned_addr learned my addr 172.21.15.112:0/2916758578 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc8affd640 1 --2- 172.21.15.112:0/2916758578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112f59d0 0x5625113066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc8bfff640 1 --2- 172.21.15.112:0/2916758578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112fab40 0x5625112f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc8b7fe640 1 -- 172.21.15.112:0/2916758578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112f59d0 msgr2=0x5625113066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc8b7fe640 1 --2- 172.21.15.112:0/2916758578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112f59d0 0x5625113066a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc8b7fe640 1 -- 172.21.15.112:0/2916758578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112fab40 msgr2=0x5625112f7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc8b7fe640 1 --2- 172.21.15.112:0/2916758578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112fab40 0x5625112f7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc8b7fe640 1 -- 172.21.15.112:0/2916758578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625112f8230 con 0x5625112f4fe0 2024-03-16T09:15:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc8b7fe640 1 --2- 172.21.15.112:0/2916758578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f4fe0 0x5625112f9340 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c013400 tx=0x7fdc7c00fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d198f4c25b79af35 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc6affd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:15:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc6affd640 1 -- 172.21.15.112:0/2916758578 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c017040 con 0x5625112f4fe0 2024-03-16T09:15:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc6affd640 1 -- 172.21.15.112:0/2916758578 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdc7c01d640 con 0x5625112f4fe0 2024-03-16T09:15:05.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.030+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2916758578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc740bb530 con 0x5625112f4fe0 2024-03-16T09:15:05.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.031+0000 7fdc6affd640 1 -- 172.21.15.112:0/2916758578 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdc7c003c80 con 0x5625112f4fe0 2024-03-16T09:15:05.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.031+0000 7fdc94c1b740 10 client.15042 fetch_fsmap learned FSMap version 21 2024-03-16T09:15:05.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.031+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2916758578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5625113708a0 con 0x5625112f4fe0 2024-03-16T09:15:05.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.031+0000 7fdc6affd640 1 -- 172.21.15.112:0/2916758578 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdc740bb530 con 0x5625112f4fe0 2024-03-16T09:15:05.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.031+0000 7fdc6affd640 1 -- 172.21.15.112:0/2916758578 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fdc7c025040 con 0x5625112f4fe0 2024-03-16T09:15:05.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.031+0000 7fdc94c1b740 10 client.15042 fetch_fsmap finished waiting for FSMap version 21 2024-03-16T09:15:05.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.031+0000 7fdc94c1b740 10 client.15042 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-16T09:15:05.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.031+0000 7fdc94c1b740 20 client.15042 populate_metadata read hostname 'smithi112' 2024-03-16T09:15:05.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.031+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/2916758578 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x562511375530 0x562511371970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.031+0000 7fdc94c1b740 4 client.15042 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T09:15:05.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.031+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2916758578 --> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] -- command(tid 0: {"prefix": "status"}) -- 0x56251125c510 con 0x562511375530 2024-03-16T09:15:05.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.031+0000 7fdc8affd640 1 --2- 172.21.15.112:0/2916758578 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x562511375530 0x562511371970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.031+0000 7fdc8affd640 1 --2- 172.21.15.112:0/2916758578 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x562511375530 0x562511371970 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d1eaa8a3c23a210 server_cookie=c97a88937152544b in_seq=0 out_seq=0 2024-03-16T09:15:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.031+0000 7fdc6affd640 10 client.15042 ms_handle_connect on v2:172.21.15.159:6836/4023942203 2024-03-16T09:15:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc6affd640 1 -- 172.21.15.112:0/2916758578 <== mds.2 v2:172.21.15.159:6836/4023942203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56251125c510 con 0x562511375530 2024-03-16T09:15:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc6affd640 10 client.15042 handle_command_reply: tid=0 2024-03-16T09:15:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc94c1b740 1 client.15042 shutdown 2024-03-16T09:15:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2916758578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f4fe0 msgr2=0x5625112f9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/2916758578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f4fe0 0x5625112f9340 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c013400 tx=0x7fdc7c00fed0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2916758578 shutdown_connections 2024-03-16T09:15:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/2916758578 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x562511375530 0x562511371970 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/2916758578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625112fab40 0x5625112f7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc6affd640 0 client.15042 ms_handle_reset on v2:172.21.15.159:6836/4023942203 2024-03-16T09:15:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/2916758578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5625112f59d0 0x5625113066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc94c1b740 1 --2- 172.21.15.112:0/2916758578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5625112f4fe0 0x5625112f9340 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2916758578 >> 172.21.15.112:0/2916758578 conn(0x5625112ec140 msgr2=0x5625112fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:05.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2916758578 shutdown_connections 2024-03-16T09:15:05.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc94c1b740 1 -- 172.21.15.112:0/2916758578 wait complete. 2024-03-16T09:15:05.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc94c1b740 20 client.15042 trim_cache size 0 max 16384 2024-03-16T09:15:05.042 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:15:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "273be2a4-e374-11ee-95c9-87774f69a715", 2024-03-16T09:15:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 2, 2024-03-16T09:15:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24452, 2024-03-16T09:15:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-03-16T09:15:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-03-16T09:15:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-03-16T09:15:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 84.901904380999994, 2024-03-16T09:15:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 21, 2024-03-16T09:15:05.043 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 87, 2024-03-16T09:15:05.043 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 85, 2024-03-16T09:15:05.043 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 105.227652857 2024-03-16T09:15:05.043 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:15:05.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc92034640 1 -- 172.21.15.112:0/2011532170 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdc780a2800 msgr2=0x7fdc780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc92034640 1 --2- 172.21.15.112:0/2011532170 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdc780a2800 0x7fdc780a4cc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c101740 tx=0x7fdc7c00fb20 comp rx=0 tx=0).stop 2024-03-16T09:15:05.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc92034640 1 -- 172.21.15.112:0/2011532170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c1018e0 msgr2=0x7fdc8c19c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc92034640 1 --2- 172.21.15.112:0/2011532170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c1018e0 0x7fdc8c19c930 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009230 tx=0x7fdc74004ab0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc92034640 1 -- 172.21.15.112:0/2011532170 shutdown_connections 2024-03-16T09:15:05.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc92034640 1 --2- 172.21.15.112:0/2011532170 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdc780a2800 0x7fdc780a4cc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.032+0000 7fdc92034640 1 --2- 172.21.15.112:0/2011532170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc8c1022a0 0x7fdc8c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.033+0000 7fdc92034640 1 --2- 172.21.15.112:0/2011532170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc8c1018e0 0x7fdc8c19c930 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.033+0000 7fdc92034640 1 --2- 172.21.15.112:0/2011532170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdc8c1006f0 0x7fdc8c06a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.033+0000 7fdc92034640 1 -- 172.21.15.112:0/2011532170 >> 172.21.15.112:0/2011532170 conn(0x7fdc8c0fbc80 msgr2=0x7fdc8c104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:05.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.033+0000 7fdc92034640 1 -- 172.21.15.112:0/2011532170 shutdown_connections 2024-03-16T09:15:05.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.033+0000 7fdc92034640 1 -- 172.21.15.112:0/2011532170 wait complete. 2024-03-16T09:15:05.046 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-16T09:15:05.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.149+0000 7f43e711d640 1 -- 172.21.15.112:0/1905660676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e01022a0 msgr2=0x7f43e0108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.149+0000 7f43e711d640 1 --2- 172.21.15.112:0/1905660676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e01022a0 0x7f43e0108b30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009ca0 tx=0x7f43d802c0f0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.149+0000 7f43e711d640 1 -- 172.21.15.112:0/1905660676 shutdown_connections 2024-03-16T09:15:05.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.149+0000 7f43e711d640 1 --2- 172.21.15.112:0/1905660676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e01022a0 0x7f43e0108b30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.149+0000 7f43e711d640 1 --2- 172.21.15.112:0/1905660676 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43e01018e0 0x7f43e0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.149+0000 7f43e711d640 1 --2- 172.21.15.112:0/1905660676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43e01006f0 0x7f43e0100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.149+0000 7f43e711d640 1 -- 172.21.15.112:0/1905660676 >> 172.21.15.112:0/1905660676 conn(0x7f43e00fbc80 msgr2=0x7f43e00fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:05.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.149+0000 7f43e711d640 1 -- 172.21.15.112:0/1905660676 shutdown_connections 2024-03-16T09:15:05.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.149+0000 7f43e711d640 1 -- 172.21.15.112:0/1905660676 wait complete. 2024-03-16T09:15:05.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.150+0000 7f43e711d640 1 Processor -- start 2024-03-16T09:15:05.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.150+0000 7f43e711d640 1 -- start start 2024-03-16T09:15:05.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.150+0000 7f43e711d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43e01006f0 0x7f43e006a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.150+0000 7f43e711d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e01018e0 0x7f43e019c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.150+0000 7f43e711d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43e01022a0 0x7f43e019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.150+0000 7f43e711d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f43e010c4a0 con 0x7f43e01018e0 2024-03-16T09:15:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.150+0000 7f43e711d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f43e010c320 con 0x7f43e01006f0 2024-03-16T09:15:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.150+0000 7f43e711d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f43e010c620 con 0x7f43e01022a0 2024-03-16T09:15:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.150+0000 7f43dffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e01018e0 0x7f43e019c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.150+0000 7f43dffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e01018e0 0x7f43e019c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38322/0 (socket says 172.21.15.112:38322) 2024-03-16T09:15:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.150+0000 7f43dffff640 1 -- 172.21.15.112:0/3376218352 learned_addr learned my addr 172.21.15.112:0/3376218352 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.150+0000 7f43e4eb9640 1 --2- 172.21.15.112:0/3376218352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43e01006f0 0x7f43e006a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.150+0000 7f43e56ba640 1 --2- 172.21.15.112:0/3376218352 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43e01022a0 0x7f43e019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.150+0000 7f43dffff640 1 -- 172.21.15.112:0/3376218352 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43e01022a0 msgr2=0x7f43e019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.150+0000 7f43dffff640 1 --2- 172.21.15.112:0/3376218352 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43e01022a0 0x7f43e019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.150+0000 7f43dffff640 1 -- 172.21.15.112:0/3376218352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43e01006f0 msgr2=0x7f43e006a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.150+0000 7f43dffff640 1 --2- 172.21.15.112:0/3376218352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43e01006f0 0x7f43e006a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.150+0000 7f43dffff640 1 -- 172.21.15.112:0/3376218352 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e019f7c0 con 0x7f43e01018e0 2024-03-16T09:15:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.151+0000 7f43dffff640 1 --2- 172.21.15.112:0/3376218352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e01018e0 0x7f43e019c8c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f43d400ea60 tx=0x7f43d400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.151+0000 7f43ddffb640 1 -- 172.21.15.112:0/3376218352 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d400cde0 con 0x7f43e01018e0 2024-03-16T09:15:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.151+0000 7f43ddffb640 1 -- 172.21.15.112:0/3376218352 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f43d40035f0 con 0x7f43e01018e0 2024-03-16T09:15:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.151+0000 7f43e711d640 1 -- 172.21.15.112:0/3376218352 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e019fab0 con 0x7f43e01018e0 2024-03-16T09:15:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.151+0000 7f43ddffb640 1 -- 172.21.15.112:0/3376218352 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4010720 con 0x7f43e01018e0 2024-03-16T09:15:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.151+0000 7f43e711d640 1 -- 172.21.15.112:0/3376218352 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e01a0000 con 0x7f43e01018e0 2024-03-16T09:15:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.153+0000 7f43ddffb640 1 -- 172.21.15.112:0/3376218352 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f43d40037a0 con 0x7f43e01018e0 2024-03-16T09:15:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.154+0000 7f43e9ddf740 1 Processor -- start 2024-03-16T09:15:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.154+0000 7f43e9ddf740 1 -- start start 2024-03-16T09:15:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.154+0000 7f43ddffb640 1 --2- 172.21.15.112:0/3376218352 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f43c80a2800 0x7f43c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.154+0000 7f43e9ddf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a98bd0 0x55e113a98ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.154+0000 7f43e9ddf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a995c0 0x55e113a9e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.154+0000 7f43e9ddf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a9e730 0x55e113aa0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.154+0000 7f43e9ddf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e1139fcc80 con 0x55e113a9e730 2024-03-16T09:15:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.154+0000 7f43e9ddf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e1139a5aa0 con 0x55e113a98bd0 2024-03-16T09:15:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.154+0000 7f43e9ddf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e1139f8d20 con 0x55e113a995c0 2024-03-16T09:15:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.154+0000 7f43e56ba640 1 --2- 172.21.15.112:0/3376218352 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f43c80a2800 0x7f43c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.154+0000 7f43e4eb9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a98bd0 0x55e113a98ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.154+0000 7f43dffff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a995c0 0x55e113a9e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.154+0000 7f43e4eb9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a98bd0 0x55e113a98ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:38876/0 (socket says 172.21.15.112:38876) 2024-03-16T09:15:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.154+0000 7f43e4eb9640 1 -- 172.21.15.112:0/1495866764 learned_addr learned my addr 172.21.15.112:0/1495866764 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.154+0000 7f43e56ba640 1 --2- 172.21.15.112:0/1495866764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a9e730 0x55e113aa0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.154+0000 7f43ddffb640 1 -- 172.21.15.112:0/3376218352 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f43d40ced70 con 0x7f43e01018e0 2024-03-16T09:15:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.154+0000 7f43e4eb9640 1 -- 172.21.15.112:0/1495866764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a995c0 msgr2=0x55e113a9e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.154+0000 7f43e4eb9640 1 --2- 172.21.15.112:0/1495866764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a995c0 0x55e113a9e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.155+0000 7f43e4eb9640 1 -- 172.21.15.112:0/1495866764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a9e730 msgr2=0x55e113aa0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.155+0000 7f43e4eb9640 1 --2- 172.21.15.112:0/1495866764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a9e730 0x55e113aa0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.155+0000 7f43e4eb9640 1 -- 172.21.15.112:0/1495866764 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1139c7ff0 con 0x55e113a98bd0 2024-03-16T09:15:05.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.155+0000 7f43dffff640 1 --2- 172.21.15.112:0/1495866764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a995c0 0x55e113a9e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:05.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.155+0000 7f43e56ba640 1 --2- 172.21.15.112:0/3376218352 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f43c80a2800 0x7f43c80a4cc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f43d8005e30 tx=0x7f43d8032450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:05.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.155+0000 7f43e4eb9640 1 --2- 172.21.15.112:0/1495866764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a98bd0 0x55e113a98ff0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009c80 tx=0x7f43d0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c88e59acd121552 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:05.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.155+0000 7f43c6ffd640 1 -- 172.21.15.112:0/1495866764 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d001e070 con 0x55e113a98bd0 2024-03-16T09:15:05.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.155+0000 7f43c6ffd640 1 -- 172.21.15.112:0/1495866764 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f43d0009e30 con 0x55e113a98bd0 2024-03-16T09:15:05.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.155+0000 7f43c6ffd640 1 -- 172.21.15.112:0/1495866764 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d000eab0 con 0x55e113a98bd0 2024-03-16T09:15:05.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.156+0000 7f43e9ddf740 1 -- 172.21.15.112:0/1495866764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a98bd0 msgr2=0x55e113a98ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.156+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/1495866764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a98bd0 0x55e113a98ff0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009c80 tx=0x7f43d0011810 comp rx=0 tx=0).stop 2024-03-16T09:15:05.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.156+0000 7f43e9ddf740 1 -- 172.21.15.112:0/1495866764 shutdown_connections 2024-03-16T09:15:05.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.156+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/1495866764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a9e730 0x55e113aa0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.156+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/1495866764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a995c0 0x55e113a9e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.156+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/1495866764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a98bd0 0x55e113a98ff0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.156+0000 7f43e9ddf740 1 -- 172.21.15.112:0/1495866764 >> 172.21.15.112:0/1495866764 conn(0x55e113a94140 msgr2=0x55e113a94540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.156+0000 7f43e9ddf740 1 -- 172.21.15.112:0/1495866764 shutdown_connections 2024-03-16T09:15:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.156+0000 7f43e9ddf740 1 -- 172.21.15.112:0/1495866764 wait complete. 2024-03-16T09:15:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.156+0000 7f43e9ddf740 1 -- start start 2024-03-16T09:15:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.156+0000 7f43e9ddf740 1 Processor -- start 2024-03-16T09:15:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.156+0000 7f43e9ddf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a98bd0 0x55e113aa2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.156+0000 7f43e9ddf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a995c0 0x55e113aa32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.156+0000 7f43e9ddf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a9e730 0x55e113aaa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.156+0000 7f43e9ddf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e113aa16e0 con 0x55e113a9e730 2024-03-16T09:15:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.156+0000 7f43e9ddf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e1139b8ef0 con 0x55e113a995c0 2024-03-16T09:15:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.156+0000 7f43e9ddf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e113aa1860 con 0x55e113a98bd0 2024-03-16T09:15:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43e9ddf740 10 client.0 _ll_register_callbacks cb 0x55e1139b3dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:15:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43dffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a995c0 0x55e113aa32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43e56ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a9e730 0x55e113aaa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43dffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a995c0 0x55e113aa32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:38886/0 (socket says 172.21.15.112:38886) 2024-03-16T09:15:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43dffff640 1 -- 172.21.15.112:0/3994736354 learned_addr learned my addr 172.21.15.112:0/3994736354 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43e4eb9640 1 --2- 172.21.15.112:0/3994736354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a98bd0 0x55e113aa2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43dffff640 1 -- 172.21.15.112:0/3994736354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a98bd0 msgr2=0x55e113aa2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43dffff640 1 --2- 172.21.15.112:0/3994736354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a98bd0 0x55e113aa2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43dffff640 1 -- 172.21.15.112:0/3994736354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a9e730 msgr2=0x55e113aaa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43dffff640 1 --2- 172.21.15.112:0/3994736354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a9e730 0x55e113aaa2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43dffff640 1 -- 172.21.15.112:0/3994736354 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1139c5a30 con 0x55e113a995c0 2024-03-16T09:15:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43e56ba640 1 --2- 172.21.15.112:0/3994736354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a9e730 0x55e113aaa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43dffff640 1 --2- 172.21.15.112:0/3994736354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a995c0 0x55e113aa32b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f43d400c760 tx=0x7f43d4015fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dab9dba2552c9f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:05.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43bffff640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-16T09:15:05.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43bffff640 1 -- 172.21.15.112:0/3994736354 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d40ae300 con 0x55e113a995c0 2024-03-16T09:15:05.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43e9ddf740 1 -- 172.21.15.112:0/3994736354 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43d0009e30 con 0x55e113a995c0 2024-03-16T09:15:05.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43bffff640 1 -- 172.21.15.112:0/3994736354 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f43d4086070 con 0x55e113a995c0 2024-03-16T09:15:05.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43bffff640 1 -- 172.21.15.112:0/3994736354 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f43d40ae120 con 0x55e113a995c0 2024-03-16T09:15:05.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43e9ddf740 10 client.24911 fetch_fsmap learned FSMap version 21 2024-03-16T09:15:05.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.157+0000 7f43e9ddf740 1 -- 172.21.15.112:0/3994736354 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e1137112d0 con 0x55e113a995c0 2024-03-16T09:15:05.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.158+0000 7f43bffff640 1 -- 172.21.15.112:0/3994736354 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f43d0009e30 con 0x55e113a995c0 2024-03-16T09:15:05.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.158+0000 7f43bffff640 1 -- 172.21.15.112:0/3994736354 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f43d4097d90 con 0x55e113a995c0 2024-03-16T09:15:05.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.158+0000 7f43e9ddf740 10 client.24911 fetch_fsmap finished waiting for FSMap version 21 2024-03-16T09:15:05.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.158+0000 7f43e9ddf740 10 client.24911 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-16T09:15:05.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.158+0000 7f43e9ddf740 20 client.24911 populate_metadata read hostname 'smithi112' 2024-03-16T09:15:05.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.158+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/3994736354 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55e113aaf7b0 0x55e113ab0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.158+0000 7f43e9ddf740 4 client.24911 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:15:05.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.158+0000 7f43e9ddf740 1 -- 172.21.15.112:0/3994736354 --> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e113a0a1c0 con 0x55e113aaf7b0 2024-03-16T09:15:05.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.158+0000 7f43e4eb9640 1 --2- 172.21.15.112:0/3994736354 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55e113aaf7b0 0x55e113ab0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.158+0000 7f43bffff640 10 client.24911 ms_handle_connect on v2:172.21.15.194:6838/2528943309 2024-03-16T09:15:05.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.158+0000 7f43e4eb9640 1 --2- 172.21.15.112:0/3994736354 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55e113aaf7b0 0x55e113ab0c20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c06bfb70e0e495e2 server_cookie=a86d5177204c2b5a in_seq=0 out_seq=0 2024-03-16T09:15:05.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.159+0000 7f43bffff640 1 -- 172.21.15.112:0/3994736354 <== mds.1 v2:172.21.15.194:6838/2528943309 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e113a0a1c0 con 0x55e113aaf7b0 2024-03-16T09:15:05.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.159+0000 7f43bffff640 10 client.24911 handle_command_reply: tid=0 2024-03-16T09:15:05.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.159+0000 7f43e9ddf740 1 client.24911 shutdown 2024-03-16T09:15:05.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.159+0000 7f43e9ddf740 1 -- 172.21.15.112:0/3994736354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a995c0 msgr2=0x55e113aa32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.159+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/3994736354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a995c0 0x55e113aa32b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f43d400c760 tx=0x7f43d4015fc0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.159+0000 7f43e9ddf740 1 -- 172.21.15.112:0/3994736354 shutdown_connections 2024-03-16T09:15:05.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.159+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/3994736354 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55e113aaf7b0 0x55e113ab0c20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.159+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/3994736354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a9e730 0x55e113aaa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.159+0000 7f43bffff640 0 client.24911 ms_handle_reset on v2:172.21.15.194:6838/2528943309 2024-03-16T09:15:05.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.159+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/3994736354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a995c0 0x55e113aa32b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.159+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/3994736354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a98bd0 0x55e113aa2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.159+0000 7f43e9ddf740 1 -- 172.21.15.112:0/3994736354 >> 172.21.15.112:0/3994736354 conn(0x55e113a94140 msgr2=0x55e113a93380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:05.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.159+0000 7f43e9ddf740 1 -- 172.21.15.112:0/3994736354 shutdown_connections 2024-03-16T09:15:05.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.159+0000 7f43e9ddf740 1 -- 172.21.15.112:0/3994736354 wait complete. 2024-03-16T09:15:05.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.160+0000 7f43e9ddf740 20 client.24911 trim_cache size 0 max 16384 2024-03-16T09:15:05.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.177+0000 7f43e9ddf740 1 Processor -- start 2024-03-16T09:15:05.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43e9ddf740 1 -- start start 2024-03-16T09:15:05.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43e9ddf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a98bd0 0x55e113a9dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43e9ddf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a995c0 0x55e113aaa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43e9ddf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a9e730 0x55e113a97ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43e9ddf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e113aaef60 con 0x55e113a9e730 2024-03-16T09:15:05.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43e9ddf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e113aaf630 con 0x55e113a98bd0 2024-03-16T09:15:05.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43e9ddf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e113aafd00 con 0x55e113a995c0 2024-03-16T09:15:05.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43e56ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a9e730 0x55e113a97ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43e56ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a9e730 0x55e113a97ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38362/0 (socket says 172.21.15.112:38362) 2024-03-16T09:15:05.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43e56ba640 1 -- 172.21.15.112:0/1152605525 learned_addr learned my addr 172.21.15.112:0/1152605525 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:05.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43e4eb9640 1 --2- 172.21.15.112:0/1152605525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a98bd0 0x55e113a9dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43dffff640 1 --2- 172.21.15.112:0/1152605525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a995c0 0x55e113aaa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43e56ba640 1 -- 172.21.15.112:0/1152605525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a995c0 msgr2=0x55e113aaa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43e56ba640 1 --2- 172.21.15.112:0/1152605525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a995c0 0x55e113aaa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43e56ba640 1 -- 172.21.15.112:0/1152605525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a98bd0 msgr2=0x55e113a9dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43e56ba640 1 --2- 172.21.15.112:0/1152605525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a98bd0 0x55e113a9dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43e56ba640 1 -- 172.21.15.112:0/1152605525 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1137db6b0 con 0x55e113a9e730 2024-03-16T09:15:05.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43e56ba640 1 --2- 172.21.15.112:0/1152605525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a9e730 0x55e113a97ab0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f43d8036000 tx=0x7f43d8047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2d8fed21d2e221 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:05.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43c67fc640 1 -- 172.21.15.112:0/1152605525 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d8039040 con 0x55e113a9e730 2024-03-16T09:15:05.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43c67fc640 1 -- 172.21.15.112:0/1152605525 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f43d8052070 con 0x55e113a9e730 2024-03-16T09:15:05.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.178+0000 7f43c67fc640 1 -- 172.21.15.112:0/1152605525 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d80052a0 con 0x55e113a9e730 2024-03-16T09:15:05.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.179+0000 7f43e9ddf740 1 -- 172.21.15.112:0/1152605525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a9e730 msgr2=0x55e113a97ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.179+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/1152605525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a9e730 0x55e113a97ab0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f43d8036000 tx=0x7f43d8047000 comp rx=0 tx=0).stop 2024-03-16T09:15:05.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.179+0000 7f43e9ddf740 1 -- 172.21.15.112:0/1152605525 shutdown_connections 2024-03-16T09:15:05.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.179+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/1152605525 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a9e730 0x55e113a97ab0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.179+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/1152605525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a995c0 0x55e113aaa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.179+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/1152605525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a98bd0 0x55e113a9dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.179+0000 7f43e9ddf740 1 -- 172.21.15.112:0/1152605525 >> 172.21.15.112:0/1152605525 conn(0x55e113a94140 msgr2=0x55e113a973a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:05.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.179+0000 7f43e9ddf740 1 -- 172.21.15.112:0/1152605525 shutdown_connections 2024-03-16T09:15:05.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.179+0000 7f43e9ddf740 1 -- 172.21.15.112:0/1152605525 wait complete. 2024-03-16T09:15:05.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.179+0000 7f43e9ddf740 1 -- start start 2024-03-16T09:15:05.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.179+0000 7f43e9ddf740 1 Processor -- start 2024-03-16T09:15:05.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e9ddf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a98bd0 0x55e113aa82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e9ddf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a995c0 0x55e113b11500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e9ddf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a9e730 0x55e113b13a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e9ddf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e113ab0aa0 con 0x55e113a98bd0 2024-03-16T09:15:05.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e9ddf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e113ab03d0 con 0x55e113a995c0 2024-03-16T09:15:05.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e9ddf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e113ab1170 con 0x55e113a9e730 2024-03-16T09:15:05.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e9ddf740 10 client.0 _ll_register_callbacks cb 0x7f43d400e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:15:05.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e4eb9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a98bd0 0x55e113aa82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e4eb9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a98bd0 0x55e113aa82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38376/0 (socket says 172.21.15.112:38376) 2024-03-16T09:15:05.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e4eb9640 1 -- 172.21.15.112:0/2702675253 learned_addr learned my addr 172.21.15.112:0/2702675253 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:05.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43dffff640 1 --2- 172.21.15.112:0/2702675253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a995c0 0x55e113b11500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e56ba640 1 --2- 172.21.15.112:0/2702675253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a9e730 0x55e113b13a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e4eb9640 1 -- 172.21.15.112:0/2702675253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a9e730 msgr2=0x55e113b13a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e4eb9640 1 --2- 172.21.15.112:0/2702675253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a9e730 0x55e113b13a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e4eb9640 1 -- 172.21.15.112:0/2702675253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a995c0 msgr2=0x55e113b11500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e4eb9640 1 --2- 172.21.15.112:0/2702675253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a995c0 0x55e113b11500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e4eb9640 1 -- 172.21.15.112:0/2702675253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e113b13f90 con 0x55e113a98bd0 2024-03-16T09:15:05.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e4eb9640 1 --2- 172.21.15.112:0/2702675253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a98bd0 0x55e113aa82a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009250 tx=0x7f43d0005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bac033b8562ccec server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:05.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43bffff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:15:05.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43bffff640 1 -- 172.21.15.112:0/2702675253 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d001e040 con 0x55e113a98bd0 2024-03-16T09:15:05.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43bffff640 1 -- 172.21.15.112:0/2702675253 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f43d00054d0 con 0x55e113a98bd0 2024-03-16T09:15:05.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e9ddf740 1 -- 172.21.15.112:0/2702675253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43d8052070 con 0x55e113a98bd0 2024-03-16T09:15:05.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43bffff640 1 -- 172.21.15.112:0/2702675253 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f43d003a050 con 0x55e113a98bd0 2024-03-16T09:15:05.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e9ddf740 10 client.15060 fetch_fsmap learned FSMap version 21 2024-03-16T09:15:05.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.180+0000 7f43e9ddf740 1 -- 172.21.15.112:0/2702675253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55e113b152a0 con 0x55e113a98bd0 2024-03-16T09:15:05.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.181+0000 7f43bffff640 1 -- 172.21.15.112:0/2702675253 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f43d8052070 con 0x55e113a98bd0 2024-03-16T09:15:05.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.181+0000 7f43bffff640 1 -- 172.21.15.112:0/2702675253 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f43d00195f0 con 0x55e113a98bd0 2024-03-16T09:15:05.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.181+0000 7f43e9ddf740 10 client.15060 fetch_fsmap finished waiting for FSMap version 21 2024-03-16T09:15:05.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.181+0000 7f43e9ddf740 10 client.15060 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-16T09:15:05.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.181+0000 7f43e9ddf740 20 client.15060 populate_metadata read hostname 'smithi112' 2024-03-16T09:15:05.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.181+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/2702675253 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55e113b19c80 0x55e113b18990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.181+0000 7f43e9ddf740 4 client.15060 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T09:15:05.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.181+0000 7f43e9ddf740 1 -- 172.21.15.112:0/2702675253 --> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] -- command(tid 0: {"prefix": "status"}) -- 0x55e113a04510 con 0x55e113b19c80 2024-03-16T09:15:05.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.181+0000 7f43dffff640 1 --2- 172.21.15.112:0/2702675253 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55e113b19c80 0x55e113b18990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.181+0000 7f43bffff640 10 client.15060 ms_handle_connect on v2:172.21.15.194:6838/2528943309 2024-03-16T09:15:05.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.181+0000 7f43dffff640 1 --2- 172.21.15.112:0/2702675253 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55e113b19c80 0x55e113b18990 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ccd090b36e5910f7 server_cookie=b08cc1030f77998 in_seq=0 out_seq=0 2024-03-16T09:15:05.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.181+0000 7f43bffff640 1 -- 172.21.15.112:0/2702675253 <== mds.1 v2:172.21.15.194:6838/2528943309 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e113a04510 con 0x55e113b19c80 2024-03-16T09:15:05.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.181+0000 7f43bffff640 10 client.15060 handle_command_reply: tid=0 2024-03-16T09:15:05.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.181+0000 7f43e9ddf740 1 client.15060 shutdown 2024-03-16T09:15:05.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e9ddf740 1 -- 172.21.15.112:0/2702675253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a98bd0 msgr2=0x55e113aa82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/2702675253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a98bd0 0x55e113aa82a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009250 tx=0x7f43d0005ab0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e9ddf740 1 -- 172.21.15.112:0/2702675253 shutdown_connections 2024-03-16T09:15:05.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/2702675253 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55e113b19c80 0x55e113b18990 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/2702675253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e113a9e730 0x55e113b13a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43bffff640 0 client.15060 ms_handle_reset on v2:172.21.15.194:6838/2528943309 2024-03-16T09:15:05.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/2702675253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e113a995c0 0x55e113b11500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e9ddf740 1 --2- 172.21.15.112:0/2702675253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e113a98bd0 0x55e113aa82a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e9ddf740 1 -- 172.21.15.112:0/2702675253 >> 172.21.15.112:0/2702675253 conn(0x55e113a94140 msgr2=0x55e113a97c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:05.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e9ddf740 1 -- 172.21.15.112:0/2702675253 shutdown_connections 2024-03-16T09:15:05.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e9ddf740 1 -- 172.21.15.112:0/2702675253 wait complete. 2024-03-16T09:15:05.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e9ddf740 20 client.15060 trim_cache size 0 max 16384 2024-03-16T09:15:05.191 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:15:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "273be2a4-e374-11ee-95c9-87774f69a715", 2024-03-16T09:15:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 1, 2024-03-16T09:15:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24466, 2024-03-16T09:15:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-03-16T09:15:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-03-16T09:15:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-03-16T09:15:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 86.060019248000003, 2024-03-16T09:15:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 21, 2024-03-16T09:15:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 87, 2024-03-16T09:15:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 85, 2024-03-16T09:15:05.192 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 102.52660004000001 2024-03-16T09:15:05.192 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:15:05.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e711d640 1 -- 172.21.15.112:0/3376218352 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f43c80a2800 msgr2=0x7f43c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e711d640 1 --2- 172.21.15.112:0/3376218352 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f43c80a2800 0x7f43c80a4cc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f43d8005e30 tx=0x7f43d8032450 comp rx=0 tx=0).stop 2024-03-16T09:15:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e711d640 1 -- 172.21.15.112:0/3376218352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e01018e0 msgr2=0x7f43e019c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e711d640 1 --2- 172.21.15.112:0/3376218352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e01018e0 0x7f43e019c8c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f43d400ea60 tx=0x7f43d400ef20 comp rx=0 tx=0).stop 2024-03-16T09:15:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e711d640 1 -- 172.21.15.112:0/3376218352 shutdown_connections 2024-03-16T09:15:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e711d640 1 --2- 172.21.15.112:0/3376218352 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f43c80a2800 0x7f43c80a4cc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e711d640 1 --2- 172.21.15.112:0/3376218352 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43e01022a0 0x7f43e019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e711d640 1 --2- 172.21.15.112:0/3376218352 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43e01018e0 0x7f43e019c8c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e711d640 1 --2- 172.21.15.112:0/3376218352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f43e01006f0 0x7f43e006a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.182+0000 7f43e711d640 1 -- 172.21.15.112:0/3376218352 >> 172.21.15.112:0/3376218352 conn(0x7f43e00fbc80 msgr2=0x7f43e0104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.183+0000 7f43e711d640 1 -- 172.21.15.112:0/3376218352 shutdown_connections 2024-03-16T09:15:05.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.183+0000 7f43e711d640 1 -- 172.21.15.112:0/3376218352 wait complete. 2024-03-16T09:15:05.194 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:04 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3801419382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:05.194 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:04 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3471205349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:05.194 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:04 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1615552221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:05.195 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-16T09:15:05.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:04 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3801419382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:05.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:04 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3471205349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:05.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:04 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1615552221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:05.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.296+0000 7f8ad23c4640 1 -- 172.21.15.112:0/3598425421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8acc1033c0 msgr2=0x7f8acc109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.296+0000 7f8ad23c4640 1 --2- 172.21.15.112:0/3598425421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8acc1033c0 0x7f8acc109c50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009be0 tx=0x7f8ac402bef0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.296+0000 7f8ad23c4640 1 -- 172.21.15.112:0/3598425421 shutdown_connections 2024-03-16T09:15:05.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.296+0000 7f8ad23c4640 1 --2- 172.21.15.112:0/3598425421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8acc1033c0 0x7f8acc109c50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.296+0000 7f8ad23c4640 1 --2- 172.21.15.112:0/3598425421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8acc102a00 0x7f8acc102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.296+0000 7f8ad23c4640 1 --2- 172.21.15.112:0/3598425421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8acc101810 0x7f8acc101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.296+0000 7f8ad23c4640 1 -- 172.21.15.112:0/3598425421 >> 172.21.15.112:0/3598425421 conn(0x7f8acc0fcd80 msgr2=0x7f8acc0ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:05.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.296+0000 7f8ad23c4640 1 -- 172.21.15.112:0/3598425421 shutdown_connections 2024-03-16T09:15:05.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.296+0000 7f8ad23c4640 1 -- 172.21.15.112:0/3598425421 wait complete. 2024-03-16T09:15:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.296+0000 7f8ad23c4640 1 Processor -- start 2024-03-16T09:15:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.296+0000 7f8ad23c4640 1 -- start start 2024-03-16T09:15:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8ad23c4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8acc101810 0x7f8acc197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8ad23c4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8acc102a00 0x7f8acc1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8ad23c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8acc1033c0 0x7f8acc19f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8ad23c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc10d5c0 con 0x7f8acc1033c0 2024-03-16T09:15:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8ad23c4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc10d440 con 0x7f8acc102a00 2024-03-16T09:15:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8ad23c4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc10d740 con 0x7f8acc101810 2024-03-16T09:15:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8acbfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8acc101810 0x7f8acc197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8ad0961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8acc1033c0 0x7f8acc19f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8acbfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8acc101810 0x7f8acc197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:38906/0 (socket says 172.21.15.112:38906) 2024-03-16T09:15:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8ad0961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8acc1033c0 0x7f8acc19f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38392/0 (socket says 172.21.15.112:38392) 2024-03-16T09:15:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8acbfff640 1 -- 172.21.15.112:0/3354883182 learned_addr learned my addr 172.21.15.112:0/3354883182 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8acb7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8acc102a00 0x7f8acc1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8ad0961640 1 -- 172.21.15.112:0/3354883182 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8acc101810 msgr2=0x7f8acc197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8ad0961640 1 --2- 172.21.15.112:0/3354883182 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8acc101810 0x7f8acc197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8ad0961640 1 -- 172.21.15.112:0/3354883182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8acc102a00 msgr2=0x7f8acc1980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8ad0961640 1 --2- 172.21.15.112:0/3354883182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8acc102a00 0x7f8acc1980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8ad0961640 1 -- 172.21.15.112:0/3354883182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc19f7c0 con 0x7f8acc1033c0 2024-03-16T09:15:05.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8ad0961640 1 --2- 172.21.15.112:0/3354883182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8acc1033c0 0x7f8acc19f150 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8ac402c430 tx=0x7f8ac4033820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:05.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8ac97fa640 1 -- 172.21.15.112:0/3354883182 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4043020 con 0x7f8acc1033c0 2024-03-16T09:15:05.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8ad23c4640 1 -- 172.21.15.112:0/3354883182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8acc19fa50 con 0x7f8acc1033c0 2024-03-16T09:15:05.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8ac97fa640 1 -- 172.21.15.112:0/3354883182 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8ac402ca80 con 0x7f8acc1033c0 2024-03-16T09:15:05.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8ac97fa640 1 -- 172.21.15.112:0/3354883182 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac403e740 con 0x7f8acc1033c0 2024-03-16T09:15:05.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.297+0000 7f8ad23c4640 1 -- 172.21.15.112:0/3354883182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8acc19ffa0 con 0x7f8acc1033c0 2024-03-16T09:15:05.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.300+0000 7f8ac97fa640 1 -- 172.21.15.112:0/3354883182 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f8ac4046020 con 0x7f8acc1033c0 2024-03-16T09:15:05.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.300+0000 7f8ad4faf740 1 Processor -- start 2024-03-16T09:15:05.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.300+0000 7f8ad4faf740 1 -- start start 2024-03-16T09:15:05.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.300+0000 7f8ac97fa640 1 --2- 172.21.15.112:0/3354883182 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8ab40a2800 0x7f8ab40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.300+0000 7f8ad4faf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc2fe0 0x563c03dc3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.300+0000 7f8ad4faf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc39d0 0x563c03dc84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.300+0000 7f8ad4faf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc8b40 0x563c03dcafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.300+0000 7f8ad4faf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563c03d22c80 con 0x563c03dc2fe0 2024-03-16T09:15:05.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.300+0000 7f8ad4faf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563c03ccbaa0 con 0x563c03dc39d0 2024-03-16T09:15:05.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.300+0000 7f8ad4faf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563c03d1ed20 con 0x563c03dc8b40 2024-03-16T09:15:05.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.300+0000 7f8acbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc2fe0 0x563c03dc3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.300+0000 7f8acb7fe640 1 --2- 172.21.15.112:0/3354883182 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8ab40a2800 0x7f8ab40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.300+0000 7f8ac97fa640 1 -- 172.21.15.112:0/3354883182 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8ac40f0fd0 con 0x7f8acc1033c0 2024-03-16T09:15:05.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.300+0000 7f8acbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc2fe0 0x563c03dc3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38404/0 (socket says 172.21.15.112:38404) 2024-03-16T09:15:05.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.300+0000 7f8acbfff640 1 -- 172.21.15.112:0/2419296223 learned_addr learned my addr 172.21.15.112:0/2419296223 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:05.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.300+0000 7f8acb7fe640 1 --2- 172.21.15.112:0/2419296223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc39d0 0x563c03dc84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.300+0000 7f8ad0961640 1 --2- 172.21.15.112:0/2419296223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc8b40 0x563c03dcafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.301+0000 7f8acbfff640 1 -- 172.21.15.112:0/2419296223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc8b40 msgr2=0x563c03dcafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.301+0000 7f8acbfff640 1 --2- 172.21.15.112:0/2419296223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc8b40 0x563c03dcafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.301+0000 7f8acbfff640 1 -- 172.21.15.112:0/2419296223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc39d0 msgr2=0x563c03dc84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.301+0000 7f8acbfff640 1 --2- 172.21.15.112:0/2419296223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc39d0 0x563c03dc84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.301+0000 7f8acbfff640 1 -- 172.21.15.112:0/2419296223 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c03cedff0 con 0x563c03dc2fe0 2024-03-16T09:15:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.301+0000 7f8acb7fe640 1 --2- 172.21.15.112:0/3354883182 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8ab40a2800 0x7f8ab40a4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009fd0 tx=0x7f8ac0009b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.301+0000 7f8acbfff640 1 --2- 172.21.15.112:0/2419296223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc2fe0 0x563c03dc3400 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009d60 tx=0x7f8abc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c275fe9899a9c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.301+0000 7f8ab27fc640 1 -- 172.21.15.112:0/2419296223 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc004220 con 0x563c03dc2fe0 2024-03-16T09:15:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.301+0000 7f8ab27fc640 1 -- 172.21.15.112:0/2419296223 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8abc0043c0 con 0x563c03dc2fe0 2024-03-16T09:15:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.301+0000 7f8ab27fc640 1 -- 172.21.15.112:0/2419296223 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc019950 con 0x563c03dc2fe0 2024-03-16T09:15:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.302+0000 7f8ad4faf740 1 -- 172.21.15.112:0/2419296223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc2fe0 msgr2=0x563c03dc3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.302+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/2419296223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc2fe0 0x563c03dc3400 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009d60 tx=0x7f8abc011810 comp rx=0 tx=0).stop 2024-03-16T09:15:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.302+0000 7f8ad4faf740 1 -- 172.21.15.112:0/2419296223 shutdown_connections 2024-03-16T09:15:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.302+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/2419296223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc8b40 0x563c03dcafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.302+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/2419296223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc39d0 0x563c03dc84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.302+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/2419296223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc2fe0 0x563c03dc3400 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.302+0000 7f8ad4faf740 1 -- 172.21.15.112:0/2419296223 >> 172.21.15.112:0/2419296223 conn(0x563c03dba140 msgr2=0x563c03dba540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:05.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.302+0000 7f8ad4faf740 1 -- 172.21.15.112:0/2419296223 shutdown_connections 2024-03-16T09:15:05.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.302+0000 7f8ad4faf740 1 -- 172.21.15.112:0/2419296223 wait complete. 2024-03-16T09:15:05.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.302+0000 7f8ad4faf740 1 -- start start 2024-03-16T09:15:05.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.302+0000 7f8ad4faf740 1 Processor -- start 2024-03-16T09:15:05.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.302+0000 7f8ad4faf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc2fe0 0x563c03dcd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.302+0000 7f8ad4faf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc39d0 0x563c03dcd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.302+0000 7f8ad4faf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc8b40 0x563c03dd46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.302+0000 7f8ad4faf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563c03dcbaf0 con 0x563c03dc8b40 2024-03-16T09:15:05.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.302+0000 7f8ad4faf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563c03cdeef0 con 0x563c03dc39d0 2024-03-16T09:15:05.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.302+0000 7f8ad4faf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563c03dcbc70 con 0x563c03dc2fe0 2024-03-16T09:15:05.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8ad4faf740 10 client.0 _ll_register_callbacks cb 0x563c03cd9dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:15:05.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8ad0961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc8b40 0x563c03dd46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8ad0961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc8b40 0x563c03dd46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38410/0 (socket says 172.21.15.112:38410) 2024-03-16T09:15:05.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8ad0961640 1 -- 172.21.15.112:0/1714392010 learned_addr learned my addr 172.21.15.112:0/1714392010 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:05.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8acbfff640 1 --2- 172.21.15.112:0/1714392010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc2fe0 0x563c03dcd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8acb7fe640 1 --2- 172.21.15.112:0/1714392010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc39d0 0x563c03dcd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8ad0961640 1 -- 172.21.15.112:0/1714392010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc2fe0 msgr2=0x563c03dcd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8ad0961640 1 --2- 172.21.15.112:0/1714392010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc2fe0 0x563c03dcd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8ad0961640 1 -- 172.21.15.112:0/1714392010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc39d0 msgr2=0x563c03dcd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8ad0961640 1 --2- 172.21.15.112:0/1714392010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc39d0 0x563c03dcd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8ad0961640 1 -- 172.21.15.112:0/1714392010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c03ceba30 con 0x563c03dc8b40 2024-03-16T09:15:05.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8ad0961640 1 --2- 172.21.15.112:0/1714392010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc8b40 0x563c03dd46f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40ba350 tx=0x7f8ac40bd4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c7eff3a396492a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:05.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8aab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:15:05.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8aab7fe640 1 -- 172.21.15.112:0/1714392010 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac40a8070 con 0x563c03dc8b40 2024-03-16T09:15:05.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8aab7fe640 1 -- 172.21.15.112:0/1714392010 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8ac4003b80 con 0x563c03dc8b40 2024-03-16T09:15:05.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1714392010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8abc0043c0 con 0x563c03dc8b40 2024-03-16T09:15:05.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8aab7fe640 1 -- 172.21.15.112:0/1714392010 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8ac40bd980 con 0x563c03dc8b40 2024-03-16T09:15:05.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8ad4faf740 10 client.15072 fetch_fsmap learned FSMap version 21 2024-03-16T09:15:05.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1714392010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563c03a372d0 con 0x563c03dc8b40 2024-03-16T09:15:05.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.303+0000 7f8aab7fe640 1 -- 172.21.15.112:0/1714392010 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8abc0043c0 con 0x563c03dc8b40 2024-03-16T09:15:05.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.304+0000 7f8aab7fe640 1 -- 172.21.15.112:0/1714392010 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8ac4004b40 con 0x563c03dc8b40 2024-03-16T09:15:05.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.304+0000 7f8ad4faf740 10 client.15072 fetch_fsmap finished waiting for FSMap version 21 2024-03-16T09:15:05.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.304+0000 7f8ad4faf740 10 client.15072 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T09:15:05.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.304+0000 7f8ad4faf740 20 client.15072 populate_metadata read hostname 'smithi112' 2024-03-16T09:15:05.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.304+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/1714392010 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x563c03dd9be0 0x563c03ddb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.304+0000 7f8ad4faf740 4 client.15072 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:15:05.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.304+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1714392010 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c03d301c0 con 0x563c03dd9be0 2024-03-16T09:15:05.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.304+0000 7f8acbfff640 1 --2- 172.21.15.112:0/1714392010 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x563c03dd9be0 0x563c03ddb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.304+0000 7f8aab7fe640 10 client.15072 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:05.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.304+0000 7f8acbfff640 1 --2- 172.21.15.112:0/1714392010 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x563c03dd9be0 0x563c03ddb050 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=614f7db04e4d10ef server_cookie=739752a4d90a1182 in_seq=0 out_seq=0 2024-03-16T09:15:05.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.305+0000 7f8aab7fe640 1 -- 172.21.15.112:0/1714392010 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563c03d301c0 con 0x563c03dd9be0 2024-03-16T09:15:05.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.305+0000 7f8aab7fe640 10 client.15072 handle_command_reply: tid=0 2024-03-16T09:15:05.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.305+0000 7f8ad4faf740 1 client.15072 shutdown 2024-03-16T09:15:05.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.305+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1714392010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc8b40 msgr2=0x563c03dd46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.305+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/1714392010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc8b40 0x563c03dd46f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40ba350 tx=0x7f8ac40bd4c0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.305+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1714392010 shutdown_connections 2024-03-16T09:15:05.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.305+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/1714392010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc8b40 0x563c03dd46f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.305+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/1714392010 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x563c03dd9be0 0x563c03ddb050 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.305+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/1714392010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc39d0 0x563c03dcd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.305+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/1714392010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc2fe0 0x563c03dcd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.305+0000 7f8aab7fe640 0 client.15072 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:05.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.305+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1714392010 >> 172.21.15.112:0/1714392010 conn(0x563c03dba140 msgr2=0x563c03db9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:05.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.305+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1714392010 shutdown_connections 2024-03-16T09:15:05.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.305+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1714392010 wait complete. 2024-03-16T09:15:05.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.305+0000 7f8ad4faf740 20 client.15072 trim_cache size 0 max 16384 2024-03-16T09:15:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.322+0000 7f8ad4faf740 1 Processor -- start 2024-03-16T09:15:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8ad4faf740 1 -- start start 2024-03-16T09:15:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8ad4faf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc2fe0 0x563c03dd8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8ad4faf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc39d0 0x563c03dd8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8ad4faf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc8b40 0x563c03dd46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8ad4faf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563c03dd9390 con 0x563c03dc8b40 2024-03-16T09:15:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8ad4faf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563c03dd9a60 con 0x563c03dc2fe0 2024-03-16T09:15:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8ad4faf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563c03dda130 con 0x563c03dc39d0 2024-03-16T09:15:05.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8ad0961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc8b40 0x563c03dd46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8acbfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc2fe0 0x563c03dd8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8ad0961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc8b40 0x563c03dd46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38412/0 (socket says 172.21.15.112:38412) 2024-03-16T09:15:05.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8acbfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc2fe0 0x563c03dd8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:38968/0 (socket says 172.21.15.112:38968) 2024-03-16T09:15:05.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8ad0961640 1 -- 172.21.15.112:0/1756265086 learned_addr learned my addr 172.21.15.112:0/1756265086 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:05.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8acb7fe640 1 --2- 172.21.15.112:0/1756265086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc39d0 0x563c03dd8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8ad0961640 1 -- 172.21.15.112:0/1756265086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc39d0 msgr2=0x563c03dd8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8ad0961640 1 --2- 172.21.15.112:0/1756265086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc39d0 0x563c03dd8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8ad0961640 1 -- 172.21.15.112:0/1756265086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc2fe0 msgr2=0x563c03dd8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8ad0961640 1 --2- 172.21.15.112:0/1756265086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc2fe0 0x563c03dd8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8ad0961640 1 -- 172.21.15.112:0/1756265086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c03b016b0 con 0x563c03dc8b40 2024-03-16T09:15:05.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8ad0961640 1 --2- 172.21.15.112:0/1756265086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc8b40 0x563c03dd46b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40bbc30 tx=0x7f8ac40bd5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8191bfb50b7c8c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:05.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8ab1ffb640 1 -- 172.21.15.112:0/1756265086 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac40b9c20 con 0x563c03dc8b40 2024-03-16T09:15:05.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8ab1ffb640 1 -- 172.21.15.112:0/1756265086 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8ac40c6120 con 0x563c03dc8b40 2024-03-16T09:15:05.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.323+0000 7f8ab1ffb640 1 -- 172.21.15.112:0/1756265086 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac40bd2e0 con 0x563c03dc8b40 2024-03-16T09:15:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.324+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1756265086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc8b40 msgr2=0x563c03dd46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.324+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/1756265086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc8b40 0x563c03dd46b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40bbc30 tx=0x7f8ac40bd5f0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.324+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1756265086 shutdown_connections 2024-03-16T09:15:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.324+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/1756265086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc8b40 0x563c03dd46b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.324+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/1756265086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc39d0 0x563c03dd8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.324+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/1756265086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc2fe0 0x563c03dd8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.324+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1756265086 >> 172.21.15.112:0/1756265086 conn(0x563c03dba140 msgr2=0x563c03dcd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.324+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1756265086 shutdown_connections 2024-03-16T09:15:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.324+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1756265086 wait complete. 2024-03-16T09:15:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.324+0000 7f8ad4faf740 1 -- start start 2024-03-16T09:15:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.324+0000 7f8ad4faf740 1 Processor -- start 2024-03-16T09:15:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8ad4faf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc2fe0 0x563c03dd26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8ad4faf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc39d0 0x563c03e3b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8ad4faf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc8b40 0x563c03e3de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8ad4faf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563c03ddaed0 con 0x563c03dc2fe0 2024-03-16T09:15:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8ad4faf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563c03dda800 con 0x563c03dc8b40 2024-03-16T09:15:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8ad4faf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563c03ddb5a0 con 0x563c03dc39d0 2024-03-16T09:15:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8ad4faf740 10 client.0 _ll_register_callbacks cb 0x7f8ac4009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:15:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8acbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc2fe0 0x563c03dd26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8acbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc2fe0 0x563c03dd26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38422/0 (socket says 172.21.15.112:38422) 2024-03-16T09:15:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8acbfff640 1 -- 172.21.15.112:0/1932953848 learned_addr learned my addr 172.21.15.112:0/1932953848 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8acb7fe640 1 --2- 172.21.15.112:0/1932953848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc39d0 0x563c03e3b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8ad0961640 1 --2- 172.21.15.112:0/1932953848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc8b40 0x563c03e3de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8acbfff640 1 -- 172.21.15.112:0/1932953848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc39d0 msgr2=0x563c03e3b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8acbfff640 1 --2- 172.21.15.112:0/1932953848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc39d0 0x563c03e3b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8acbfff640 1 -- 172.21.15.112:0/1932953848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc8b40 msgr2=0x563c03e3de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8acbfff640 1 --2- 172.21.15.112:0/1932953848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc8b40 0x563c03e3de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8acbfff640 1 -- 172.21.15.112:0/1932953848 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c03e3e3c0 con 0x563c03dc2fe0 2024-03-16T09:15:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8acbfff640 1 --2- 172.21.15.112:0/1932953848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc2fe0 0x563c03dd26a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009d30 tx=0x7f8abc009460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196b7481d5ae8f4e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8aab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:15:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8aab7fe640 1 -- 172.21.15.112:0/1932953848 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc0094d0 con 0x563c03dc2fe0 2024-03-16T09:15:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8aab7fe640 1 -- 172.21.15.112:0/1932953848 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8abc004310 con 0x563c03dc2fe0 2024-03-16T09:15:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1932953848 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ac40c6120 con 0x563c03dc2fe0 2024-03-16T09:15:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8aab7fe640 1 -- 172.21.15.112:0/1932953848 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8abc02ca30 con 0x563c03dc2fe0 2024-03-16T09:15:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8ad4faf740 10 client.15078 fetch_fsmap learned FSMap version 21 2024-03-16T09:15:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.325+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1932953848 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x563c03e3f6d0 con 0x563c03dc2fe0 2024-03-16T09:15:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.326+0000 7f8aab7fe640 1 -- 172.21.15.112:0/1932953848 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8ac40c6120 con 0x563c03dc2fe0 2024-03-16T09:15:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.326+0000 7f8aab7fe640 1 -- 172.21.15.112:0/1932953848 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8abc02c3a0 con 0x563c03dc2fe0 2024-03-16T09:15:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.326+0000 7f8ad4faf740 10 client.15078 fetch_fsmap finished waiting for FSMap version 21 2024-03-16T09:15:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.326+0000 7f8ad4faf740 10 client.15078 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T09:15:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.326+0000 7f8ad4faf740 20 client.15078 populate_metadata read hostname 'smithi112' 2024-03-16T09:15:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.326+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/1932953848 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x563c03e44120 0x563c03e40560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.326+0000 7f8ad4faf740 4 client.15078 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T09:15:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.326+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1932953848 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "status"}) -- 0x563c03d2a510 con 0x563c03e44120 2024-03-16T09:15:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.326+0000 7f8acb7fe640 1 --2- 172.21.15.112:0/1932953848 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x563c03e44120 0x563c03e40560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.326+0000 7f8acb7fe640 1 --2- 172.21.15.112:0/1932953848 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x563c03e44120 0x563c03e40560 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40a5d4360b884285 server_cookie=ec927f16596504ce in_seq=0 out_seq=0 2024-03-16T09:15:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.326+0000 7f8aab7fe640 10 client.15078 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8aab7fe640 1 -- 172.21.15.112:0/1932953848 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563c03d2a510 con 0x563c03e44120 2024-03-16T09:15:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8aab7fe640 10 client.15078 handle_command_reply: tid=0 2024-03-16T09:15:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad4faf740 1 client.15078 shutdown 2024-03-16T09:15:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1932953848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc2fe0 msgr2=0x563c03dd26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/1932953848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc2fe0 0x563c03dd26a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009d30 tx=0x7f8abc009460 comp rx=0 tx=0).stop 2024-03-16T09:15:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1932953848 shutdown_connections 2024-03-16T09:15:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/1932953848 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x563c03e44120 0x563c03e40560 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/1932953848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c03dc8b40 0x563c03e3de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/1932953848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563c03dc39d0 0x563c03e3b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad4faf740 1 --2- 172.21.15.112:0/1932953848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563c03dc2fe0 0x563c03dd26a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1932953848 >> 172.21.15.112:0/1932953848 conn(0x563c03dba140 msgr2=0x563c03dd8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:05.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8aab7fe640 0 client.15078 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:05.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1932953848 shutdown_connections 2024-03-16T09:15:05.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad4faf740 1 -- 172.21.15.112:0/1932953848 wait complete. 2024-03-16T09:15:05.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad4faf740 20 client.15078 trim_cache size 0 max 16384 2024-03-16T09:15:05.337 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:15:05.337 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "273be2a4-e374-11ee-95c9-87774f69a715", 2024-03-16T09:15:05.337 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-03-16T09:15:05.337 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24494, 2024-03-16T09:15:05.337 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-03-16T09:15:05.337 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-03-16T09:15:05.337 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-03-16T09:15:05.337 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 88.216868903000005, 2024-03-16T09:15:05.338 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 21, 2024-03-16T09:15:05.338 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 87, 2024-03-16T09:15:05.338 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 85, 2024-03-16T09:15:05.338 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 93.889833808999995 2024-03-16T09:15:05.338 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:15:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad23c4640 1 -- 172.21.15.112:0/3354883182 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8ab40a2800 msgr2=0x7f8ab40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad23c4640 1 --2- 172.21.15.112:0/3354883182 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8ab40a2800 0x7f8ab40a4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009fd0 tx=0x7f8ac0009b80 comp rx=0 tx=0).stop 2024-03-16T09:15:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad23c4640 1 -- 172.21.15.112:0/3354883182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8acc1033c0 msgr2=0x7f8acc19f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad23c4640 1 --2- 172.21.15.112:0/3354883182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8acc1033c0 0x7f8acc19f150 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8ac402c430 tx=0x7f8ac4033820 comp rx=0 tx=0).stop 2024-03-16T09:15:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad23c4640 1 -- 172.21.15.112:0/3354883182 shutdown_connections 2024-03-16T09:15:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad23c4640 1 --2- 172.21.15.112:0/3354883182 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8ab40a2800 0x7f8ab40a4cc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad23c4640 1 --2- 172.21.15.112:0/3354883182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8acc1033c0 0x7f8acc19f150 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad23c4640 1 --2- 172.21.15.112:0/3354883182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8acc102a00 0x7f8acc1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad23c4640 1 --2- 172.21.15.112:0/3354883182 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8acc101810 0x7f8acc197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.327+0000 7f8ad23c4640 1 -- 172.21.15.112:0/3354883182 >> 172.21.15.112:0/3354883182 conn(0x7f8acc0fcd80 msgr2=0x7f8acc105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.328+0000 7f8ad23c4640 1 -- 172.21.15.112:0/3354883182 shutdown_connections 2024-03-16T09:15:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:05.328+0000 7f8ad23c4640 1 -- 172.21.15.112:0/3354883182 wait complete. 2024-03-16T09:15:05.340 INFO:tasks.cephadm:Setup complete, yielding 2024-03-16T09:15:05.341 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T09:15:05.349 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-03-16T09:15:05.349 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- bash -c 'ceph orch status' 2024-03-16T09:15:05.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:04 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3801419382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:05.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:04 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3471205349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:05.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:04 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1615552221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:05.586 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:15:06.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:05 smithi159 ceph-mon[36739]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-16T09:15:06.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:05 smithi112 ceph-mon[32307]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-16T09:15:06.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:05 smithi194 ceph-mon[36729]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-16T09:15:07.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.269+0000 7f77d4820700 1 -- 172.21.15.112:0/333042755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77d0103520 msgr2=0x7f77d0103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:07.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.269+0000 7f77d4820700 1 --2- 172.21.15.112:0/333042755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77d0103520 0x7f77d0103980 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f77b80099c0 tx=0x7f77b8009cd0 comp rx=0 tx=0).stop 2024-03-16T09:15:07.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.269+0000 7f77d4820700 1 -- 172.21.15.112:0/333042755 shutdown_connections 2024-03-16T09:15:07.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.269+0000 7f77d4820700 1 --2- 172.21.15.112:0/333042755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d0103ec0 0x7f77d010a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:07.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.269+0000 7f77d4820700 1 --2- 172.21.15.112:0/333042755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77d0102320 0x7f77d0102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:07.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.269+0000 7f77d4820700 1 --2- 172.21.15.112:0/333042755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77d0103520 0x7f77d0103980 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:07.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.269+0000 7f77d4820700 1 -- 172.21.15.112:0/333042755 >> 172.21.15.112:0/333042755 conn(0x7f77d00fd8e0 msgr2=0x7f77d00ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:07.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.269+0000 7f77d4820700 1 -- 172.21.15.112:0/333042755 shutdown_connections 2024-03-16T09:15:07.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.269+0000 7f77d4820700 1 -- 172.21.15.112:0/333042755 wait complete. 2024-03-16T09:15:07.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77d4820700 1 Processor -- start 2024-03-16T09:15:07.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77d4820700 1 -- start start 2024-03-16T09:15:07.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77d4820700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d0102320 0x7f77d0076070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:07.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77d4820700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77d0103520 0x7f77d00746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:07.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77d4820700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77d0103ec0 0x7f77d0074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:07.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77d4820700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f77d010e050 con 0x7f77d0103520 2024-03-16T09:15:07.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77d4820700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f77d010ded0 con 0x7f77d0102320 2024-03-16T09:15:07.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77d4820700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77d010e1d0 con 0x7f77d0103ec0 2024-03-16T09:15:07.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77c5d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77d0103520 0x7f77d00746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:07.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77ce59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d0102320 0x7f77d0076070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:07.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77c5d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77d0103520 0x7f77d00746c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38432/0 (socket says 172.21.15.112:38432) 2024-03-16T09:15:07.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77ce59c700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d0102320 0x7f77d0076070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:38992/0 (socket says 172.21.15.112:38992) 2024-03-16T09:15:07.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77c5d9b700 1 -- 172.21.15.112:0/1996473781 learned_addr learned my addr 172.21.15.112:0/1996473781 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:07.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77ced9d700 1 --2- 172.21.15.112:0/1996473781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77d0103ec0 0x7f77d0074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:07.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77c5d9b700 1 -- 172.21.15.112:0/1996473781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77d0103ec0 msgr2=0x7f77d0074c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:07.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77c5d9b700 1 --2- 172.21.15.112:0/1996473781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77d0103ec0 0x7f77d0074c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:07.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77c5d9b700 1 -- 172.21.15.112:0/1996473781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d0102320 msgr2=0x7f77d0076070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:07.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77c5d9b700 1 --2- 172.21.15.112:0/1996473781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d0102320 0x7f77d0076070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:07.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77c5d9b700 1 -- 172.21.15.112:0/1996473781 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d00754b0 con 0x7f77d0103520 2024-03-16T09:15:07.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77ced9d700 1 --2- 172.21.15.112:0/1996473781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77d0103ec0 0x7f77d0074c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:07.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.270+0000 7f77ce59c700 1 --2- 172.21.15.112:0/1996473781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d0102320 0x7f77d0076070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:07.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.271+0000 7f77c5d9b700 1 --2- 172.21.15.112:0/1996473781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77d0103520 0x7f77d00746c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f77b8003930 tx=0x7f77b800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:07.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.271+0000 7f77c7fff700 1 -- 172.21.15.112:0/1996473781 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8009e60 con 0x7f77d0103520 2024-03-16T09:15:07.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.271+0000 7f77c7fff700 1 -- 172.21.15.112:0/1996473781 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f77b800be40 con 0x7f77d0103520 2024-03-16T09:15:07.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.271+0000 7f77d4820700 1 -- 172.21.15.112:0/1996473781 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d01a26d0 con 0x7f77d0103520 2024-03-16T09:15:07.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.271+0000 7f77c7fff700 1 -- 172.21.15.112:0/1996473781 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b80173f0 con 0x7f77d0103520 2024-03-16T09:15:07.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.271+0000 7f77d4820700 1 -- 172.21.15.112:0/1996473781 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d01a2b20 con 0x7f77d0103520 2024-03-16T09:15:07.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.272+0000 7f77d4820700 1 -- 172.21.15.112:0/1996473781 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d0068f90 con 0x7f77d0103520 2024-03-16T09:15:07.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.273+0000 7f77c7fff700 1 -- 172.21.15.112:0/1996473781 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f77b8003b00 con 0x7f77d0103520 2024-03-16T09:15:07.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.273+0000 7f77c7fff700 1 --2- 172.21.15.112:0/1996473781 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f77bc0a2890 0x7f77bc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:07.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.273+0000 7f77ce59c700 1 --2- 172.21.15.112:0/1996473781 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f77bc0a2890 0x7f77bc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:07.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.274+0000 7f77c7fff700 1 -- 172.21.15.112:0/1996473781 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f77b80cf9e0 con 0x7f77d0103520 2024-03-16T09:15:07.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.274+0000 7f77ce59c700 1 --2- 172.21.15.112:0/1996473781 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f77bc0a2890 0x7f77bc0a4d50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f77d0103380 tx=0x7f77c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:07.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.277+0000 7f77c7fff700 1 -- 172.21.15.112:0/1996473781 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f77b805d740 con 0x7f77d0103520 2024-03-16T09:15:07.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.488+0000 7f77d4820700 1 -- 172.21.15.112:0/1996473781 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f77d0062710 con 0x7f77bc0a2890 2024-03-16T09:15:07.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.490+0000 7f77c7fff700 1 -- 172.21.15.112:0/1996473781 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f77d0062710 con 0x7f77bc0a2890 2024-03-16T09:15:07.491 INFO:teuthology.orchestra.run.smithi112.stdout:Backend: cephadm 2024-03-16T09:15:07.492 INFO:teuthology.orchestra.run.smithi112.stdout:Available: Yes 2024-03-16T09:15:07.492 INFO:teuthology.orchestra.run.smithi112.stdout:Paused: No 2024-03-16T09:15:07.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.492+0000 7f77d4820700 1 -- 172.21.15.112:0/1996473781 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f77bc0a2890 msgr2=0x7f77bc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:07.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.492+0000 7f77d4820700 1 --2- 172.21.15.112:0/1996473781 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f77bc0a2890 0x7f77bc0a4d50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f77d0103380 tx=0x7f77c0009260 comp rx=0 tx=0).stop 2024-03-16T09:15:07.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.492+0000 7f77d4820700 1 -- 172.21.15.112:0/1996473781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77d0103520 msgr2=0x7f77d00746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:07.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.492+0000 7f77d4820700 1 --2- 172.21.15.112:0/1996473781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77d0103520 0x7f77d00746c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f77b8003930 tx=0x7f77b800f690 comp rx=0 tx=0).stop 2024-03-16T09:15:07.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.492+0000 7f77d4820700 1 -- 172.21.15.112:0/1996473781 shutdown_connections 2024-03-16T09:15:07.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.492+0000 7f77d4820700 1 --2- 172.21.15.112:0/1996473781 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f77bc0a2890 0x7f77bc0a4d50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:07.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.492+0000 7f77d4820700 1 --2- 172.21.15.112:0/1996473781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d0102320 0x7f77d0076070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:07.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.492+0000 7f77d4820700 1 --2- 172.21.15.112:0/1996473781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77d0103ec0 0x7f77d0074c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:07.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.492+0000 7f77d4820700 1 --2- 172.21.15.112:0/1996473781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77d0103520 0x7f77d00746c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:07.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.492+0000 7f77d4820700 1 -- 172.21.15.112:0/1996473781 >> 172.21.15.112:0/1996473781 conn(0x7f77d00fd8e0 msgr2=0x7f77d0106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:07.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.492+0000 7f77d4820700 1 -- 172.21.15.112:0/1996473781 shutdown_connections 2024-03-16T09:15:07.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:07.492+0000 7f77d4820700 1 -- 172.21.15.112:0/1996473781 wait complete. 2024-03-16T09:15:08.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:07 smithi112 ceph-mon[32307]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:15:08.142 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-16T09:15:08.392 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:15:08.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:07 smithi194 ceph-mon[36729]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:15:08.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:07 smithi159 ceph-mon[36739]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:15:09.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:08 smithi112 ceph-mon[32307]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:15:09.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:08 smithi194 ceph-mon[36729]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:15:09.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:08 smithi159 ceph-mon[36739]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:15:10.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.078+0000 7f5f27831700 1 -- 172.21.15.112:0/4137270905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f20103ec0 msgr2=0x7f5f2010a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:10.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.078+0000 7f5f27831700 1 --2- 172.21.15.112:0/4137270905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f20103ec0 0x7f5f2010a770 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0099c0 tx=0x7f5f1c009cd0 comp rx=0 tx=0).stop 2024-03-16T09:15:10.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.078+0000 7f5f27831700 1 -- 172.21.15.112:0/4137270905 shutdown_connections 2024-03-16T09:15:10.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.078+0000 7f5f27831700 1 --2- 172.21.15.112:0/4137270905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f20103ec0 0x7f5f2010a770 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:10.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.078+0000 7f5f27831700 1 --2- 172.21.15.112:0/4137270905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f20102320 0x7f5f20102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:10.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.078+0000 7f5f27831700 1 --2- 172.21.15.112:0/4137270905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f20103520 0x7f5f20103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:10.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.078+0000 7f5f27831700 1 -- 172.21.15.112:0/4137270905 >> 172.21.15.112:0/4137270905 conn(0x7f5f200fd8e0 msgr2=0x7f5f200ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:10.081 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:09 smithi112 ceph-mon[32307]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:15:10.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.078+0000 7f5f27831700 1 -- 172.21.15.112:0/4137270905 shutdown_connections 2024-03-16T09:15:10.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.078+0000 7f5f27831700 1 -- 172.21.15.112:0/4137270905 wait complete. 2024-03-16T09:15:10.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.079+0000 7f5f27831700 1 Processor -- start 2024-03-16T09:15:10.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.079+0000 7f5f27831700 1 -- start start 2024-03-16T09:15:10.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.079+0000 7f5f27831700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f20102320 0x7f5f20198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:10.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.079+0000 7f5f27831700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f20103520 0x7f5f20198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:10.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.079+0000 7f5f27831700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f20103ec0 0x7f5f2019fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:10.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.079+0000 7f5f27831700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2010e010 con 0x7f5f20102320 2024-03-16T09:15:10.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.079+0000 7f5f27831700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2010de90 con 0x7f5f20103ec0 2024-03-16T09:15:10.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.079+0000 7f5f27831700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2010e190 con 0x7f5f20103520 2024-03-16T09:15:10.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.079+0000 7f5f255cd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f20102320 0x7f5f20198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:10.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.079+0000 7f5f255cd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f20102320 0x7f5f20198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38458/0 (socket says 172.21.15.112:38458) 2024-03-16T09:15:10.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.079+0000 7f5f255cd700 1 -- 172.21.15.112:0/1530954459 learned_addr learned my addr 172.21.15.112:0/1530954459 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:10.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.079+0000 7f5f24dcc700 1 --2- 172.21.15.112:0/1530954459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f20103520 0x7f5f20198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:10.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.079+0000 7f5f25dce700 1 --2- 172.21.15.112:0/1530954459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f20103ec0 0x7f5f2019fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:10.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.080+0000 7f5f255cd700 1 -- 172.21.15.112:0/1530954459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f20103520 msgr2=0x7f5f20198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:10.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.080+0000 7f5f255cd700 1 --2- 172.21.15.112:0/1530954459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f20103520 0x7f5f20198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:10.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.080+0000 7f5f255cd700 1 -- 172.21.15.112:0/1530954459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f20103ec0 msgr2=0x7f5f2019fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:10.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.080+0000 7f5f255cd700 1 --2- 172.21.15.112:0/1530954459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f20103ec0 0x7f5f2019fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:10.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.080+0000 7f5f255cd700 1 -- 172.21.15.112:0/1530954459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f201a0160 con 0x7f5f20102320 2024-03-16T09:15:10.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.080+0000 7f5f24dcc700 1 --2- 172.21.15.112:0/1530954459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f20103520 0x7f5f20198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:10.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.080+0000 7f5f25dce700 1 --2- 172.21.15.112:0/1530954459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f20103ec0 0x7f5f2019fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:10.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.080+0000 7f5f255cd700 1 --2- 172.21.15.112:0/1530954459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f20102320 0x7f5f20198500 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009f40 tx=0x7f5f1800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:10.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.080+0000 7f5f167fc700 1 -- 172.21.15.112:0/1530954459 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f180092e0 con 0x7f5f20102320 2024-03-16T09:15:10.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.080+0000 7f5f167fc700 1 -- 172.21.15.112:0/1530954459 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5f1800c9c0 con 0x7f5f20102320 2024-03-16T09:15:10.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.080+0000 7f5f27831700 1 -- 172.21.15.112:0/1530954459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f201a03f0 con 0x7f5f20102320 2024-03-16T09:15:10.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.080+0000 7f5f167fc700 1 -- 172.21.15.112:0/1530954459 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f18004640 con 0x7f5f20102320 2024-03-16T09:15:10.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.080+0000 7f5f27831700 1 -- 172.21.15.112:0/1530954459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f201a0930 con 0x7f5f20102320 2024-03-16T09:15:10.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.081+0000 7f5f27831700 1 -- 172.21.15.112:0/1530954459 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f20068f90 con 0x7f5f20102320 2024-03-16T09:15:10.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.082+0000 7f5f167fc700 1 -- 172.21.15.112:0/1530954459 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f5f1801b030 con 0x7f5f20102320 2024-03-16T09:15:10.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.083+0000 7f5f167fc700 1 --2- 172.21.15.112:0/1530954459 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5f0c0a2890 0x7f5f0c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:10.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.083+0000 7f5f24dcc700 1 --2- 172.21.15.112:0/1530954459 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5f0c0a2890 0x7f5f0c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:10.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.083+0000 7f5f167fc700 1 -- 172.21.15.112:0/1530954459 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f5f180ce890 con 0x7f5f20102320 2024-03-16T09:15:10.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.083+0000 7f5f24dcc700 1 --2- 172.21.15.112:0/1530954459 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5f0c0a2890 0x7f5f0c0a4d50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5f10011490 tx=0x7f5f10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:10.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.087+0000 7f5f167fc700 1 -- 172.21.15.112:0/1530954459 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5f18022020 con 0x7f5f20102320 2024-03-16T09:15:10.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.289+0000 7f5f27831700 1 -- 172.21.15.112:0/1530954459 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f5f200627d0 con 0x7f5f0c0a2890 2024-03-16T09:15:10.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.303+0000 7f5f167fc700 1 -- 172.21.15.112:0/1530954459 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f5f200627d0 con 0x7f5f0c0a2890 2024-03-16T09:15:10.305 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-16T09:15:10.305 INFO:teuthology.orchestra.run.smithi112.stdout:mds.a smithi112 running (102s) 17s ago 101s 13.6M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 5b2e1e3e3a94 2024-03-16T09:15:10.305 INFO:teuthology.orchestra.run.smithi112.stdout:mds.b smithi159 running (99s) 32s ago 98s 25.5M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 404cd1638c02 2024-03-16T09:15:10.305 INFO:teuthology.orchestra.run.smithi112.stdout:mds.c smithi194 running (113s) 32s ago 113s 19.4M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 42b46e581e52 2024-03-16T09:15:10.305 INFO:teuthology.orchestra.run.smithi112.stdout:mds.d smithi112 running (2m) 17s ago 2m 16.2M - 19.0.0-2162-g3d78f5ba be86dc8cb22b efe476354faa 2024-03-16T09:15:10.305 INFO:teuthology.orchestra.run.smithi112.stdout:mds.e smithi159 running (119s) 32s ago 119s 19.2M - 19.0.0-2162-g3d78f5ba be86dc8cb22b f5bbe49efcbb 2024-03-16T09:15:10.305 INFO:teuthology.orchestra.run.smithi112.stdout:mds.f smithi194 running (116s) 32s ago 116s 19.2M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 0b0b811fcf98 2024-03-16T09:15:10.305 INFO:teuthology.orchestra.run.smithi112.stdout:mds.g smithi112 running (105s) 17s ago 104s 16.3M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 32db2fe1a805 2024-03-16T09:15:10.306 INFO:teuthology.orchestra.run.smithi112.stdout:mds.h smithi159 running (110s) 32s ago 110s 17.5M - 19.0.0-2162-g3d78f5ba be86dc8cb22b d9c2c9c80700 2024-03-16T09:15:10.306 INFO:teuthology.orchestra.run.smithi112.stdout:mds.i smithi194 running (108s) 32s ago 107s 16.6M - 19.0.0-2162-g3d78f5ba be86dc8cb22b bfe012df85d8 2024-03-16T09:15:10.306 INFO:teuthology.orchestra.run.smithi112.stdout:mds.j smithi112 running (2m) 17s ago 2m 13.8M - 19.0.0-2162-g3d78f5ba be86dc8cb22b ed78459aba8d 2024-03-16T09:15:10.306 INFO:teuthology.orchestra.run.smithi112.stdout:mds.k smithi159 running (2m) 32s ago 2m 13.1M - 19.0.0-2162-g3d78f5ba be86dc8cb22b c4afbd131895 2024-03-16T09:15:10.306 INFO:teuthology.orchestra.run.smithi112.stdout:mds.l smithi194 running (2m) 32s ago 2m 14.0M - 19.0.0-2162-g3d78f5ba be86dc8cb22b acce808a8f08 2024-03-16T09:15:10.306 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.x smithi112 *:9283,8765 running (9m) 17s ago 9m 513M - 19.0.0-2162-g3d78f5ba be86dc8cb22b f6fd1d82bafd 2024-03-16T09:15:10.306 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.y smithi159 *:8443,8765 running (7m) 32s ago 7m 450M - 19.0.0-2162-g3d78f5ba be86dc8cb22b c35bb3f40b25 2024-03-16T09:15:10.306 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.z smithi194 *:8443,8765 running (7m) 32s ago 7m 453M - 19.0.0-2162-g3d78f5ba be86dc8cb22b d6c887a04c55 2024-03-16T09:15:10.306 INFO:teuthology.orchestra.run.smithi112.stdout:mon.a smithi112 running (9m) 17s ago 9m 49.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b c7d8c6135c17 2024-03-16T09:15:10.306 INFO:teuthology.orchestra.run.smithi112.stdout:mon.b smithi159 running (7m) 32s ago 7m 41.8M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 0bfb0dba10ba 2024-03-16T09:15:10.306 INFO:teuthology.orchestra.run.smithi112.stdout:mon.c smithi194 running (7m) 32s ago 7m 42.4M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b e32ced22c7fb 2024-03-16T09:15:10.306 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (7m) 17s ago 7m 45.0M 4272M 19.0.0-2162-g3d78f5ba be86dc8cb22b d7067641f71f 2024-03-16T09:15:10.306 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi159 running (6m) 32s ago 6m 57.7M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 19f89b4b51d2 2024-03-16T09:15:10.306 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi194 running (6m) 32s ago 6m 59.9M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 5acab86b1c5d 2024-03-16T09:15:10.307 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi112 running (5m) 17s ago 5m 43.5M 4272M 19.0.0-2162-g3d78f5ba be86dc8cb22b 59c37f4a81d0 2024-03-16T09:15:10.307 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi159 running (5m) 32s ago 5m 52.7M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 828aa1d92d65 2024-03-16T09:15:10.307 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi194 running (5m) 32s ago 5m 45.8M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 1a93c2469af9 2024-03-16T09:15:10.307 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 smithi112 running (4m) 17s ago 4m 42.2M 4272M 19.0.0-2162-g3d78f5ba be86dc8cb22b c1310282009d 2024-03-16T09:15:10.307 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 smithi159 running (4m) 32s ago 4m 54.3M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 9d88e43df177 2024-03-16T09:15:10.307 INFO:teuthology.orchestra.run.smithi112.stdout:osd.8 smithi194 running (3m) 32s ago 3m 45.0M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 65ea366ef279 2024-03-16T09:15:10.307 INFO:teuthology.orchestra.run.smithi112.stdout:osd.9 smithi112 running (3m) 17s ago 3m 42.3M 4272M 19.0.0-2162-g3d78f5ba be86dc8cb22b 30e2ed2f1586 2024-03-16T09:15:10.307 INFO:teuthology.orchestra.run.smithi112.stdout:osd.10 smithi159 running (2m) 32s ago 2m 46.1M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 6a402381ea0e 2024-03-16T09:15:10.307 INFO:teuthology.orchestra.run.smithi112.stdout:osd.11 smithi194 running (2m) 32s ago 2m 40.5M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 3e2a934bfd05 2024-03-16T09:15:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.305+0000 7f5f27831700 1 -- 172.21.15.112:0/1530954459 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5f0c0a2890 msgr2=0x7f5f0c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.305+0000 7f5f27831700 1 --2- 172.21.15.112:0/1530954459 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5f0c0a2890 0x7f5f0c0a4d50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5f10011490 tx=0x7f5f10009260 comp rx=0 tx=0).stop 2024-03-16T09:15:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.305+0000 7f5f27831700 1 -- 172.21.15.112:0/1530954459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f20102320 msgr2=0x7f5f20198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.305+0000 7f5f27831700 1 --2- 172.21.15.112:0/1530954459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f20102320 0x7f5f20198500 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009f40 tx=0x7f5f1800c3a0 comp rx=0 tx=0).stop 2024-03-16T09:15:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.305+0000 7f5f27831700 1 -- 172.21.15.112:0/1530954459 shutdown_connections 2024-03-16T09:15:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.305+0000 7f5f27831700 1 --2- 172.21.15.112:0/1530954459 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5f0c0a2890 0x7f5f0c0a4d50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.305+0000 7f5f27831700 1 --2- 172.21.15.112:0/1530954459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f20103ec0 0x7f5f2019fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.305+0000 7f5f27831700 1 --2- 172.21.15.112:0/1530954459 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f20102320 0x7f5f20198500 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:10.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.305+0000 7f5f27831700 1 --2- 172.21.15.112:0/1530954459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f20103520 0x7f5f20198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:10.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.305+0000 7f5f27831700 1 -- 172.21.15.112:0/1530954459 >> 172.21.15.112:0/1530954459 conn(0x7f5f200fd8e0 msgr2=0x7f5f201067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:10.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.305+0000 7f5f27831700 1 -- 172.21.15.112:0/1530954459 shutdown_connections 2024-03-16T09:15:10.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:10.305+0000 7f5f27831700 1 -- 172.21.15.112:0/1530954459 wait complete. 2024-03-16T09:15:10.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:09 smithi194 ceph-mon[36729]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:15:10.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:09 smithi159 ceph-mon[36739]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:15:11.022 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-16T09:15:11.265 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:15:12.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:11 smithi112 ceph-mon[32307]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:15:12.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:11 smithi112 ceph-mon[32307]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T09:15:12.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:11 smithi194 ceph-mon[36729]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:15:12.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:11 smithi194 ceph-mon[36729]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T09:15:12.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:11 smithi159 ceph-mon[36739]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:15:12.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:11 smithi159 ceph-mon[36739]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T09:15:13.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.014+0000 7f6430d4d700 1 -- 172.21.15.112:0/3910044323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f642c102da0 msgr2=0x7f642c109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:13.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.014+0000 7f6430d4d700 1 --2- 172.21.15.112:0/3910044323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f642c102da0 0x7f642c109650 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f642400b0d0 tx=0x7f642400b3e0 comp rx=0 tx=0).stop 2024-03-16T09:15:13.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.015+0000 7f6430d4d700 1 -- 172.21.15.112:0/3910044323 shutdown_connections 2024-03-16T09:15:13.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.015+0000 7f6430d4d700 1 --2- 172.21.15.112:0/3910044323 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f642c102da0 0x7f642c109650 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:13.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.015+0000 7f6430d4d700 1 --2- 172.21.15.112:0/3910044323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f642c101200 0x7f642c101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:13.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.015+0000 7f6430d4d700 1 --2- 172.21.15.112:0/3910044323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f642c102400 0x7f642c102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:13.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.015+0000 7f6430d4d700 1 -- 172.21.15.112:0/3910044323 >> 172.21.15.112:0/3910044323 conn(0x7f642c0fc7a0 msgr2=0x7f642c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:13.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.015+0000 7f6430d4d700 1 -- 172.21.15.112:0/3910044323 shutdown_connections 2024-03-16T09:15:13.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.015+0000 7f6430d4d700 1 -- 172.21.15.112:0/3910044323 wait complete. 2024-03-16T09:15:13.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.015+0000 7f6430d4d700 1 Processor -- start 2024-03-16T09:15:13.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.015+0000 7f6430d4d700 1 -- start start 2024-03-16T09:15:13.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f6430d4d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f642c101200 0x7f642c06ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:13.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f6430d4d700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f642c102400 0x7f642c19d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:13.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f6430d4d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f642c102da0 0x7f642c19f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:13.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f6430d4d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f642c10cf80 con 0x7f642c101200 2024-03-16T09:15:13.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f6430d4d700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f642c10ce00 con 0x7f642c102400 2024-03-16T09:15:13.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f6430d4d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f642c10d100 con 0x7f642c102da0 2024-03-16T09:15:13.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f642a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f642c101200 0x7f642c06ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:13.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f642a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f642c101200 0x7f642c06ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48204/0 (socket says 172.21.15.112:48204) 2024-03-16T09:15:13.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f642a59c700 1 -- 172.21.15.112:0/2599442204 learned_addr learned my addr 172.21.15.112:0/2599442204 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:13.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f6429d9b700 1 --2- 172.21.15.112:0/2599442204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f642c102400 0x7f642c19d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:13.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f642ad9d700 1 --2- 172.21.15.112:0/2599442204 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f642c102da0 0x7f642c19f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:13.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f642a59c700 1 -- 172.21.15.112:0/2599442204 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f642c102da0 msgr2=0x7f642c19f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:13.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f642a59c700 1 --2- 172.21.15.112:0/2599442204 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f642c102da0 0x7f642c19f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:13.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f642a59c700 1 -- 172.21.15.112:0/2599442204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f642c102400 msgr2=0x7f642c19d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:13.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f642a59c700 1 --2- 172.21.15.112:0/2599442204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f642c102400 0x7f642c19d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:13.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f642a59c700 1 -- 172.21.15.112:0/2599442204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642c1a0210 con 0x7f642c101200 2024-03-16T09:15:13.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f642a59c700 1 --2- 172.21.15.112:0/2599442204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f642c101200 0x7f642c06ab00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f641c00e950 tx=0x7f641c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:13.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f641b7fe700 1 -- 172.21.15.112:0/2599442204 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c004230 con 0x7f642c101200 2024-03-16T09:15:13.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f641b7fe700 1 -- 172.21.15.112:0/2599442204 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f641c0043d0 con 0x7f642c101200 2024-03-16T09:15:13.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f6430d4d700 1 -- 172.21.15.112:0/2599442204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f642c1a0500 con 0x7f642c101200 2024-03-16T09:15:13.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f641b7fe700 1 -- 172.21.15.112:0/2599442204 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c010430 con 0x7f642c101200 2024-03-16T09:15:13.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.016+0000 7f6430d4d700 1 -- 172.21.15.112:0/2599442204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f642c1a09f0 con 0x7f642c101200 2024-03-16T09:15:13.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.017+0000 7f6430d4d700 1 -- 172.21.15.112:0/2599442204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f642c068f90 con 0x7f642c101200 2024-03-16T09:15:13.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.018+0000 7f641b7fe700 1 -- 172.21.15.112:0/2599442204 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f641c00c7e0 con 0x7f642c101200 2024-03-16T09:15:13.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.019+0000 7f641b7fe700 1 --2- 172.21.15.112:0/2599442204 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f64140a2890 0x7f64140a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:13.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.019+0000 7f6429d9b700 1 --2- 172.21.15.112:0/2599442204 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f64140a2890 0x7f64140a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:13.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.019+0000 7f641b7fe700 1 -- 172.21.15.112:0/2599442204 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f641c0cec70 con 0x7f642c101200 2024-03-16T09:15:13.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.019+0000 7f6429d9b700 1 --2- 172.21.15.112:0/2599442204 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f64140a2890 0x7f64140a4d50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f642c06b060 tx=0x7f64200073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:13.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.023+0000 7f641b7fe700 1 -- 172.21.15.112:0/2599442204 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f641c09bd20 con 0x7f642c101200 2024-03-16T09:15:13.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.230+0000 7f6430d4d700 1 -- 172.21.15.112:0/2599442204 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f642c062710 con 0x7f64140a2890 2024-03-16T09:15:13.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.236+0000 7f641b7fe700 1 -- 172.21.15.112:0/2599442204 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f642c062710 con 0x7f64140a2890 2024-03-16T09:15:13.239 INFO:teuthology.orchestra.run.smithi112.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-16T09:15:13.239 INFO:teuthology.orchestra.run.smithi112.stdout:mds.all 12/12 35s ago 2m smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2024-03-16T09:15:13.239 INFO:teuthology.orchestra.run.smithi112.stdout:mgr 3/3 35s ago 7m smithi112=x;smithi159=y;smithi194=z;count:3 2024-03-16T09:15:13.239 INFO:teuthology.orchestra.run.smithi112.stdout:mon 3/3 35s ago 8m smithi112:172.21.15.112=a;smithi159:172.21.15.159=b;smithi194:172.21.15.194=c;count:3 2024-03-16T09:15:13.239 INFO:teuthology.orchestra.run.smithi112.stdout:osd 12 35s ago - 2024-03-16T09:15:13.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.238+0000 7f6430d4d700 1 -- 172.21.15.112:0/2599442204 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f64140a2890 msgr2=0x7f64140a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:13.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.238+0000 7f6430d4d700 1 --2- 172.21.15.112:0/2599442204 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f64140a2890 0x7f64140a4d50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f642c06b060 tx=0x7f64200073d0 comp rx=0 tx=0).stop 2024-03-16T09:15:13.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.238+0000 7f6430d4d700 1 -- 172.21.15.112:0/2599442204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f642c101200 msgr2=0x7f642c06ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:13.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.238+0000 7f6430d4d700 1 --2- 172.21.15.112:0/2599442204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f642c101200 0x7f642c06ab00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f641c00e950 tx=0x7f641c00ec60 comp rx=0 tx=0).stop 2024-03-16T09:15:13.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.238+0000 7f6430d4d700 1 -- 172.21.15.112:0/2599442204 shutdown_connections 2024-03-16T09:15:13.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.238+0000 7f6430d4d700 1 --2- 172.21.15.112:0/2599442204 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f64140a2890 0x7f64140a4d50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:13.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.238+0000 7f6430d4d700 1 --2- 172.21.15.112:0/2599442204 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f642c102da0 0x7f642c19f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:13.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.238+0000 7f6430d4d700 1 --2- 172.21.15.112:0/2599442204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f642c101200 0x7f642c06ab00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:13.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.238+0000 7f6430d4d700 1 --2- 172.21.15.112:0/2599442204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f642c102400 0x7f642c19d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:13.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.238+0000 7f6430d4d700 1 -- 172.21.15.112:0/2599442204 >> 172.21.15.112:0/2599442204 conn(0x7f642c0fc7a0 msgr2=0x7f642c105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:13.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.238+0000 7f6430d4d700 1 -- 172.21.15.112:0/2599442204 shutdown_connections 2024-03-16T09:15:13.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:13.238+0000 7f6430d4d700 1 -- 172.21.15.112:0/2599442204 wait complete. 2024-03-16T09:15:13.964 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-16T09:15:14.203 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:15:14.324 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:13 smithi112 ceph-mon[32307]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T09:15:14.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:13 smithi194 ceph-mon[36729]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T09:15:14.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:13 smithi159 ceph-mon[36739]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T09:15:15.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:15 smithi112 ceph-mon[32307]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:15:15.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:15 smithi194 ceph-mon[36729]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:15:15.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:15 smithi159 ceph-mon[36739]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:15:16.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.056+0000 7f03b499f700 1 -- 172.21.15.112:0/1605096344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03b0102420 msgr2=0x7f03b0102880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:16.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.056+0000 7f03b499f700 1 --2- 172.21.15.112:0/1605096344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03b0102420 0x7f03b0102880 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f03980099c0 tx=0x7f0398009cd0 comp rx=0 tx=0).stop 2024-03-16T09:15:16.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.056+0000 7f03b499f700 1 -- 172.21.15.112:0/1605096344 shutdown_connections 2024-03-16T09:15:16.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.056+0000 7f03b499f700 1 --2- 172.21.15.112:0/1605096344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03b0101220 0x7f03b0101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:16.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.056+0000 7f03b499f700 1 --2- 172.21.15.112:0/1605096344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03b0102dc0 0x7f03b0109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:16.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.056+0000 7f03b499f700 1 --2- 172.21.15.112:0/1605096344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03b0102420 0x7f03b0102880 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:16.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.056+0000 7f03b499f700 1 -- 172.21.15.112:0/1605096344 >> 172.21.15.112:0/1605096344 conn(0x7f03b00fc7c0 msgr2=0x7f03b00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:16.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.056+0000 7f03b499f700 1 -- 172.21.15.112:0/1605096344 shutdown_connections 2024-03-16T09:15:16.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.056+0000 7f03b499f700 1 -- 172.21.15.112:0/1605096344 wait complete. 2024-03-16T09:15:16.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.057+0000 7f03b499f700 1 Processor -- start 2024-03-16T09:15:16.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.057+0000 7f03b499f700 1 -- start start 2024-03-16T09:15:16.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.057+0000 7f03b499f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03b0101220 0x7f03b01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:16.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.057+0000 7f03b499f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03b0102420 0x7f03b0198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:16.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.057+0000 7f03b499f700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03b0102dc0 0x7f03b019fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:16.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.057+0000 7f03b499f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f03b010cf50 con 0x7f03b0101220 2024-03-16T09:15:16.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.057+0000 7f03b499f700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f03b010cdd0 con 0x7f03b0102dc0 2024-03-16T09:15:16.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.057+0000 7f03b499f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f03b010d0d0 con 0x7f03b0102420 2024-03-16T09:15:16.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.057+0000 7f03ae59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03b0101220 0x7f03b01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:16.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.057+0000 7f03ae59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03b0101220 0x7f03b01985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48224/0 (socket says 172.21.15.112:48224) 2024-03-16T09:15:16.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.057+0000 7f03ae59c700 1 -- 172.21.15.112:0/1349817938 learned_addr learned my addr 172.21.15.112:0/1349817938 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:16.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.057+0000 7f03add9b700 1 --2- 172.21.15.112:0/1349817938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03b0102420 0x7f03b0198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:16.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.057+0000 7f03aed9d700 1 --2- 172.21.15.112:0/1349817938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03b0102dc0 0x7f03b019fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:16.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.057+0000 7f03ae59c700 1 -- 172.21.15.112:0/1349817938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03b0102420 msgr2=0x7f03b0198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:16.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.057+0000 7f03ae59c700 1 --2- 172.21.15.112:0/1349817938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03b0102420 0x7f03b0198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:16.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.057+0000 7f03ae59c700 1 -- 172.21.15.112:0/1349817938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03b0102dc0 msgr2=0x7f03b019fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:16.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.057+0000 7f03ae59c700 1 --2- 172.21.15.112:0/1349817938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03b0102dc0 0x7f03b019fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:16.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.057+0000 7f03ae59c700 1 -- 172.21.15.112:0/1349817938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b01a0200 con 0x7f03b0101220 2024-03-16T09:15:16.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.058+0000 7f03ae59c700 1 --2- 172.21.15.112:0/1349817938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03b0101220 0x7f03b01985a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009f40 tx=0x7f03a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:16.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.058+0000 7f03a77fe700 1 -- 172.21.15.112:0/1349817938 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a80092e0 con 0x7f03b0101220 2024-03-16T09:15:16.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.058+0000 7f03a77fe700 1 -- 172.21.15.112:0/1349817938 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f03a800c9c0 con 0x7f03b0101220 2024-03-16T09:15:16.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.058+0000 7f03a77fe700 1 -- 172.21.15.112:0/1349817938 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a8004640 con 0x7f03b0101220 2024-03-16T09:15:16.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.058+0000 7f03b499f700 1 -- 172.21.15.112:0/1349817938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b01a0490 con 0x7f03b0101220 2024-03-16T09:15:16.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.058+0000 7f03b499f700 1 -- 172.21.15.112:0/1349817938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b01a09d0 con 0x7f03b0101220 2024-03-16T09:15:16.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.059+0000 7f03b499f700 1 -- 172.21.15.112:0/1349817938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b0068f90 con 0x7f03b0101220 2024-03-16T09:15:16.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.060+0000 7f03a77fe700 1 -- 172.21.15.112:0/1349817938 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f03a801b030 con 0x7f03b0101220 2024-03-16T09:15:16.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.060+0000 7f03a77fe700 1 --2- 172.21.15.112:0/1349817938 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f039c0a2890 0x7f039c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:16.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.060+0000 7f03add9b700 1 --2- 172.21.15.112:0/1349817938 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f039c0a2890 0x7f039c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:16.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.061+0000 7f03a77fe700 1 -- 172.21.15.112:0/1349817938 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f03a80cf2a0 con 0x7f03b0101220 2024-03-16T09:15:16.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.061+0000 7f03add9b700 1 --2- 172.21.15.112:0/1349817938 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f039c0a2890 0x7f039c0a4d50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0398000c00 tx=0x7f0398005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:16.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.064+0000 7f03a77fe700 1 -- 172.21.15.112:0/1349817938 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f03a8022020 con 0x7f03b0101220 2024-03-16T09:15:16.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.265+0000 7f03b499f700 1 -- 172.21.15.112:0/1349817938 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f03b00fe930 con 0x7f039c0a2890 2024-03-16T09:15:16.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:16 smithi112 ceph-mon[32307]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-03-16T09:15:16.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.267+0000 7f03a77fe700 1 -- 172.21.15.112:0/1349817938 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f03b00fe930 con 0x7f039c0a2890 2024-03-16T09:15:16.270 INFO:teuthology.orchestra.run.smithi112.stdout:HOST ADDR LABELS STATUS 2024-03-16T09:15:16.270 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112 172.21.15.112 2024-03-16T09:15:16.270 INFO:teuthology.orchestra.run.smithi112.stdout:smithi159 172.21.15.159 2024-03-16T09:15:16.270 INFO:teuthology.orchestra.run.smithi112.stdout:smithi194 172.21.15.194 2024-03-16T09:15:16.270 INFO:teuthology.orchestra.run.smithi112.stdout:3 hosts in cluster 2024-03-16T09:15:16.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.269+0000 7f03b499f700 1 -- 172.21.15.112:0/1349817938 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f039c0a2890 msgr2=0x7f039c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:16.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.269+0000 7f03b499f700 1 --2- 172.21.15.112:0/1349817938 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f039c0a2890 0x7f039c0a4d50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0398000c00 tx=0x7f0398005fb0 comp rx=0 tx=0).stop 2024-03-16T09:15:16.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.269+0000 7f03b499f700 1 -- 172.21.15.112:0/1349817938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03b0101220 msgr2=0x7f03b01985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:16.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.269+0000 7f03b499f700 1 --2- 172.21.15.112:0/1349817938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03b0101220 0x7f03b01985a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009f40 tx=0x7f03a800c3a0 comp rx=0 tx=0).stop 2024-03-16T09:15:16.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.269+0000 7f03b499f700 1 -- 172.21.15.112:0/1349817938 shutdown_connections 2024-03-16T09:15:16.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.269+0000 7f03b499f700 1 --2- 172.21.15.112:0/1349817938 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f039c0a2890 0x7f039c0a4d50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:16.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.269+0000 7f03b499f700 1 --2- 172.21.15.112:0/1349817938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03b0102dc0 0x7f03b019fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:16.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.269+0000 7f03b499f700 1 --2- 172.21.15.112:0/1349817938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03b0101220 0x7f03b01985a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:16.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.269+0000 7f03b499f700 1 --2- 172.21.15.112:0/1349817938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03b0102420 0x7f03b0198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:16.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.269+0000 7f03b499f700 1 -- 172.21.15.112:0/1349817938 >> 172.21.15.112:0/1349817938 conn(0x7f03b00fc7c0 msgr2=0x7f03b00fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:16.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.269+0000 7f03b499f700 1 -- 172.21.15.112:0/1349817938 shutdown_connections 2024-03-16T09:15:16.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:16.269+0000 7f03b499f700 1 -- 172.21.15.112:0/1349817938 wait complete. 2024-03-16T09:15:16.387 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:16 smithi159 ceph-mon[36739]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-03-16T09:15:16.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:16 smithi194 ceph-mon[36729]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-03-16T09:15:16.919 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-16T09:15:17.162 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:15:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:18 smithi112 ceph-mon[32307]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:15:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:18 smithi112 ceph-mon[32307]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-03-16T09:15:18.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:18 smithi194 ceph-mon[36729]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:15:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:18 smithi194 ceph-mon[36729]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-03-16T09:15:18.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:18 smithi159 ceph-mon[36739]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:15:18.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:18 smithi159 ceph-mon[36739]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-03-16T09:15:18.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.897+0000 7f2b11672700 1 -- 172.21.15.112:0/753971812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c100b80 msgr2=0x7f2b0c102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:18.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.897+0000 7f2b11672700 1 --2- 172.21.15.112:0/753971812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c100b80 0x7f2b0c102fb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009990 tx=0x7f2afc009ca0 comp rx=0 tx=0).stop 2024-03-16T09:15:18.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.898+0000 7f2b11672700 1 -- 172.21.15.112:0/753971812 shutdown_connections 2024-03-16T09:15:18.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.898+0000 7f2b11672700 1 --2- 172.21.15.112:0/753971812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b0c105ec0 0x7f2b0c106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:18.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.898+0000 7f2b11672700 1 --2- 172.21.15.112:0/753971812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c100b80 0x7f2b0c102fb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:18.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.898+0000 7f2b11672700 1 --2- 172.21.15.112:0/753971812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b0c1034f0 0x7f2b0c105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:18.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.898+0000 7f2b11672700 1 -- 172.21.15.112:0/753971812 >> 172.21.15.112:0/753971812 conn(0x7f2b0c0fc7c0 msgr2=0x7f2b0c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:18.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.898+0000 7f2b11672700 1 -- 172.21.15.112:0/753971812 shutdown_connections 2024-03-16T09:15:18.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.898+0000 7f2b11672700 1 -- 172.21.15.112:0/753971812 wait complete. 2024-03-16T09:15:18.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.898+0000 7f2b11672700 1 Processor -- start 2024-03-16T09:15:18.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.898+0000 7f2b11672700 1 -- start start 2024-03-16T09:15:18.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.898+0000 7f2b11672700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b0c100b80 0x7f2b0c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:18.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b11672700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b0c1034f0 0x7f2b0c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:18.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b11672700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c105ec0 0x7f2b0c1a6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:18.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b11672700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c114580 con 0x7f2b0c105ec0 2024-03-16T09:15:18.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b11672700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c114400 con 0x7f2b0c100b80 2024-03-16T09:15:18.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b11672700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c114700 con 0x7f2b0c1034f0 2024-03-16T09:15:18.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b0b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c105ec0 0x7f2b0c1a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:18.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b0b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c105ec0 0x7f2b0c1a6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48228/0 (socket says 172.21.15.112:48228) 2024-03-16T09:15:18.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b0b7fe700 1 -- 172.21.15.112:0/2907174274 learned_addr learned my addr 172.21.15.112:0/2907174274 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:18.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b0affd700 1 --2- 172.21.15.112:0/2907174274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b0c100b80 0x7f2b0c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:18.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b027fc700 1 --2- 172.21.15.112:0/2907174274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b0c1034f0 0x7f2b0c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:18.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b0b7fe700 1 -- 172.21.15.112:0/2907174274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b0c1034f0 msgr2=0x7f2b0c19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:18.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b0b7fe700 1 --2- 172.21.15.112:0/2907174274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b0c1034f0 0x7f2b0c19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:18.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b0b7fe700 1 -- 172.21.15.112:0/2907174274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b0c100b80 msgr2=0x7f2b0c19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:18.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b0b7fe700 1 --2- 172.21.15.112:0/2907174274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b0c100b80 0x7f2b0c19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:18.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b0b7fe700 1 -- 172.21.15.112:0/2907174274 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0c1a67c0 con 0x7f2b0c105ec0 2024-03-16T09:15:18.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b0b7fe700 1 --2- 172.21.15.112:0/2907174274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c105ec0 0x7f2b0c1a6120 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400ef60 tx=0x7f2b0400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:18.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b08ff9700 1 -- 172.21.15.112:0/2907174274 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b040092e0 con 0x7f2b0c105ec0 2024-03-16T09:15:18.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b11672700 1 -- 172.21.15.112:0/2907174274 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0c1a6a50 con 0x7f2b0c105ec0 2024-03-16T09:15:18.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b08ff9700 1 -- 172.21.15.112:0/2907174274 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2b04004bb0 con 0x7f2b0c105ec0 2024-03-16T09:15:18.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b08ff9700 1 -- 172.21.15.112:0/2907174274 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04004280 con 0x7f2b0c105ec0 2024-03-16T09:15:18.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.899+0000 7f2b11672700 1 -- 172.21.15.112:0/2907174274 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b0c1a6f90 con 0x7f2b0c105ec0 2024-03-16T09:15:18.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.900+0000 7f2b11672700 1 -- 172.21.15.112:0/2907174274 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b0c068f90 con 0x7f2b0c105ec0 2024-03-16T09:15:18.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.901+0000 7f2b08ff9700 1 -- 172.21.15.112:0/2907174274 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f2b0401b030 con 0x7f2b0c105ec0 2024-03-16T09:15:18.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.902+0000 7f2b08ff9700 1 --2- 172.21.15.112:0/2907174274 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2af80a2890 0x7f2af80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:18.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.902+0000 7f2b0affd700 1 --2- 172.21.15.112:0/2907174274 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2af80a2890 0x7f2af80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:18.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.902+0000 7f2b08ff9700 1 -- 172.21.15.112:0/2907174274 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f2b040ce4a0 con 0x7f2b0c105ec0 2024-03-16T09:15:18.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.902+0000 7f2b0affd700 1 --2- 172.21.15.112:0/2907174274 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2af80a2890 0x7f2af80a4d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00b2e0 tx=0x7f2afc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:18.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:18.906+0000 7f2b08ff9700 1 -- 172.21.15.112:0/2907174274 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2b04022020 con 0x7f2b0c105ec0 2024-03-16T09:15:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:19.114+0000 7f2b11672700 1 -- 172.21.15.112:0/2907174274 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f2b0c0627d0 con 0x7f2af80a2890 2024-03-16T09:15:19.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:19.116+0000 7f2b08ff9700 1 -- 172.21.15.112:0/2907174274 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f2b0c0627d0 con 0x7f2af80a2890 2024-03-16T09:15:19.119 INFO:teuthology.orchestra.run.smithi112.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-16T09:15:19.119 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NV400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-16T09:15:19.119 INFO:teuthology.orchestra.run.smithi112.stdout:smithi159 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GH400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-16T09:15:19.119 INFO:teuthology.orchestra.run.smithi112.stdout:smithi194 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6240002A400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-16T09:15:19.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:19.118+0000 7f2b11672700 1 -- 172.21.15.112:0/2907174274 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2af80a2890 msgr2=0x7f2af80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:19.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:19.118+0000 7f2b11672700 1 --2- 172.21.15.112:0/2907174274 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2af80a2890 0x7f2af80a4d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00b2e0 tx=0x7f2afc005fd0 comp rx=0 tx=0).stop 2024-03-16T09:15:19.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:19.118+0000 7f2b11672700 1 -- 172.21.15.112:0/2907174274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c105ec0 msgr2=0x7f2b0c1a6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:19.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:19.118+0000 7f2b11672700 1 --2- 172.21.15.112:0/2907174274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c105ec0 0x7f2b0c1a6120 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400ef60 tx=0x7f2b0400ef90 comp rx=0 tx=0).stop 2024-03-16T09:15:19.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:19.118+0000 7f2b11672700 1 -- 172.21.15.112:0/2907174274 shutdown_connections 2024-03-16T09:15:19.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:19.118+0000 7f2b11672700 1 --2- 172.21.15.112:0/2907174274 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2af80a2890 0x7f2af80a4d50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:19.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:19.118+0000 7f2b11672700 1 --2- 172.21.15.112:0/2907174274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b0c100b80 0x7f2b0c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:19.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:19.118+0000 7f2b11672700 1 --2- 172.21.15.112:0/2907174274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b0c105ec0 0x7f2b0c1a6120 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:19.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:19.118+0000 7f2b11672700 1 --2- 172.21.15.112:0/2907174274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2b0c1034f0 0x7f2b0c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:19.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:19.118+0000 7f2b11672700 1 -- 172.21.15.112:0/2907174274 >> 172.21.15.112:0/2907174274 conn(0x7f2b0c0fc7c0 msgr2=0x7f2b0c104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:19.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:19.119+0000 7f2b11672700 1 -- 172.21.15.112:0/2907174274 shutdown_connections 2024-03-16T09:15:19.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:19.119+0000 7f2b11672700 1 -- 172.21.15.112:0/2907174274 wait complete. 2024-03-16T09:15:19.794 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-16T09:15:19.803 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-03-16T09:15:19.803 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-16T09:15:20.045 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:15:20.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:20 smithi112 ceph-mon[32307]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T09:15:20.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:20 smithi194 ceph-mon[36729]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T09:15:20.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:20 smithi159 ceph-mon[36739]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T09:15:21.177 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:21 smithi112 ceph-mon[32307]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:15:21.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:21 smithi194 ceph-mon[36729]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:15:21.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:21 smithi159 ceph-mon[36739]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:15:21.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.741+0000 7f9be38ee700 1 -- 172.21.15.112:0/4203797841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bdc103660 msgr2=0x7f9bdc103ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:21.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.741+0000 7f9be38ee700 1 --2- 172.21.15.112:0/4203797841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bdc103660 0x7f9bdc103ac0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc0099c0 tx=0x7f9bcc009cd0 comp rx=0 tx=0).stop 2024-03-16T09:15:21.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.741+0000 7f9be38ee700 1 -- 172.21.15.112:0/4203797841 shutdown_connections 2024-03-16T09:15:21.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.741+0000 7f9be38ee700 1 --2- 172.21.15.112:0/4203797841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bdc104000 0x7f9bdc10a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:21.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.741+0000 7f9be38ee700 1 --2- 172.21.15.112:0/4203797841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bdc1005e0 0x7f9bdc100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:21.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.741+0000 7f9be38ee700 1 --2- 172.21.15.112:0/4203797841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bdc103660 0x7f9bdc103ac0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:21.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.741+0000 7f9be38ee700 1 -- 172.21.15.112:0/4203797841 >> 172.21.15.112:0/4203797841 conn(0x7f9bdc0fbf40 msgr2=0x7f9bdc0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:21.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.741+0000 7f9be38ee700 1 -- 172.21.15.112:0/4203797841 shutdown_connections 2024-03-16T09:15:21.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.741+0000 7f9be38ee700 1 -- 172.21.15.112:0/4203797841 wait complete. 2024-03-16T09:15:21.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.741+0000 7f9be38ee700 1 Processor -- start 2024-03-16T09:15:21.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.741+0000 7f9be38ee700 1 -- start start 2024-03-16T09:15:21.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.741+0000 7f9be38ee700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bdc1005e0 0x7f9bdc102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:21.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.741+0000 7f9be38ee700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bdc103660 0x7f9bdc101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:21.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.741+0000 7f9be38ee700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bdc104000 0x7f9bdc101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:21.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.742+0000 7f9be38ee700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc10e010 con 0x7f9bdc104000 2024-03-16T09:15:21.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.742+0000 7f9be38ee700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc10de90 con 0x7f9bdc103660 2024-03-16T09:15:21.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.742+0000 7f9be38ee700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc10e190 con 0x7f9bdc1005e0 2024-03-16T09:15:21.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.742+0000 7f9be1e8b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bdc104000 0x7f9bdc101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:21.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.742+0000 7f9be1e8b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bdc104000 0x7f9bdc101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57406/0 (socket says 172.21.15.112:57406) 2024-03-16T09:15:21.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.742+0000 7f9be1e8b700 1 -- 172.21.15.112:0/100389470 learned_addr learned my addr 172.21.15.112:0/100389470 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:21.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.742+0000 7f9be168a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bdc1005e0 0x7f9bdc102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:21.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.742+0000 7f9be0e89700 1 --2- 172.21.15.112:0/100389470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bdc103660 0x7f9bdc101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:21.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.742+0000 7f9be168a700 1 -- 172.21.15.112:0/100389470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bdc103660 msgr2=0x7f9bdc101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:21.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.742+0000 7f9be168a700 1 --2- 172.21.15.112:0/100389470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bdc103660 0x7f9bdc101610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:21.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.742+0000 7f9be168a700 1 -- 172.21.15.112:0/100389470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bdc104000 msgr2=0x7f9bdc101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:21.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.742+0000 7f9be168a700 1 --2- 172.21.15.112:0/100389470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bdc104000 0x7f9bdc101b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:21.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.742+0000 7f9be168a700 1 -- 172.21.15.112:0/100389470 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bdc102400 con 0x7f9bdc1005e0 2024-03-16T09:15:21.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.742+0000 7f9be168a700 1 --2- 172.21.15.112:0/100389470 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bdc1005e0 0x7f9bdc102fc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9bd800e9c0 tx=0x7f9bd800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:21.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.742+0000 7f9bd27fc700 1 -- 172.21.15.112:0/100389470 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8004230 con 0x7f9bdc1005e0 2024-03-16T09:15:21.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.742+0000 7f9bd27fc700 1 -- 172.21.15.112:0/100389470 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9bd80043d0 con 0x7f9bdc1005e0 2024-03-16T09:15:21.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.742+0000 7f9be38ee700 1 -- 172.21.15.112:0/100389470 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bdc1a2770 con 0x7f9bdc1005e0 2024-03-16T09:15:21.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.742+0000 7f9bd27fc700 1 -- 172.21.15.112:0/100389470 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8010430 con 0x7f9bdc1005e0 2024-03-16T09:15:21.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.742+0000 7f9be38ee700 1 -- 172.21.15.112:0/100389470 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bdc1a2c30 con 0x7f9bdc1005e0 2024-03-16T09:15:21.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.743+0000 7f9be38ee700 1 -- 172.21.15.112:0/100389470 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bdc068f90 con 0x7f9bdc1005e0 2024-03-16T09:15:21.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.744+0000 7f9bd27fc700 1 -- 172.21.15.112:0/100389470 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f9bd800c7c0 con 0x7f9bdc1005e0 2024-03-16T09:15:21.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.745+0000 7f9bd27fc700 1 --2- 172.21.15.112:0/100389470 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9bc80a2890 0x7f9bc80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:21.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.745+0000 7f9be0e89700 1 --2- 172.21.15.112:0/100389470 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9bc80a2890 0x7f9bc80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:21.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.745+0000 7f9bd27fc700 1 -- 172.21.15.112:0/100389470 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f9bd80ced50 con 0x7f9bdc1005e0 2024-03-16T09:15:21.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.745+0000 7f9be0e89700 1 --2- 172.21.15.112:0/100389470 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9bc80a2890 0x7f9bc80a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc000c00 tx=0x7f9bcc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:21.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.749+0000 7f9bd27fc700 1 -- 172.21.15.112:0/100389470 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9bd809bda0 con 0x7f9bdc1005e0 2024-03-16T09:15:22.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:21.999+0000 7f9be38ee700 1 -- 172.21.15.112:0/100389470 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f9bdc062ee0 con 0x7f9bdc1005e0 2024-03-16T09:15:22.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:22.000+0000 7f9bd27fc700 1 -- 172.21.15.112:0/100389470 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) ==== 76+0+3056 (secure 0 0 0) 0x7f9bd809b6a0 con 0x7f9bdc1005e0 2024-03-16T09:15:22.003 INFO:teuthology.orchestra.run.smithi112.stdout:e21 2024-03-16T09:15:22.003 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-16T09:15:22.003 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-16T09:15:22.003 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-03-16T09:15:22.003 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:22.003 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-03-16T09:15:22.004 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-03-16T09:15:22.004 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 21 2024-03-16T09:15:22.004 INFO:teuthology.orchestra.run.smithi112.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-16T09:15:22.004 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-03-16T09:13:36.450216+0000 2024-03-16T09:15:22.004 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-03-16T09:13:40.527486+0000 2024-03-16T09:15:22.004 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-03-16T09:15:22.004 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-03-16T09:15:22.004 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 300 2024-03-16T09:15:22.004 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-03-16T09:15:22.004 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-03-16T09:15:22.004 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-03-16T09:15:22.005 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-03-16T09:15:22.005 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-03-16T09:15:22.005 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 0 2024-03-16T09:15:22.005 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-16T09:15:22.005 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 3 2024-03-16T09:15:22.005 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1,2 2024-03-16T09:15:22.006 INFO:teuthology.orchestra.run.smithi112.stdout:up {0=24494,1=24466,2=24452} 2024-03-16T09:15:22.006 INFO:teuthology.orchestra.run.smithi112.stdout:failed 2024-03-16T09:15:22.006 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-03-16T09:15:22.006 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-03-16T09:15:22.007 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-03-16T09:15:22.007 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-03-16T09:15:22.007 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data disabled 2024-03-16T09:15:22.007 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-03-16T09:15:22.007 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-03-16T09:15:22.007 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 0 2024-03-16T09:15:22.007 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 24494 members: 24452,24466,24494 2024-03-16T09:15:22.008 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b{0:24494} state up:active seq 3 addr [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:15:22.008 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c{0:24448} state up:standby-replay seq 1 addr [v2:172.21.15.194:6836/1682076,v1:172.21.15.194:6837/1682076] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:15:22.008 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.i{1:24466} state up:active seq 6 addr [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:15:22.008 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.f{1:24436} state up:standby-replay seq 1 addr [v2:172.21.15.194:6834/2682964784,v1:172.21.15.194:6835/2682964784] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:15:22.008 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.h{2:24452} state up:active seq 7 addr [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:15:22.008 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.e{2:24413} state up:standby-replay seq 1 addr [v2:172.21.15.159:6834/567707106,v1:172.21.15.159:6835/567707106] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:15:22.009 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:22.009 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:22.009 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-03-16T09:15:22.010 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:22.010 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.j{-1:14526} state up:standby seq 1 addr [v2:172.21.15.112:6834/1370033819,v1:172.21.15.112:6835/1370033819] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:15:22.010 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d{-1:14538} state up:standby seq 1 addr [v2:172.21.15.112:6836/1353236253,v1:172.21.15.112:6837/1353236253] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:15:22.010 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.g{-1:14601} state up:standby seq 1 addr [v2:172.21.15.112:6838/3939748854,v1:172.21.15.112:6839/3939748854] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:15:22.010 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a{-1:14613} state up:standby seq 1 addr [v2:172.21.15.112:6840/3118523258,v1:172.21.15.112:6841/3118523258] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:15:22.011 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.k{-1:24386} state up:standby seq 1 addr [v2:172.21.15.159:6832/2947166433,v1:172.21.15.159:6833/2947166433] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:15:22.011 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.l{-1:24397} state up:standby seq 1 addr [v2:172.21.15.194:6832/1514670407,v1:172.21.15.194:6833/1514670407] compat {c=[1],r=[1],i=[1fff]}] 2024-03-16T09:15:22.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:22.002+0000 7f9be38ee700 1 -- 172.21.15.112:0/100389470 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9bc80a2890 msgr2=0x7f9bc80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:22.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:22.002+0000 7f9be38ee700 1 --2- 172.21.15.112:0/100389470 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9bc80a2890 0x7f9bc80a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc000c00 tx=0x7f9bcc005fb0 comp rx=0 tx=0).stop 2024-03-16T09:15:22.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:22.002+0000 7f9be38ee700 1 -- 172.21.15.112:0/100389470 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bdc1005e0 msgr2=0x7f9bdc102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:22.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:22.002+0000 7f9be38ee700 1 --2- 172.21.15.112:0/100389470 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bdc1005e0 0x7f9bdc102fc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9bd800e9c0 tx=0x7f9bd800ecd0 comp rx=0 tx=0).stop 2024-03-16T09:15:22.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:22.002+0000 7f9be38ee700 1 -- 172.21.15.112:0/100389470 shutdown_connections 2024-03-16T09:15:22.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:22.002+0000 7f9be38ee700 1 --2- 172.21.15.112:0/100389470 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9bc80a2890 0x7f9bc80a4d50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:22.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:22.002+0000 7f9be38ee700 1 --2- 172.21.15.112:0/100389470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9bdc104000 0x7f9bdc101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:22.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:22.002+0000 7f9be38ee700 1 --2- 172.21.15.112:0/100389470 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9bdc1005e0 0x7f9bdc102fc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:22.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:22.002+0000 7f9be38ee700 1 --2- 172.21.15.112:0/100389470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9bdc103660 0x7f9bdc101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:22.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:22.002+0000 7f9be38ee700 1 -- 172.21.15.112:0/100389470 >> 172.21.15.112:0/100389470 conn(0x7f9bdc0fbf40 msgr2=0x7f9bdc106a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:22.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:22.002+0000 7f9be38ee700 1 -- 172.21.15.112:0/100389470 shutdown_connections 2024-03-16T09:15:22.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:22.002+0000 7f9be38ee700 1 -- 172.21.15.112:0/100389470 wait complete. 2024-03-16T09:15:22.015 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2024-03-16T09:15:22.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:22 smithi112 ceph-mon[32307]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T09:15:22.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:22 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/100389470' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:15:22.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:22 smithi194 ceph-mon[36729]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T09:15:22.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:22 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/100389470' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:15:22.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:22 smithi159 ceph-mon[36739]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T09:15:22.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:22 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/100389470' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-16T09:15:22.795 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-16T09:15:23.039 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:15:24.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:24 smithi112 ceph-mon[32307]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:15:24.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:24 smithi194 ceph-mon[36729]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:15:24.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:24 smithi159 ceph-mon[36739]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:15:24.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.725+0000 7f5fe28e8700 1 -- 172.21.15.112:0/3802207942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fdc101200 msgr2=0x7f5fdc101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.725+0000 7f5fe28e8700 1 --2- 172.21.15.112:0/3802207942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fdc101200 0x7f5fdc101620 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0099c0 tx=0x7f5fcc009cd0 comp rx=0 tx=0).stop 2024-03-16T09:15:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.725+0000 7f5fe28e8700 1 -- 172.21.15.112:0/3802207942 shutdown_connections 2024-03-16T09:15:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.725+0000 7f5fe28e8700 1 --2- 172.21.15.112:0/3802207942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fdc102da0 0x7f5fdc109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.725+0000 7f5fe28e8700 1 --2- 172.21.15.112:0/3802207942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fdc101200 0x7f5fdc101620 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.725+0000 7f5fe28e8700 1 --2- 172.21.15.112:0/3802207942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fdc102400 0x7f5fdc102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.725+0000 7f5fe28e8700 1 -- 172.21.15.112:0/3802207942 >> 172.21.15.112:0/3802207942 conn(0x7f5fdc0fc7a0 msgr2=0x7f5fdc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.725+0000 7f5fe28e8700 1 -- 172.21.15.112:0/3802207942 shutdown_connections 2024-03-16T09:15:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.725+0000 7f5fe28e8700 1 -- 172.21.15.112:0/3802207942 wait complete. 2024-03-16T09:15:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.725+0000 7f5fe28e8700 1 Processor -- start 2024-03-16T09:15:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.725+0000 7f5fe28e8700 1 -- start start 2024-03-16T09:15:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.726+0000 7f5fe28e8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fdc101200 0x7f5fdc06aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:24.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.726+0000 7f5fe28e8700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fdc102400 0x7f5fdc19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.726+0000 7f5fe28e8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fdc102da0 0x7f5fdc1a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.726+0000 7f5fe28e8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc10cf20 con 0x7f5fdc102da0 2024-03-16T09:15:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.726+0000 7f5fe28e8700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc10cda0 con 0x7f5fdc102400 2024-03-16T09:15:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.726+0000 7f5fe28e8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc10d0a0 con 0x7f5fdc101200 2024-03-16T09:15:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.726+0000 7f5fe0e85700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fdc102da0 0x7f5fdc1a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.726+0000 7f5fe0e85700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fdc102da0 0x7f5fdc1a1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57428/0 (socket says 172.21.15.112:57428) 2024-03-16T09:15:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.726+0000 7f5fdbfff700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fdc101200 0x7f5fdc06aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.726+0000 7f5fdb7fe700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fdc102400 0x7f5fdc19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.726+0000 7f5fe0e85700 1 -- 172.21.15.112:0/1146310190 learned_addr learned my addr 172.21.15.112:0/1146310190 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.726+0000 7f5fe0e85700 1 -- 172.21.15.112:0/1146310190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fdc101200 msgr2=0x7f5fdc06aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.726+0000 7f5fe0e85700 1 --2- 172.21.15.112:0/1146310190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fdc101200 0x7f5fdc06aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.726+0000 7f5fe0e85700 1 -- 172.21.15.112:0/1146310190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fdc102400 msgr2=0x7f5fdc19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.726+0000 7f5fe0e85700 1 --2- 172.21.15.112:0/1146310190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fdc102400 0x7f5fdc19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:24.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.726+0000 7f5fe0e85700 1 -- 172.21.15.112:0/1146310190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fdc1a2530 con 0x7f5fdc102da0 2024-03-16T09:15:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.726+0000 7f5fdb7fe700 1 --2- 172.21.15.112:0/1146310190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fdc102400 0x7f5fdc19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.726+0000 7f5fdbfff700 1 --2- 172.21.15.112:0/1146310190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fdc101200 0x7f5fdc06aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.726+0000 7f5fe0e85700 1 --2- 172.21.15.112:0/1146310190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fdc102da0 0x7f5fdc1a1d50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400ef30 tx=0x7f5fd400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.727+0000 7f5fd97fa700 1 -- 172.21.15.112:0/1146310190 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd40092e0 con 0x7f5fdc102da0 2024-03-16T09:15:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.727+0000 7f5fe28e8700 1 -- 172.21.15.112:0/1146310190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fdc1a27c0 con 0x7f5fdc102da0 2024-03-16T09:15:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.727+0000 7f5fd97fa700 1 -- 172.21.15.112:0/1146310190 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5fd4004bb0 con 0x7f5fdc102da0 2024-03-16T09:15:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.727+0000 7f5fd97fa700 1 -- 172.21.15.112:0/1146310190 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4004280 con 0x7f5fdc102da0 2024-03-16T09:15:24.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.727+0000 7f5fe28e8700 1 -- 172.21.15.112:0/1146310190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fdc1a2d00 con 0x7f5fdc102da0 2024-03-16T09:15:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.728+0000 7f5fe28e8700 1 -- 172.21.15.112:0/1146310190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fdc068f90 con 0x7f5fdc102da0 2024-03-16T09:15:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.729+0000 7f5fd97fa700 1 -- 172.21.15.112:0/1146310190 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f5fd401b030 con 0x7f5fdc102da0 2024-03-16T09:15:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.729+0000 7f5fd97fa700 1 --2- 172.21.15.112:0/1146310190 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5fc40a2890 0x7f5fc40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.729+0000 7f5fdbfff700 1 --2- 172.21.15.112:0/1146310190 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5fc40a2890 0x7f5fc40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.730+0000 7f5fd97fa700 1 -- 172.21.15.112:0/1146310190 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f5fd40ce550 con 0x7f5fdc102da0 2024-03-16T09:15:24.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.730+0000 7f5fdbfff700 1 --2- 172.21.15.112:0/1146310190 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5fc40a2890 0x7f5fc40a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc000c00 tx=0x7f5fcc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:24.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.734+0000 7f5fd97fa700 1 -- 172.21.15.112:0/1146310190 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5fd4022020 con 0x7f5fdc102da0 2024-03-16T09:15:24.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.928+0000 7f5fe28e8700 1 -- 172.21.15.112:0/1146310190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f5fdc062ee0 con 0x7f5fdc102da0 2024-03-16T09:15:24.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.929+0000 7f5fd97fa700 1 -- 172.21.15.112:0/1146310190 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) ==== 56+0+5316 (secure 0 0 0) 0x7f5fd4010040 con 0x7f5fdc102da0 2024-03-16T09:15:24.932 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 87 2024-03-16T09:15:24.932 INFO:teuthology.orchestra.run.smithi112.stdout:fsid 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:15:24.932 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-03-16T09:05:30.615922+0000 2024-03-16T09:15:24.932 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-03-16T09:14:31.991998+0000 2024-03-16T09:15:24.932 INFO:teuthology.orchestra.run.smithi112.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-16T09:15:24.932 INFO:teuthology.orchestra.run.smithi112.stdout:crush_version 26 2024-03-16T09:15:24.932 INFO:teuthology.orchestra.run.smithi112.stdout:full_ratio 0.95 2024-03-16T09:15:24.932 INFO:teuthology.orchestra.run.smithi112.stdout:backfillfull_ratio 0.9 2024-03-16T09:15:24.932 INFO:teuthology.orchestra.run.smithi112.stdout:nearfull_ratio 0.85 2024-03-16T09:15:24.932 INFO:teuthology.orchestra.run.smithi112.stdout:require_min_compat_client luminous 2024-03-16T09:15:24.933 INFO:teuthology.orchestra.run.smithi112.stdout:min_compat_client luminous 2024-03-16T09:15:24.933 INFO:teuthology.orchestra.run.smithi112.stdout:require_osd_release squid 2024-03-16T09:15:24.933 INFO:teuthology.orchestra.run.smithi112.stdout:stretch_mode_enabled false 2024-03-16T09:15:24.933 INFO:teuthology.orchestra.run.smithi112.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-16T09:15:24.933 INFO:teuthology.orchestra.run.smithi112.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-16T09:15:24.933 INFO:teuthology.orchestra.run.smithi112.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-16T09:15:24.933 INFO:teuthology.orchestra.run.smithi112.stdout:max_osd 12 2024-03-16T09:15:24.933 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6802/1848821115,v1:172.21.15.112:6803/1848821115] [v2:172.21.15.112:6804/1848821115,v1:172.21.15.112:6805/1848821115] exists,up cac77ab4-0b17-41b1-ac04-3ca10c14ea93 2024-03-16T09:15:24.933 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6800/140175088,v1:172.21.15.159:6801/140175088] [v2:172.21.15.159:6802/140175088,v1:172.21.15.159:6803/140175088] exists,up b46a1d96-d8cc-4b30-b9d7-b656c94ac581 2024-03-16T09:15:24.933 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6800/4037530979,v1:172.21.15.194:6801/4037530979] [v2:172.21.15.194:6802/4037530979,v1:172.21.15.194:6803/4037530979] exists,up 4debd434-e089-4ee8-a944-c22a8405f8be 2024-03-16T09:15:24.933 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6810/3090522464,v1:172.21.15.112:6811/3090522464] [v2:172.21.15.112:6812/3090522464,v1:172.21.15.112:6813/3090522464] exists,up f81eddbc-f120-4077-a6f5-2bee53c270e2 2024-03-16T09:15:24.933 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 up in weight 1 up_from 33 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6808/4036563014,v1:172.21.15.159:6809/4036563014] [v2:172.21.15.159:6810/4036563014,v1:172.21.15.159:6811/4036563014] exists,up 9e27e743-08f5-4722-a903-1adac60ada69 2024-03-16T09:15:24.933 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 up in weight 1 up_from 38 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6808/1459524918,v1:172.21.15.194:6809/1459524918] [v2:172.21.15.194:6810/1459524918,v1:172.21.15.194:6811/1459524918] exists,up f7440b71-313a-42ff-9d9b-4badc07608cd 2024-03-16T09:15:24.933 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 up in weight 1 up_from 46 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6818/3853469923,v1:172.21.15.112:6819/3853469923] [v2:172.21.15.112:6820/3853469923,v1:172.21.15.112:6821/3853469923] exists,up 1e250eec-8d30-465d-8352-3f7db892c370 2024-03-16T09:15:24.934 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 up in weight 1 up_from 51 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6816/2484095468,v1:172.21.15.159:6817/2484095468] [v2:172.21.15.159:6818/2484095468,v1:172.21.15.159:6819/2484095468] exists,up f2eb2220-107d-411e-9572-8fae0faf0a20 2024-03-16T09:15:24.934 INFO:teuthology.orchestra.run.smithi112.stdout:osd.8 up in weight 1 up_from 59 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6816/3334422298,v1:172.21.15.194:6817/3334422298] [v2:172.21.15.194:6818/3334422298,v1:172.21.15.194:6819/3334422298] exists,up c9ce2ec1-b2db-4ce1-b306-ea6e333b02de 2024-03-16T09:15:24.934 INFO:teuthology.orchestra.run.smithi112.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6826/952596860,v1:172.21.15.112:6827/952596860] [v2:172.21.15.112:6828/952596860,v1:172.21.15.112:6829/952596860] exists,up e2b22c20-177b-41ec-b579-f18bc619d79c 2024-03-16T09:15:24.934 INFO:teuthology.orchestra.run.smithi112.stdout:osd.10 up in weight 1 up_from 72 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6824/1906671120,v1:172.21.15.159:6825/1906671120] [v2:172.21.15.159:6826/1906671120,v1:172.21.15.159:6827/1906671120] exists,up 1922eaad-cff8-46cc-b837-238133613e28 2024-03-16T09:15:24.934 INFO:teuthology.orchestra.run.smithi112.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6824/3230458513,v1:172.21.15.194:6825/3230458513] [v2:172.21.15.194:6826/3230458513,v1:172.21.15.194:6827/3230458513] exists,up eb0e58e0-7d51-4a00-9004-03607251189f 2024-03-16T09:15:24.934 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.c [2,4] 2024-03-16T09:15:24.934 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.16 [2,4] 2024-03-16T09:15:24.934 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.1d [8,11] 2024-03-16T09:15:24.934 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.1f [7,11] 2024-03-16T09:15:24.934 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.0 [5,9] 2024-03-16T09:15:24.934 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.13 [4,9] 2024-03-16T09:15:24.934 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.2c [4,9] 2024-03-16T09:15:24.934 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.31 [4,8] 2024-03-16T09:15:24.934 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/1871711739 expires 2024-03-17T09:06:30.792961+0000 2024-03-16T09:15:24.935 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6801/2629104617 expires 2024-03-17T09:06:30.792961+0000 2024-03-16T09:15:24.935 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/3938289317 expires 2024-03-17T09:06:30.792961+0000 2024-03-16T09:15:24.935 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6801/1790808883 expires 2024-03-17T09:06:04.186776+0000 2024-03-16T09:15:24.935 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6800/1790808883 expires 2024-03-17T09:06:04.186776+0000 2024-03-16T09:15:24.935 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/4123520095 expires 2024-03-17T09:06:04.186776+0000 2024-03-16T09:15:24.935 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/2535377744 expires 2024-03-17T09:06:04.186776+0000 2024-03-16T09:15:24.935 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/1888627499 expires 2024-03-17T09:06:30.792961+0000 2024-03-16T09:15:24.935 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6800/2629104617 expires 2024-03-17T09:06:30.792961+0000 2024-03-16T09:15:24.935 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/3006861017 expires 2024-03-17T09:06:04.186776+0000 2024-03-16T09:15:24.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.931+0000 7f5fe28e8700 1 -- 172.21.15.112:0/1146310190 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5fc40a2890 msgr2=0x7f5fc40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:24.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.931+0000 7f5fe28e8700 1 --2- 172.21.15.112:0/1146310190 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5fc40a2890 0x7f5fc40a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc000c00 tx=0x7f5fcc005fb0 comp rx=0 tx=0).stop 2024-03-16T09:15:24.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.931+0000 7f5fe28e8700 1 -- 172.21.15.112:0/1146310190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fdc102da0 msgr2=0x7f5fdc1a1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:24.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.931+0000 7f5fe28e8700 1 --2- 172.21.15.112:0/1146310190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fdc102da0 0x7f5fdc1a1d50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400ef30 tx=0x7f5fd400c3a0 comp rx=0 tx=0).stop 2024-03-16T09:15:24.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.931+0000 7f5fe28e8700 1 -- 172.21.15.112:0/1146310190 shutdown_connections 2024-03-16T09:15:24.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.931+0000 7f5fe28e8700 1 --2- 172.21.15.112:0/1146310190 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5fc40a2890 0x7f5fc40a4d50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:24.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.931+0000 7f5fe28e8700 1 --2- 172.21.15.112:0/1146310190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5fdc102da0 0x7f5fdc1a1d50 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:24.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.931+0000 7f5fe28e8700 1 --2- 172.21.15.112:0/1146310190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fdc101200 0x7f5fdc06aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:24.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.931+0000 7f5fe28e8700 1 --2- 172.21.15.112:0/1146310190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fdc102400 0x7f5fdc19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:24.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.931+0000 7f5fe28e8700 1 -- 172.21.15.112:0/1146310190 >> 172.21.15.112:0/1146310190 conn(0x7f5fdc0fc7a0 msgr2=0x7f5fdc105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:24.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.931+0000 7f5fe28e8700 1 -- 172.21.15.112:0/1146310190 shutdown_connections 2024-03-16T09:15:24.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:24.931+0000 7f5fe28e8700 1 -- 172.21.15.112:0/1146310190 wait complete. 2024-03-16T09:15:25.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:25 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1146310190' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-16T09:15:25.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:25 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1146310190' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-16T09:15:25.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:25 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1146310190' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-16T09:15:25.738 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-16T09:15:25.749 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:15:25.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.855+0000 7f6856859640 1 -- 172.21.15.112:0/4169016127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68501033b0 msgr2=0x7f6850109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.855+0000 7f6856859640 1 --2- 172.21.15.112:0/4169016127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68501033b0 0x7f6850109c40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6848009ca0 tx=0x7f684802c0f0 comp rx=0 tx=0).stop 2024-03-16T09:15:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.855+0000 7f6856859640 1 -- 172.21.15.112:0/4169016127 shutdown_connections 2024-03-16T09:15:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.855+0000 7f6856859640 1 --2- 172.21.15.112:0/4169016127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68501033b0 0x7f6850109c40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.855+0000 7f6856859640 1 --2- 172.21.15.112:0/4169016127 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f68501029f0 0x7f6850102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.855+0000 7f6856859640 1 --2- 172.21.15.112:0/4169016127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6850101800 0x7f6850101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.855+0000 7f6856859640 1 -- 172.21.15.112:0/4169016127 >> 172.21.15.112:0/4169016127 conn(0x7f68500fcd90 msgr2=0x7f68500ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.855+0000 7f6856859640 1 -- 172.21.15.112:0/4169016127 shutdown_connections 2024-03-16T09:15:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.855+0000 7f6856859640 1 -- 172.21.15.112:0/4169016127 wait complete. 2024-03-16T09:15:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.856+0000 7f6856859640 1 Processor -- start 2024-03-16T09:15:25.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.856+0000 7f6856859640 1 -- start start 2024-03-16T09:15:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.856+0000 7f6856859640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6850101800 0x7f6850197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.856+0000 7f6856859640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68501029f0 0x7f68501980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.856+0000 7f6856859640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68501033b0 0x7f685019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.856+0000 7f6856859640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f685010d5b0 con 0x7f68501033b0 2024-03-16T09:15:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.856+0000 7f6856859640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f685010d430 con 0x7f68501029f0 2024-03-16T09:15:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.856+0000 7f6856859640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f685010d730 con 0x7f6850101800 2024-03-16T09:15:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.856+0000 7f6854df6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68501033b0 0x7f685019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.856+0000 7f6854df6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68501033b0 0x7f685019f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57454/0 (socket says 172.21.15.112:57454) 2024-03-16T09:15:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.856+0000 7f6854df6640 1 -- 172.21.15.112:0/1168327168 learned_addr learned my addr 172.21.15.112:0/1168327168 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.856+0000 7f684ffff640 1 --2- 172.21.15.112:0/1168327168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6850101800 0x7f6850197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.856+0000 7f684f7fe640 1 --2- 172.21.15.112:0/1168327168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68501029f0 0x7f68501980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.857+0000 7f6854df6640 1 -- 172.21.15.112:0/1168327168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6850101800 msgr2=0x7f6850197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:25.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.857+0000 7f6854df6640 1 --2- 172.21.15.112:0/1168327168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6850101800 0x7f6850197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.857+0000 7f6854df6640 1 -- 172.21.15.112:0/1168327168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68501029f0 msgr2=0x7f68501980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.857+0000 7f6854df6640 1 --2- 172.21.15.112:0/1168327168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68501029f0 0x7f68501980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.857+0000 7f6854df6640 1 -- 172.21.15.112:0/1168327168 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685019f780 con 0x7f68501033b0 2024-03-16T09:15:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.857+0000 7f6854df6640 1 --2- 172.21.15.112:0/1168327168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68501033b0 0x7f685019f110 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f684802c700 tx=0x7f68480346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.857+0000 7f684d7fa640 1 -- 172.21.15.112:0/1168327168 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848039040 con 0x7f68501033b0 2024-03-16T09:15:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.857+0000 7f684d7fa640 1 -- 172.21.15.112:0/1168327168 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6848044070 con 0x7f68501033b0 2024-03-16T09:15:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.857+0000 7f6856859640 1 -- 172.21.15.112:0/1168327168 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685019fa10 con 0x7f68501033b0 2024-03-16T09:15:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.857+0000 7f684d7fa640 1 -- 172.21.15.112:0/1168327168 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68480043c0 con 0x7f68501033b0 2024-03-16T09:15:25.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.857+0000 7f6856859640 1 -- 172.21.15.112:0/1168327168 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f685019ff00 con 0x7f68501033b0 2024-03-16T09:15:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.858+0000 7f6856859640 1 -- 172.21.15.112:0/1168327168 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6850108ac0 con 0x7f68501033b0 2024-03-16T09:15:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.859+0000 7f684d7fa640 1 -- 172.21.15.112:0/1168327168 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f6848004580 con 0x7f68501033b0 2024-03-16T09:15:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.860+0000 7f684d7fa640 1 --2- 172.21.15.112:0/1168327168 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f68380a2800 0x7f68380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:25.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.860+0000 7f684ffff640 1 --2- 172.21.15.112:0/1168327168 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f68380a2800 0x7f68380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:25.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.860+0000 7f684d7fa640 1 -- 172.21.15.112:0/1168327168 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f68480f0f90 con 0x7f68501033b0 2024-03-16T09:15:25.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.860+0000 7f684ffff640 1 --2- 172.21.15.112:0/1168327168 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f68380a2800 0x7f68380a4cc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6850102850 tx=0x7f6844005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:25.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:25.864+0000 7f684d7fa640 1 -- 172.21.15.112:0/1168327168 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f68480be0e0 con 0x7f68501033b0 2024-03-16T09:15:26.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.043+0000 7f6856859640 1 -- 172.21.15.112:0/1168327168 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68500623e0 con 0x7f68501033b0 2024-03-16T09:15:26.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.044+0000 7f684d7fa640 1 -- 172.21.15.112:0/1168327168 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22725 (secure 0 0 0) 0x7f68480bd9e0 con 0x7f68501033b0 2024-03-16T09:15:26.047 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:26.047 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":87,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:14:31.991998+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T09:13:35.040628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T09:13:35.689419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:15:26.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.047+0000 7f6856859640 1 -- 172.21.15.112:0/1168327168 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f68380a2800 msgr2=0x7f68380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:26.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.047+0000 7f6856859640 1 --2- 172.21.15.112:0/1168327168 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f68380a2800 0x7f68380a4cc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6850102850 tx=0x7f6844005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:26.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.047+0000 7f6856859640 1 -- 172.21.15.112:0/1168327168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68501033b0 msgr2=0x7f685019f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:26.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.047+0000 7f6856859640 1 --2- 172.21.15.112:0/1168327168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68501033b0 0x7f685019f110 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f684802c700 tx=0x7f68480346d0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.047+0000 7f6856859640 1 -- 172.21.15.112:0/1168327168 shutdown_connections 2024-03-16T09:15:26.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.047+0000 7f6856859640 1 --2- 172.21.15.112:0/1168327168 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f68380a2800 0x7f68380a4cc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.047+0000 7f6856859640 1 --2- 172.21.15.112:0/1168327168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68501033b0 0x7f685019f110 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.047+0000 7f6856859640 1 --2- 172.21.15.112:0/1168327168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68501029f0 0x7f68501980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.047+0000 7f6856859640 1 --2- 172.21.15.112:0/1168327168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6850101800 0x7f6850197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.047+0000 7f6856859640 1 -- 172.21.15.112:0/1168327168 >> 172.21.15.112:0/1168327168 conn(0x7f68500fcd90 msgr2=0x7f6850105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:26.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.047+0000 7f6856859640 1 -- 172.21.15.112:0/1168327168 shutdown_connections 2024-03-16T09:15:26.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.047+0000 7f6856859640 1 -- 172.21.15.112:0/1168327168 wait complete. 2024-03-16T09:15:26.060 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T09:08:57.135312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T09:13:35.040628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T09:13:35.689419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T09:15:26.061 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T09:15:26.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.162+0000 7f02db885640 1 -- 172.21.15.112:0/1467715347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d41022d0 msgr2=0x7f02d41046c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:26.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.162+0000 7f02db885640 1 --2- 172.21.15.112:0/1467715347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d41022d0 0x7f02d41046c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f02d0009900 tx=0x7f02d002bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:26.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.162+0000 7f02db885640 1 -- 172.21.15.112:0/1467715347 shutdown_connections 2024-03-16T09:15:26.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.162+0000 7f02db885640 1 --2- 172.21.15.112:0/1467715347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02d4107530 0x7f02d4109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.162+0000 7f02db885640 1 --2- 172.21.15.112:0/1467715347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02d4104c00 0x7f02d4106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.162+0000 7f02db885640 1 --2- 172.21.15.112:0/1467715347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d41022d0 0x7f02d41046c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.162+0000 7f02db885640 1 -- 172.21.15.112:0/1467715347 >> 172.21.15.112:0/1467715347 conn(0x7f02d40fbc80 msgr2=0x7f02d40fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:26.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.162+0000 7f02db885640 1 -- 172.21.15.112:0/1467715347 shutdown_connections 2024-03-16T09:15:26.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.162+0000 7f02db885640 1 -- 172.21.15.112:0/1467715347 wait complete. 2024-03-16T09:15:26.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02db885640 1 Processor -- start 2024-03-16T09:15:26.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02db885640 1 -- start start 2024-03-16T09:15:26.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02db885640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d41022d0 0x7f02d4197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:26.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02db885640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02d4104c00 0x7f02d41981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:26.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02db885640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02d4107530 0x7f02d419f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:26.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02db885640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02d410d550 con 0x7f02d41022d0 2024-03-16T09:15:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02db885640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f02d410d3d0 con 0x7f02d4104c00 2024-03-16T09:15:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02db885640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f02d410d6d0 con 0x7f02d4107530 2024-03-16T09:15:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02d9621640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d41022d0 0x7f02d4197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02d9621640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d41022d0 0x7f02d4197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57462/0 (socket says 172.21.15.112:57462) 2024-03-16T09:15:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02d9621640 1 -- 172.21.15.112:0/224362170 learned_addr learned my addr 172.21.15.112:0/224362170 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02d9e22640 1 --2- 172.21.15.112:0/224362170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02d4107530 0x7f02d419f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02d8e20640 1 --2- 172.21.15.112:0/224362170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02d4104c00 0x7f02d41981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02d9621640 1 -- 172.21.15.112:0/224362170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02d4107530 msgr2=0x7f02d419f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02d9621640 1 --2- 172.21.15.112:0/224362170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02d4107530 0x7f02d419f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02d9621640 1 -- 172.21.15.112:0/224362170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02d4104c00 msgr2=0x7f02d41981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02d9621640 1 --2- 172.21.15.112:0/224362170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02d4104c00 0x7f02d41981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02d9621640 1 -- 172.21.15.112:0/224362170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d419f880 con 0x7f02d41022d0 2024-03-16T09:15:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02d9621640 1 --2- 172.21.15.112:0/224362170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d41022d0 0x7f02d4197c70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f02d0009230 tx=0x7f02d0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.163+0000 7f02ca7fc640 1 -- 172.21.15.112:0/224362170 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0043020 con 0x7f02d41022d0 2024-03-16T09:15:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.164+0000 7f02ca7fc640 1 -- 172.21.15.112:0/224362170 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f02d0004250 con 0x7f02d41022d0 2024-03-16T09:15:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.164+0000 7f02db885640 1 -- 172.21.15.112:0/224362170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d419fb10 con 0x7f02d41022d0 2024-03-16T09:15:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.164+0000 7f02ca7fc640 1 -- 172.21.15.112:0/224362170 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0003b50 con 0x7f02d41022d0 2024-03-16T09:15:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.164+0000 7f02db885640 1 -- 172.21.15.112:0/224362170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d41a0060 con 0x7f02d41022d0 2024-03-16T09:15:26.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.164+0000 7f02db885640 1 -- 172.21.15.112:0/224362170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d4068a20 con 0x7f02d41022d0 2024-03-16T09:15:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.166+0000 7f02ca7fc640 1 -- 172.21.15.112:0/224362170 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f02d0046020 con 0x7f02d41022d0 2024-03-16T09:15:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.167+0000 7f02ca7fc640 1 --2- 172.21.15.112:0/224362170 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f02c00a2800 0x7f02c00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.167+0000 7f02d8e20640 1 --2- 172.21.15.112:0/224362170 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f02c00a2800 0x7f02c00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.167+0000 7f02ca7fc640 1 -- 172.21.15.112:0/224362170 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f02d00f0fd0 con 0x7f02d41022d0 2024-03-16T09:15:26.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.167+0000 7f02d8e20640 1 --2- 172.21.15.112:0/224362170 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f02c00a2800 0x7f02c00a4cc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f02c40095f0 tx=0x7f02c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:26.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.170+0000 7f02ca7fc640 1 -- 172.21.15.112:0/224362170 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f02d004d020 con 0x7f02d41022d0 2024-03-16T09:15:26.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:26 smithi112 ceph-mon[32307]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T09:15:26.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.345+0000 7f02db885640 1 -- 172.21.15.112:0/224362170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f02d402d250 con 0x7f02d41022d0 2024-03-16T09:15:26.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.345+0000 7f02ca7fc640 1 -- 172.21.15.112:0/224362170 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f02d402d250 con 0x7f02d41022d0 2024-03-16T09:15:26.347 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-03-16T09:15:26.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.347+0000 7f02db885640 1 -- 172.21.15.112:0/224362170 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f02c00a2800 msgr2=0x7f02c00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:26.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.347+0000 7f02db885640 1 --2- 172.21.15.112:0/224362170 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f02c00a2800 0x7f02c00a4cc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f02c40095f0 tx=0x7f02c4005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:26.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.347+0000 7f02db885640 1 -- 172.21.15.112:0/224362170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d41022d0 msgr2=0x7f02d4197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:26.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.347+0000 7f02db885640 1 --2- 172.21.15.112:0/224362170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d41022d0 0x7f02d4197c70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f02d0009230 tx=0x7f02d0004ab0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.347+0000 7f02db885640 1 -- 172.21.15.112:0/224362170 shutdown_connections 2024-03-16T09:15:26.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.347+0000 7f02db885640 1 --2- 172.21.15.112:0/224362170 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f02c00a2800 0x7f02c00a4cc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.347+0000 7f02db885640 1 --2- 172.21.15.112:0/224362170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02d4107530 0x7f02d419f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.347+0000 7f02db885640 1 --2- 172.21.15.112:0/224362170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02d4104c00 0x7f02d41981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.347+0000 7f02db885640 1 --2- 172.21.15.112:0/224362170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d41022d0 0x7f02d4197c70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.347+0000 7f02db885640 1 -- 172.21.15.112:0/224362170 >> 172.21.15.112:0/224362170 conn(0x7f02d40fbc80 msgr2=0x7f02d4102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:26.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.347+0000 7f02db885640 1 -- 172.21.15.112:0/224362170 shutdown_connections 2024-03-16T09:15:26.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.347+0000 7f02db885640 1 -- 172.21.15.112:0/224362170 wait complete. 2024-03-16T09:15:26.361 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T09:15:26.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:26 smithi194 ceph-mon[36729]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T09:15:26.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.462+0000 7fc33a122640 1 -- 172.21.15.112:0/3742783690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc334103470 msgr2=0x7fc334109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.463+0000 7fc33a122640 1 --2- 172.21.15.112:0/3742783690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc334103470 0x7fc334109d00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009cd0 tx=0x7fc32c02c0f0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.463+0000 7fc33a122640 1 -- 172.21.15.112:0/3742783690 shutdown_connections 2024-03-16T09:15:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.463+0000 7fc33a122640 1 --2- 172.21.15.112:0/3742783690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc334103470 0x7fc334109d00 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.463+0000 7fc33a122640 1 --2- 172.21.15.112:0/3742783690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc334102ab0 0x7fc334102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.463+0000 7fc33a122640 1 --2- 172.21.15.112:0/3742783690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3340ffae0 0x7fc3340fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.463+0000 7fc33a122640 1 -- 172.21.15.112:0/3742783690 >> 172.21.15.112:0/3742783690 conn(0x7fc3340fb410 msgr2=0x7fc3340fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.463+0000 7fc33a122640 1 -- 172.21.15.112:0/3742783690 shutdown_connections 2024-03-16T09:15:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.463+0000 7fc33a122640 1 -- 172.21.15.112:0/3742783690 wait complete. 2024-03-16T09:15:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.463+0000 7fc33a122640 1 Processor -- start 2024-03-16T09:15:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.463+0000 7fc33a122640 1 -- start start 2024-03-16T09:15:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.463+0000 7fc33a122640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3340ffae0 0x7fc334197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:26.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc33a122640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc334102ab0 0x7fc334198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc33a122640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc334103470 0x7fc33419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc33a122640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc33410d5f0 con 0x7fc334103470 2024-03-16T09:15:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc33a122640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc33410d470 con 0x7fc334102ab0 2024-03-16T09:15:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc33a122640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc33410d770 con 0x7fc3340ffae0 2024-03-16T09:15:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc333fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc334103470 0x7fc33419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc333fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc334103470 0x7fc33419f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57490/0 (socket says 172.21.15.112:57490) 2024-03-16T09:15:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc333fff640 1 -- 172.21.15.112:0/2027058757 learned_addr learned my addr 172.21.15.112:0/2027058757 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc3337fe640 1 --2- 172.21.15.112:0/2027058757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3340ffae0 0x7fc334197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc332ffd640 1 --2- 172.21.15.112:0/2027058757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc334102ab0 0x7fc334198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc333fff640 1 -- 172.21.15.112:0/2027058757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3340ffae0 msgr2=0x7fc334197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc333fff640 1 --2- 172.21.15.112:0/2027058757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3340ffae0 0x7fc334197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc333fff640 1 -- 172.21.15.112:0/2027058757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc334102ab0 msgr2=0x7fc334198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc333fff640 1 --2- 172.21.15.112:0/2027058757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc334102ab0 0x7fc334198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc333fff640 1 -- 172.21.15.112:0/2027058757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33419f830 con 0x7fc334103470 2024-03-16T09:15:26.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc333fff640 1 --2- 172.21.15.112:0/2027058757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc334103470 0x7fc33419f190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009ca0 tx=0x7fc32c035d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:26.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc330ff9640 1 -- 172.21.15.112:0/2027058757 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c039040 con 0x7fc334103470 2024-03-16T09:15:26.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc330ff9640 1 -- 172.21.15.112:0/2027058757 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc32c044070 con 0x7fc334103470 2024-03-16T09:15:26.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc33a122640 1 -- 172.21.15.112:0/2027058757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc33419fac0 con 0x7fc334103470 2024-03-16T09:15:26.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc330ff9640 1 -- 172.21.15.112:0/2027058757 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0044a0 con 0x7fc334103470 2024-03-16T09:15:26.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.464+0000 7fc33a122640 1 -- 172.21.15.112:0/2027058757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc33419ffb0 con 0x7fc334103470 2024-03-16T09:15:26.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.465+0000 7fc33a122640 1 -- 172.21.15.112:0/2027058757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc334108b80 con 0x7fc334103470 2024-03-16T09:15:26.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.467+0000 7fc330ff9640 1 -- 172.21.15.112:0/2027058757 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7fc32c004640 con 0x7fc334103470 2024-03-16T09:15:26.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.467+0000 7fc330ff9640 1 --2- 172.21.15.112:0/2027058757 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc31c0a2800 0x7fc31c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:26.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.467+0000 7fc3337fe640 1 --2- 172.21.15.112:0/2027058757 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc31c0a2800 0x7fc31c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:26.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.467+0000 7fc330ff9640 1 -- 172.21.15.112:0/2027058757 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc32c0f16e0 con 0x7fc334103470 2024-03-16T09:15:26.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.468+0000 7fc3337fe640 1 --2- 172.21.15.112:0/2027058757 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc31c0a2800 0x7fc31c0a4cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc3341990f0 tx=0x7fc328005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:26.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.471+0000 7fc330ff9640 1 -- 172.21.15.112:0/2027058757 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc32c0be790 con 0x7fc334103470 2024-03-16T09:15:26.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:26 smithi159 ceph-mon[36739]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T09:15:26.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.646+0000 7fc33a122640 1 -- 172.21.15.112:0/2027058757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc33402d400 con 0x7fc334103470 2024-03-16T09:15:26.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.646+0000 7fc330ff9640 1 -- 172.21.15.112:0/2027058757 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fc32c0be090 con 0x7fc334103470 2024-03-16T09:15:26.648 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-03-16T09:15:26.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.648+0000 7fc33a122640 1 -- 172.21.15.112:0/2027058757 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc31c0a2800 msgr2=0x7fc31c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:26.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.648+0000 7fc33a122640 1 --2- 172.21.15.112:0/2027058757 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc31c0a2800 0x7fc31c0a4cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc3341990f0 tx=0x7fc328005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:26.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.648+0000 7fc33a122640 1 -- 172.21.15.112:0/2027058757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc334103470 msgr2=0x7fc33419f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:26.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.648+0000 7fc33a122640 1 --2- 172.21.15.112:0/2027058757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc334103470 0x7fc33419f190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc32c009ca0 tx=0x7fc32c035d30 comp rx=0 tx=0).stop 2024-03-16T09:15:26.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.648+0000 7fc33a122640 1 -- 172.21.15.112:0/2027058757 shutdown_connections 2024-03-16T09:15:26.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.648+0000 7fc33a122640 1 --2- 172.21.15.112:0/2027058757 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc31c0a2800 0x7fc31c0a4cc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.648+0000 7fc33a122640 1 --2- 172.21.15.112:0/2027058757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc334103470 0x7fc33419f190 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.648+0000 7fc33a122640 1 --2- 172.21.15.112:0/2027058757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc334102ab0 0x7fc334198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.648+0000 7fc33a122640 1 --2- 172.21.15.112:0/2027058757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3340ffae0 0x7fc334197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.648+0000 7fc33a122640 1 -- 172.21.15.112:0/2027058757 >> 172.21.15.112:0/2027058757 conn(0x7fc3340fb410 msgr2=0x7fc334105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:26.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.648+0000 7fc33a122640 1 -- 172.21.15.112:0/2027058757 shutdown_connections 2024-03-16T09:15:26.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.648+0000 7fc33a122640 1 -- 172.21.15.112:0/2027058757 wait complete. 2024-03-16T09:15:26.661 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T09:15:26.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.765+0000 7f70341cd640 1 -- 172.21.15.112:0/446221968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f702c100700 msgr2=0x7f702c100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:26.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.765+0000 7f70341cd640 1 --2- 172.21.15.112:0/446221968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f702c100700 0x7f702c100b20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f701c009900 tx=0x7f701c02bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:26.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.765+0000 7f70341cd640 1 -- 172.21.15.112:0/446221968 shutdown_connections 2024-03-16T09:15:26.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.765+0000 7f70341cd640 1 --2- 172.21.15.112:0/446221968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f702c1022b0 0x7f702c108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.765+0000 7f70341cd640 1 --2- 172.21.15.112:0/446221968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f702c1018f0 0x7f702c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.765+0000 7f70341cd640 1 --2- 172.21.15.112:0/446221968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f702c100700 0x7f702c100b20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.765+0000 7f70341cd640 1 -- 172.21.15.112:0/446221968 >> 172.21.15.112:0/446221968 conn(0x7f702c0fbc90 msgr2=0x7f702c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:26.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.765+0000 7f70341cd640 1 -- 172.21.15.112:0/446221968 shutdown_connections 2024-03-16T09:15:26.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.765+0000 7f70341cd640 1 -- 172.21.15.112:0/446221968 wait complete. 2024-03-16T09:15:26.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f70341cd640 1 Processor -- start 2024-03-16T09:15:26.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f70341cd640 1 -- start start 2024-03-16T09:15:26.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f70341cd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f702c100700 0x7f702c06a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:26.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f70341cd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f702c1018f0 0x7f702c19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:26.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f70341cd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f702c1022b0 0x7f702c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:26.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f70341cd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f702c10c460 con 0x7f702c1022b0 2024-03-16T09:15:26.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f70341cd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f702c10c2e0 con 0x7f702c1018f0 2024-03-16T09:15:26.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f70341cd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f702c10c5e0 con 0x7f702c100700 2024-03-16T09:15:26.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f703276a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f702c1022b0 0x7f702c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:26.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f7031f69640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f702c100700 0x7f702c06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:26.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f7031f69640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f702c100700 0x7f702c06a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:39672/0 (socket says 172.21.15.112:39672) 2024-03-16T09:15:26.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f703276a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f702c1022b0 0x7f702c19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57510/0 (socket says 172.21.15.112:57510) 2024-03-16T09:15:26.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f7031f69640 1 -- 172.21.15.112:0/837370799 learned_addr learned my addr 172.21.15.112:0/837370799 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:26.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f7031768640 1 --2- 172.21.15.112:0/837370799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f702c1018f0 0x7f702c19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:26.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f7031f69640 1 -- 172.21.15.112:0/837370799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f702c1018f0 msgr2=0x7f702c19c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:26.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f7031f69640 1 --2- 172.21.15.112:0/837370799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f702c1018f0 0x7f702c19c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f7031f69640 1 -- 172.21.15.112:0/837370799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f702c1022b0 msgr2=0x7f702c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:26.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f7031f69640 1 --2- 172.21.15.112:0/837370799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f702c1022b0 0x7f702c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f7031f69640 1 -- 172.21.15.112:0/837370799 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702c19f820 con 0x7f702c100700 2024-03-16T09:15:26.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f703276a640 1 --2- 172.21.15.112:0/837370799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f702c1022b0 0x7f702c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:26.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.766+0000 7f7031f69640 1 --2- 172.21.15.112:0/837370799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f702c100700 0x7f702c06a960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f701c000c00 tx=0x7f701c004a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:26.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.767+0000 7f702affd640 1 -- 172.21.15.112:0/837370799 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c043020 con 0x7f702c100700 2024-03-16T09:15:26.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.767+0000 7f702affd640 1 -- 172.21.15.112:0/837370799 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f701c004250 con 0x7f702c100700 2024-03-16T09:15:26.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.767+0000 7f70341cd640 1 -- 172.21.15.112:0/837370799 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702c19fab0 con 0x7f702c100700 2024-03-16T09:15:26.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.767+0000 7f702affd640 1 -- 172.21.15.112:0/837370799 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c003b90 con 0x7f702c100700 2024-03-16T09:15:26.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.767+0000 7f70341cd640 1 -- 172.21.15.112:0/837370799 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f702c1a0050 con 0x7f702c100700 2024-03-16T09:15:26.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.768+0000 7f70341cd640 1 -- 172.21.15.112:0/837370799 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f702c107aa0 con 0x7f702c100700 2024-03-16T09:15:26.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.769+0000 7f702affd640 1 -- 172.21.15.112:0/837370799 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f701c046020 con 0x7f702c100700 2024-03-16T09:15:26.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.769+0000 7f702affd640 1 --2- 172.21.15.112:0/837370799 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f70140a2880 0x7f70140a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:26.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.769+0000 7f7031768640 1 --2- 172.21.15.112:0/837370799 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f70140a2880 0x7f70140a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:26.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.769+0000 7f702affd640 1 -- 172.21.15.112:0/837370799 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f701c0f1020 con 0x7f702c100700 2024-03-16T09:15:26.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.770+0000 7f7031768640 1 --2- 172.21.15.112:0/837370799 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f70140a2880 0x7f70140a4d40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f702c06a6d0 tx=0x7f7020009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:26.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.777+0000 7f702affd640 1 -- 172.21.15.112:0/837370799 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f701c04d020 con 0x7f702c100700 2024-03-16T09:15:26.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.952+0000 7f70341cd640 1 -- 172.21.15.112:0/837370799 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f702c02d400 con 0x7f702c100700 2024-03-16T09:15:26.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.953+0000 7f702affd640 1 -- 172.21.15.112:0/837370799 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f701c0be0d0 con 0x7f702c100700 2024-03-16T09:15:26.955 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-03-16T09:15:26.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.955+0000 7f70341cd640 1 -- 172.21.15.112:0/837370799 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f70140a2880 msgr2=0x7f70140a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:26.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.955+0000 7f70341cd640 1 --2- 172.21.15.112:0/837370799 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f70140a2880 0x7f70140a4d40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f702c06a6d0 tx=0x7f7020009500 comp rx=0 tx=0).stop 2024-03-16T09:15:26.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.955+0000 7f70341cd640 1 -- 172.21.15.112:0/837370799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f702c100700 msgr2=0x7f702c06a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:26.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.955+0000 7f70341cd640 1 --2- 172.21.15.112:0/837370799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f702c100700 0x7f702c06a960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f701c000c00 tx=0x7f701c004a30 comp rx=0 tx=0).stop 2024-03-16T09:15:26.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.955+0000 7f70341cd640 1 -- 172.21.15.112:0/837370799 shutdown_connections 2024-03-16T09:15:26.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.955+0000 7f70341cd640 1 --2- 172.21.15.112:0/837370799 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f70140a2880 0x7f70140a4d40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.955+0000 7f70341cd640 1 --2- 172.21.15.112:0/837370799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f702c1022b0 0x7f702c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.955+0000 7f70341cd640 1 --2- 172.21.15.112:0/837370799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f702c1018f0 0x7f702c19c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.955+0000 7f70341cd640 1 --2- 172.21.15.112:0/837370799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f702c100700 0x7f702c06a960 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:26.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.955+0000 7f70341cd640 1 -- 172.21.15.112:0/837370799 >> 172.21.15.112:0/837370799 conn(0x7f702c0fbc90 msgr2=0x7f702c104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:26.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.955+0000 7f70341cd640 1 -- 172.21.15.112:0/837370799 shutdown_connections 2024-03-16T09:15:26.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:26.955+0000 7f70341cd640 1 -- 172.21.15.112:0/837370799 wait complete. 2024-03-16T09:15:26.968 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:15:27.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.071+0000 7fb27d959640 1 -- 172.21.15.112:0/182080152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb278101910 msgr2=0x7fb278101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:27.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.071+0000 7fb27d959640 1 --2- 172.21.15.112:0/182080152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb278101910 0x7fb278101d90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb260009900 tx=0x7fb26002bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:27.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.071+0000 7fb27d959640 1 -- 172.21.15.112:0/182080152 shutdown_connections 2024-03-16T09:15:27.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.071+0000 7fb27d959640 1 --2- 172.21.15.112:0/182080152 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2781022d0 0x7fb278108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.071+0000 7fb27d959640 1 --2- 172.21.15.112:0/182080152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb278101910 0x7fb278101d90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.071+0000 7fb27d959640 1 --2- 172.21.15.112:0/182080152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb278100720 0x7fb278100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.071+0000 7fb27d959640 1 -- 172.21.15.112:0/182080152 >> 172.21.15.112:0/182080152 conn(0x7fb2780fbc90 msgr2=0x7fb2780fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.071+0000 7fb27d959640 1 -- 172.21.15.112:0/182080152 shutdown_connections 2024-03-16T09:15:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.071+0000 7fb27d959640 1 -- 172.21.15.112:0/182080152 wait complete. 2024-03-16T09:15:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.071+0000 7fb27d959640 1 Processor -- start 2024-03-16T09:15:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.072+0000 7fb27d959640 1 -- start start 2024-03-16T09:15:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.072+0000 7fb27d959640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb278100720 0x7fb27806a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.072+0000 7fb27d959640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb278101910 0x7fb27819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.072+0000 7fb27d959640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2781022d0 0x7fb27819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.072+0000 7fb27d959640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb27810c480 con 0x7fb278100720 2024-03-16T09:15:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.072+0000 7fb27d959640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb27810c300 con 0x7fb278101910 2024-03-16T09:15:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.072+0000 7fb27d959640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb27810c600 con 0x7fb2781022d0 2024-03-16T09:15:27.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.072+0000 7fb276ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb278100720 0x7fb27806a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.072+0000 7fb276ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb278100720 0x7fb27806a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57532/0 (socket says 172.21.15.112:57532) 2024-03-16T09:15:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.072+0000 7fb276ffd640 1 -- 172.21.15.112:0/2446529376 learned_addr learned my addr 172.21.15.112:0/2446529376 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.072+0000 7fb2777fe640 1 --2- 172.21.15.112:0/2446529376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2781022d0 0x7fb27819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.072+0000 7fb2767fc640 1 --2- 172.21.15.112:0/2446529376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb278101910 0x7fb27819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.072+0000 7fb276ffd640 1 -- 172.21.15.112:0/2446529376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2781022d0 msgr2=0x7fb27819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.072+0000 7fb276ffd640 1 --2- 172.21.15.112:0/2446529376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2781022d0 0x7fb27819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.072+0000 7fb276ffd640 1 -- 172.21.15.112:0/2446529376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb278101910 msgr2=0x7fb27819c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.072+0000 7fb276ffd640 1 --2- 172.21.15.112:0/2446529376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb278101910 0x7fb27819c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.072+0000 7fb276ffd640 1 -- 172.21.15.112:0/2446529376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27819f870 con 0x7fb278100720 2024-03-16T09:15:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.072+0000 7fb276ffd640 1 --2- 172.21.15.112:0/2446529376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb278100720 0x7fb27806a650 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb27000e9c0 tx=0x7fb27000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.072+0000 7fb27c957640 1 -- 172.21.15.112:0/2446529376 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27000cd80 con 0x7fb278100720 2024-03-16T09:15:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.073+0000 7fb27d959640 1 -- 172.21.15.112:0/2446529376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27819fb00 con 0x7fb278100720 2024-03-16T09:15:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.073+0000 7fb27c957640 1 -- 172.21.15.112:0/2446529376 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb2700035f0 con 0x7fb278100720 2024-03-16T09:15:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.073+0000 7fb27c957640 1 -- 172.21.15.112:0/2446529376 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270010690 con 0x7fb278100720 2024-03-16T09:15:27.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.073+0000 7fb27d959640 1 -- 172.21.15.112:0/2446529376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2781a0020 con 0x7fb278100720 2024-03-16T09:15:27.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.073+0000 7fb27d959640 1 -- 172.21.15.112:0/2446529376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb278107ac0 con 0x7fb278100720 2024-03-16T09:15:27.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.075+0000 7fb27c957640 1 -- 172.21.15.112:0/2446529376 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7fb2700037a0 con 0x7fb278100720 2024-03-16T09:15:27.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.075+0000 7fb27c957640 1 --2- 172.21.15.112:0/2446529376 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb2640a2800 0x7fb2640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:27.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.075+0000 7fb2767fc640 1 --2- 172.21.15.112:0/2446529376 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb2640a2800 0x7fb2640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:27.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.075+0000 7fb27c957640 1 -- 172.21.15.112:0/2446529376 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fb2700ceeb0 con 0x7fb278100720 2024-03-16T09:15:27.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.076+0000 7fb2767fc640 1 --2- 172.21.15.112:0/2446529376 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb2640a2800 0x7fb2640a4cc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb260009230 tx=0x7fb260005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:27.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.079+0000 7fb27c957640 1 -- 172.21.15.112:0/2446529376 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb27009bff0 con 0x7fb278100720 2024-03-16T09:15:27.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.290+0000 7fb27d959640 1 -- 172.21.15.112:0/2446529376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb27802d3d0 con 0x7fb278100720 2024-03-16T09:15:27.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.291+0000 7fb27c957640 1 -- 172.21.15.112:0/2446529376 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11881 (secure 0 0 0) 0x7fb27009b8f0 con 0x7fb278100720 2024-03-16T09:15:27.293 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:27.293 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3939748854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3939748854},{"type":"v1","addr":"172.21.15.112:6839","nonce":3939748854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3118523258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3118523258},{"type":"v1","addr":"172.21.15.112:6841","nonce":3118523258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T09:13:36.450216+0000","modified":"2024-03-16T09:13:40.527486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/1682076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":1682076},{"type":"v1","addr":"172.21.15.194:6837","nonce":1682076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.194:6839/2528943309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2528943309},{"type":"v1","addr":"172.21.15.194:6839","nonce":2528943309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/272895965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":272895965},{"type":"v1","addr":"172.21.15.159:6839","nonce":272895965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24452,24466,24494]},"id":1}]} 2024-03-16T09:15:27.293 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2024-03-16T09:15:27.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.292+0000 7fb27d959640 1 -- 172.21.15.112:0/2446529376 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb2640a2800 msgr2=0x7fb2640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:27.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.292+0000 7fb27d959640 1 --2- 172.21.15.112:0/2446529376 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb2640a2800 0x7fb2640a4cc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb260009230 tx=0x7fb260005fd0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.292+0000 7fb27d959640 1 -- 172.21.15.112:0/2446529376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb278100720 msgr2=0x7fb27806a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:27.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.293+0000 7fb27d959640 1 --2- 172.21.15.112:0/2446529376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb278100720 0x7fb27806a650 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb27000e9c0 tx=0x7fb27000ee80 comp rx=0 tx=0).stop 2024-03-16T09:15:27.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.293+0000 7fb27d959640 1 -- 172.21.15.112:0/2446529376 shutdown_connections 2024-03-16T09:15:27.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.293+0000 7fb27d959640 1 --2- 172.21.15.112:0/2446529376 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb2640a2800 0x7fb2640a4cc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.293+0000 7fb27d959640 1 --2- 172.21.15.112:0/2446529376 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2781022d0 0x7fb27819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.293+0000 7fb27d959640 1 --2- 172.21.15.112:0/2446529376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb278101910 0x7fb27819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.293+0000 7fb27d959640 1 --2- 172.21.15.112:0/2446529376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb278100720 0x7fb27806a650 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.293+0000 7fb27d959640 1 -- 172.21.15.112:0/2446529376 >> 172.21.15.112:0/2446529376 conn(0x7fb2780fbc90 msgr2=0x7fb278104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:27.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.293+0000 7fb27d959640 1 -- 172.21.15.112:0/2446529376 shutdown_connections 2024-03-16T09:15:27.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.293+0000 7fb27d959640 1 -- 172.21.15.112:0/2446529376 wait complete. 2024-03-16T09:15:27.305 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:27 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1168327168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:27.305 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:27 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/224362170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:27.305 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:27 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2027058757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:27.305 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:27 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/837370799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:27.306 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:15:27.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:27 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1168327168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:27.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:27 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/224362170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:27.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:27 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2027058757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:27.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:27 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/837370799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:27.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.406+0000 7f0377a59640 1 -- 172.21.15.112:0/924723855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0370101800 msgr2=0x7f0370101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:27.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.406+0000 7f0377a59640 1 --2- 172.21.15.112:0/924723855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0370101800 0x7f0370101c20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0360009900 tx=0x7f036002bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:27.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.406+0000 7f0377a59640 1 -- 172.21.15.112:0/924723855 shutdown_connections 2024-03-16T09:15:27.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.406+0000 7f0377a59640 1 --2- 172.21.15.112:0/924723855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f03701033b0 0x7f0370109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.406+0000 7f0377a59640 1 --2- 172.21.15.112:0/924723855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03701029f0 0x7f0370102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.406+0000 7f0377a59640 1 --2- 172.21.15.112:0/924723855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0370101800 0x7f0370101c20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.406+0000 7f0377a59640 1 -- 172.21.15.112:0/924723855 >> 172.21.15.112:0/924723855 conn(0x7f03700fcd90 msgr2=0x7f03700ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:27.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.406+0000 7f0377a59640 1 -- 172.21.15.112:0/924723855 shutdown_connections 2024-03-16T09:15:27.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.406+0000 7f0377a59640 1 -- 172.21.15.112:0/924723855 wait complete. 2024-03-16T09:15:27.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.406+0000 7f0377a59640 1 Processor -- start 2024-03-16T09:15:27.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f0377a59640 1 -- start start 2024-03-16T09:15:27.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f0377a59640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0370101800 0x7f0370075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:27.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f0377a59640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03701029f0 0x7f0370073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:27.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f0377a59640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03701033b0 0x7f0370074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:27.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f0377a59640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f037010d550 con 0x7f03701029f0 2024-03-16T09:15:27.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f0377a59640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f037010d3d0 con 0x7f03701033b0 2024-03-16T09:15:27.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f0377a59640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f037010d6d0 con 0x7f0370101800 2024-03-16T09:15:27.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f0374ff4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03701029f0 0x7f0370073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:27.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f0374ff4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03701029f0 0x7f0370073bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57552/0 (socket says 172.21.15.112:57552) 2024-03-16T09:15:27.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f0374ff4640 1 -- 172.21.15.112:0/1905886471 learned_addr learned my addr 172.21.15.112:0/1905886471 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:27.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f03757f5640 1 --2- 172.21.15.112:0/1905886471 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0370101800 0x7f0370075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:27.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f0375ff6640 1 --2- 172.21.15.112:0/1905886471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03701033b0 0x7f0370074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:27.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f0374ff4640 1 -- 172.21.15.112:0/1905886471 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0370101800 msgr2=0x7f0370075590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:27.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f0374ff4640 1 --2- 172.21.15.112:0/1905886471 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0370101800 0x7f0370075590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f0374ff4640 1 -- 172.21.15.112:0/1905886471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03701033b0 msgr2=0x7f0370074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:27.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f0374ff4640 1 --2- 172.21.15.112:0/1905886471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03701033b0 0x7f0370074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f0374ff4640 1 -- 172.21.15.112:0/1905886471 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03700749a0 con 0x7f03701029f0 2024-03-16T09:15:27.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f0374ff4640 1 --2- 172.21.15.112:0/1905886471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03701029f0 0x7f0370073bd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f036c00e9f0 tx=0x7f036c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:27.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f03667fc640 1 -- 172.21.15.112:0/1905886471 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c00cd80 con 0x7f03701029f0 2024-03-16T09:15:27.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f0377a59640 1 -- 172.21.15.112:0/1905886471 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03701a1d10 con 0x7f03701029f0 2024-03-16T09:15:27.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f03667fc640 1 -- 172.21.15.112:0/1905886471 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f036c010930 con 0x7f03701029f0 2024-03-16T09:15:27.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f03667fc640 1 -- 172.21.15.112:0/1905886471 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c018b00 con 0x7f03701029f0 2024-03-16T09:15:27.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.407+0000 7f0377a59640 1 -- 172.21.15.112:0/1905886471 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03701a21f0 con 0x7f03701029f0 2024-03-16T09:15:27.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.408+0000 7f0377a59640 1 -- 172.21.15.112:0/1905886471 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0370108b20 con 0x7f03701029f0 2024-03-16T09:15:27.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.409+0000 7f03667fc640 1 -- 172.21.15.112:0/1905886471 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f036c018ca0 con 0x7f03701029f0 2024-03-16T09:15:27.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.410+0000 7f03667fc640 1 --2- 172.21.15.112:0/1905886471 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f035c0a2800 0x7f035c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:27.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.410+0000 7f03667fc640 1 -- 172.21.15.112:0/1905886471 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f036c0cf0f0 con 0x7f03701029f0 2024-03-16T09:15:27.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.410+0000 7f03757f5640 1 --2- 172.21.15.112:0/1905886471 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f035c0a2800 0x7f035c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:27.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.410+0000 7f03757f5640 1 --2- 172.21.15.112:0/1905886471 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f035c0a2800 0x7f035c0a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0360000c00 tx=0x7f0360005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:27.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.414+0000 7f03667fc640 1 -- 172.21.15.112:0/1905886471 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f036c09bb50 con 0x7f03701029f0 2024-03-16T09:15:27.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:27 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1168327168' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:27.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:27 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/224362170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:27.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:27 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2027058757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:27.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:27 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/837370799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:27.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.591+0000 7f0377a59640 1 -- 172.21.15.112:0/1905886471 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f037002d330 con 0x7f03701029f0 2024-03-16T09:15:27.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.592+0000 7f03667fc640 1 -- 172.21.15.112:0/1905886471 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22725 (secure 0 0 0) 0x7f036c09fbf0 con 0x7f03701029f0 2024-03-16T09:15:27.594 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:27.594 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":87,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:14:31.991998+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T09:13:35.040628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T09:13:35.689419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:15:27.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.594+0000 7f0377a59640 1 -- 172.21.15.112:0/1905886471 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f035c0a2800 msgr2=0x7f035c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:27.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.594+0000 7f0377a59640 1 --2- 172.21.15.112:0/1905886471 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f035c0a2800 0x7f035c0a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0360000c00 tx=0x7f0360005fd0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.594+0000 7f0377a59640 1 -- 172.21.15.112:0/1905886471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03701029f0 msgr2=0x7f0370073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:27.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.594+0000 7f0377a59640 1 --2- 172.21.15.112:0/1905886471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03701029f0 0x7f0370073bd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f036c00e9f0 tx=0x7f036c00eeb0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.594+0000 7f0377a59640 1 -- 172.21.15.112:0/1905886471 shutdown_connections 2024-03-16T09:15:27.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.594+0000 7f0377a59640 1 --2- 172.21.15.112:0/1905886471 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f035c0a2800 0x7f035c0a4cc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.594+0000 7f0377a59640 1 --2- 172.21.15.112:0/1905886471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03701033b0 0x7f0370074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.594+0000 7f0377a59640 1 --2- 172.21.15.112:0/1905886471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03701029f0 0x7f0370073bd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.594+0000 7f0377a59640 1 --2- 172.21.15.112:0/1905886471 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0370101800 0x7f0370075590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.594+0000 7f0377a59640 1 -- 172.21.15.112:0/1905886471 >> 172.21.15.112:0/1905886471 conn(0x7f03700fcd90 msgr2=0x7f0370105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:27.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.594+0000 7f0377a59640 1 -- 172.21.15.112:0/1905886471 shutdown_connections 2024-03-16T09:15:27.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.594+0000 7f0377a59640 1 -- 172.21.15.112:0/1905886471 wait complete. 2024-03-16T09:15:27.608 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T09:08:57.135312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T09:13:35.040628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T09:13:35.689419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T09:15:27.608 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T09:15:27.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.711+0000 7f0bc1e8c640 1 -- 172.21.15.112:0/1937695224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bbc0ffad0 msgr2=0x7f0bbc0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:27.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.711+0000 7f0bc1e8c640 1 --2- 172.21.15.112:0/1937695224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bbc0ffad0 0x7f0bbc0ffef0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0bac009930 tx=0x7f0bac02bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:27.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.711+0000 7f0bc1e8c640 1 -- 172.21.15.112:0/1937695224 shutdown_connections 2024-03-16T09:15:27.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.711+0000 7f0bc1e8c640 1 --2- 172.21.15.112:0/1937695224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bbc103460 0x7f0bbc109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.711+0000 7f0bc1e8c640 1 --2- 172.21.15.112:0/1937695224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bbc102aa0 0x7f0bbc102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.711+0000 7f0bc1e8c640 1 --2- 172.21.15.112:0/1937695224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bbc0ffad0 0x7f0bbc0ffef0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.711+0000 7f0bc1e8c640 1 -- 172.21.15.112:0/1937695224 >> 172.21.15.112:0/1937695224 conn(0x7f0bbc0fb3e0 msgr2=0x7f0bbc0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:27.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.711+0000 7f0bc1e8c640 1 -- 172.21.15.112:0/1937695224 shutdown_connections 2024-03-16T09:15:27.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.712+0000 7f0bc1e8c640 1 -- 172.21.15.112:0/1937695224 wait complete. 2024-03-16T09:15:27.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.712+0000 7f0bc1e8c640 1 Processor -- start 2024-03-16T09:15:27.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.712+0000 7f0bc1e8c640 1 -- start start 2024-03-16T09:15:27.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.712+0000 7f0bc1e8c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bbc0ffad0 0x7f0bbc197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:27.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.712+0000 7f0bc1e8c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bbc102aa0 0x7f0bbc198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:27.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.712+0000 7f0bc1e8c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bbc103460 0x7f0bbc19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:27.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.712+0000 7f0bc1e8c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc10d590 con 0x7f0bbc103460 2024-03-16T09:15:27.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.712+0000 7f0bc1e8c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc10d410 con 0x7f0bbc0ffad0 2024-03-16T09:15:27.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.712+0000 7f0bc1e8c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc10d710 con 0x7f0bbc102aa0 2024-03-16T09:15:27.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.712+0000 7f0bbbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bbc103460 0x7f0bbc19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:27.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.712+0000 7f0bbbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bbc103460 0x7f0bbc19f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57564/0 (socket says 172.21.15.112:57564) 2024-03-16T09:15:27.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.712+0000 7f0bbbfff640 1 -- 172.21.15.112:0/1149089385 learned_addr learned my addr 172.21.15.112:0/1149089385 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:27.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.712+0000 7f0bbb7fe640 1 --2- 172.21.15.112:0/1149089385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bbc0ffad0 0x7f0bbc197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:27.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.712+0000 7f0bb3fff640 1 --2- 172.21.15.112:0/1149089385 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bbc102aa0 0x7f0bbc198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:27.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.713+0000 7f0bbbfff640 1 -- 172.21.15.112:0/1149089385 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bbc102aa0 msgr2=0x7f0bbc198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:27.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.713+0000 7f0bbbfff640 1 --2- 172.21.15.112:0/1149089385 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bbc102aa0 0x7f0bbc198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.713+0000 7f0bbbfff640 1 -- 172.21.15.112:0/1149089385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bbc0ffad0 msgr2=0x7f0bbc197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:27.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.713+0000 7f0bbbfff640 1 --2- 172.21.15.112:0/1149089385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bbc0ffad0 0x7f0bbc197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.713+0000 7f0bbbfff640 1 -- 172.21.15.112:0/1149089385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bbc19f860 con 0x7f0bbc103460 2024-03-16T09:15:27.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.713+0000 7f0bbbfff640 1 --2- 172.21.15.112:0/1149089385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bbc103460 0x7f0bbc19f1c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400eb10 tx=0x7f0bb400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:27.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.713+0000 7f0bb97fa640 1 -- 172.21.15.112:0/1149089385 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4004b20 con 0x7f0bbc103460 2024-03-16T09:15:27.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.713+0000 7f0bb97fa640 1 -- 172.21.15.112:0/1149089385 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0bb4004cc0 con 0x7f0bbc103460 2024-03-16T09:15:27.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.713+0000 7f0bb97fa640 1 -- 172.21.15.112:0/1149089385 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb40107f0 con 0x7f0bbc103460 2024-03-16T09:15:27.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.713+0000 7f0bc1e8c640 1 -- 172.21.15.112:0/1149089385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bbc19fb50 con 0x7f0bbc103460 2024-03-16T09:15:27.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.713+0000 7f0bc1e8c640 1 -- 172.21.15.112:0/1149089385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bbc1a0020 con 0x7f0bbc103460 2024-03-16T09:15:27.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.714+0000 7f0bc1e8c640 1 -- 172.21.15.112:0/1149089385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bbc108bd0 con 0x7f0bbc103460 2024-03-16T09:15:27.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.715+0000 7f0bb97fa640 1 -- 172.21.15.112:0/1149089385 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f0bb4005020 con 0x7f0bbc103460 2024-03-16T09:15:27.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.715+0000 7f0bb97fa640 1 --2- 172.21.15.112:0/1149089385 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f0ba80a2800 0x7f0ba80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:27.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.715+0000 7f0bbb7fe640 1 --2- 172.21.15.112:0/1149089385 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f0ba80a2800 0x7f0ba80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:27.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.715+0000 7f0bb97fa640 1 -- 172.21.15.112:0/1149089385 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f0bb40cefd0 con 0x7f0bbc103460 2024-03-16T09:15:27.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.716+0000 7f0bbb7fe640 1 --2- 172.21.15.112:0/1149089385 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f0ba80a2800 0x7f0ba80a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0bac02c900 tx=0x7f0bac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:27.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.720+0000 7f0bb97fa640 1 -- 172.21.15.112:0/1149089385 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0bb409c120 con 0x7f0bbc103460 2024-03-16T09:15:27.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.892+0000 7f0bc1e8c640 1 -- 172.21.15.112:0/1149089385 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0bbc02d400 con 0x7f0bbc103460 2024-03-16T09:15:27.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.892+0000 7f0bb97fa640 1 -- 172.21.15.112:0/1149089385 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f0bb409ba20 con 0x7f0bbc103460 2024-03-16T09:15:27.894 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-03-16T09:15:27.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.894+0000 7f0bc1e8c640 1 -- 172.21.15.112:0/1149089385 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f0ba80a2800 msgr2=0x7f0ba80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:27.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.894+0000 7f0bc1e8c640 1 --2- 172.21.15.112:0/1149089385 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f0ba80a2800 0x7f0ba80a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0bac02c900 tx=0x7f0bac005fd0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.894+0000 7f0bc1e8c640 1 -- 172.21.15.112:0/1149089385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bbc103460 msgr2=0x7f0bbc19f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:27.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.894+0000 7f0bc1e8c640 1 --2- 172.21.15.112:0/1149089385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bbc103460 0x7f0bbc19f1c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400eb10 tx=0x7f0bb400c2d0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.894+0000 7f0bc1e8c640 1 -- 172.21.15.112:0/1149089385 shutdown_connections 2024-03-16T09:15:27.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.894+0000 7f0bc1e8c640 1 --2- 172.21.15.112:0/1149089385 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f0ba80a2800 0x7f0ba80a4cc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.894+0000 7f0bc1e8c640 1 --2- 172.21.15.112:0/1149089385 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bbc103460 0x7f0bbc19f1c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.894+0000 7f0bc1e8c640 1 --2- 172.21.15.112:0/1149089385 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0bbc102aa0 0x7f0bbc198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.894+0000 7f0bc1e8c640 1 --2- 172.21.15.112:0/1149089385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bbc0ffad0 0x7f0bbc197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:27.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.894+0000 7f0bc1e8c640 1 -- 172.21.15.112:0/1149089385 >> 172.21.15.112:0/1149089385 conn(0x7f0bbc0fb3e0 msgr2=0x7f0bbc105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:27.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.894+0000 7f0bc1e8c640 1 -- 172.21.15.112:0/1149089385 shutdown_connections 2024-03-16T09:15:27.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:27.894+0000 7f0bc1e8c640 1 -- 172.21.15.112:0/1149089385 wait complete. 2024-03-16T09:15:27.907 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T09:15:28.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.011+0000 7f6c04309640 1 -- 172.21.15.112:0/3700942007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bfc1018e0 msgr2=0x7f6bfc101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:28.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.011+0000 7f6c04309640 1 --2- 172.21.15.112:0/3700942007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bfc1018e0 0x7f6bfc101d60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009900 tx=0x7f6bec02bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:28.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.011+0000 7f6c04309640 1 -- 172.21.15.112:0/3700942007 shutdown_connections 2024-03-16T09:15:28.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.011+0000 7f6c04309640 1 --2- 172.21.15.112:0/3700942007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bfc1022a0 0x7f6bfc108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.011+0000 7f6c04309640 1 --2- 172.21.15.112:0/3700942007 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bfc1018e0 0x7f6bfc101d60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.011+0000 7f6c04309640 1 --2- 172.21.15.112:0/3700942007 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6bfc1006f0 0x7f6bfc100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.011+0000 7f6c04309640 1 -- 172.21.15.112:0/3700942007 >> 172.21.15.112:0/3700942007 conn(0x7f6bfc0fbc80 msgr2=0x7f6bfc0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.011+0000 7f6c04309640 1 -- 172.21.15.112:0/3700942007 shutdown_connections 2024-03-16T09:15:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.011+0000 7f6c04309640 1 -- 172.21.15.112:0/3700942007 wait complete. 2024-03-16T09:15:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.011+0000 7f6c04309640 1 Processor -- start 2024-03-16T09:15:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c04309640 1 -- start start 2024-03-16T09:15:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c04309640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bfc1006f0 0x7f6bfc06a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c04309640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6bfc1018e0 0x7f6bfc19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c04309640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bfc1022a0 0x7f6bfc19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c04309640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc10c450 con 0x7f6bfc1022a0 2024-03-16T09:15:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c04309640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc10c2d0 con 0x7f6bfc1006f0 2024-03-16T09:15:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c04309640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc10c5d0 con 0x7f6bfc1018e0 2024-03-16T09:15:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c028a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bfc1022a0 0x7f6bfc19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c028a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bfc1022a0 0x7f6bfc19ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57576/0 (socket says 172.21.15.112:57576) 2024-03-16T09:15:28.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c028a6640 1 -- 172.21.15.112:0/3218717631 learned_addr learned my addr 172.21.15.112:0/3218717631 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:28.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c020a5640 1 --2- 172.21.15.112:0/3218717631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bfc1006f0 0x7f6bfc06a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:28.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c018a4640 1 --2- 172.21.15.112:0/3218717631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6bfc1018e0 0x7f6bfc19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:28.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c028a6640 1 -- 172.21.15.112:0/3218717631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6bfc1018e0 msgr2=0x7f6bfc19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:28.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c028a6640 1 --2- 172.21.15.112:0/3218717631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6bfc1018e0 0x7f6bfc19c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c028a6640 1 -- 172.21.15.112:0/3218717631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bfc1006f0 msgr2=0x7f6bfc06a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:28.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c028a6640 1 --2- 172.21.15.112:0/3218717631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bfc1006f0 0x7f6bfc06a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c028a6640 1 -- 172.21.15.112:0/3218717631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bfc19f810 con 0x7f6bfc1022a0 2024-03-16T09:15:28.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c020a5640 1 --2- 172.21.15.112:0/3218717631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bfc1006f0 0x7f6bfc06a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:28.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c028a6640 1 --2- 172.21.15.112:0/3218717631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bfc1022a0 0x7f6bfc19ef50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400eb10 tx=0x7f6bf400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:28.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6bf37fe640 1 -- 172.21.15.112:0/3218717631 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4004b20 con 0x7f6bfc1022a0 2024-03-16T09:15:28.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6bf37fe640 1 -- 172.21.15.112:0/3218717631 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6bf4004cc0 con 0x7f6bfc1022a0 2024-03-16T09:15:28.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c04309640 1 -- 172.21.15.112:0/3218717631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bfc19fb00 con 0x7f6bfc1022a0 2024-03-16T09:15:28.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6bf37fe640 1 -- 172.21.15.112:0/3218717631 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40107f0 con 0x7f6bfc1022a0 2024-03-16T09:15:28.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.012+0000 7f6c04309640 1 -- 172.21.15.112:0/3218717631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bfc19ffd0 con 0x7f6bfc1022a0 2024-03-16T09:15:28.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.013+0000 7f6c04309640 1 -- 172.21.15.112:0/3218717631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bfc107a90 con 0x7f6bfc1022a0 2024-03-16T09:15:28.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.014+0000 7f6bf37fe640 1 -- 172.21.15.112:0/3218717631 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f6bf4005020 con 0x7f6bfc1022a0 2024-03-16T09:15:28.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.015+0000 7f6bf37fe640 1 --2- 172.21.15.112:0/3218717631 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6be80a2800 0x7f6be80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:28.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.015+0000 7f6c020a5640 1 --2- 172.21.15.112:0/3218717631 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6be80a2800 0x7f6be80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:28.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.015+0000 7f6bf37fe640 1 -- 172.21.15.112:0/3218717631 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f6bf40cf000 con 0x7f6bfc1022a0 2024-03-16T09:15:28.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.015+0000 7f6c020a5640 1 --2- 172.21.15.112:0/3218717631 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6be80a2800 0x7f6be80a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc101740 tx=0x7f6bf8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:28.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.020+0000 7f6bf37fe640 1 -- 172.21.15.112:0/3218717631 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6bf409c0b0 con 0x7f6bfc1022a0 2024-03-16T09:15:28.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.196+0000 7f6c04309640 1 -- 172.21.15.112:0/3218717631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6bfc0624e0 con 0x7f6bfc1022a0 2024-03-16T09:15:28.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.196+0000 7f6bf37fe640 1 -- 172.21.15.112:0/3218717631 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f6bf409b9b0 con 0x7f6bfc1022a0 2024-03-16T09:15:28.198 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-03-16T09:15:28.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.198+0000 7f6c04309640 1 -- 172.21.15.112:0/3218717631 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6be80a2800 msgr2=0x7f6be80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:28.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.198+0000 7f6c04309640 1 --2- 172.21.15.112:0/3218717631 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6be80a2800 0x7f6be80a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc101740 tx=0x7f6bf8005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:28.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.198+0000 7f6c04309640 1 -- 172.21.15.112:0/3218717631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bfc1022a0 msgr2=0x7f6bfc19ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.198+0000 7f6c04309640 1 --2- 172.21.15.112:0/3218717631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bfc1022a0 0x7f6bfc19ef50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400eb10 tx=0x7f6bf400c2d0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.198+0000 7f6c04309640 1 -- 172.21.15.112:0/3218717631 shutdown_connections 2024-03-16T09:15:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.198+0000 7f6c04309640 1 --2- 172.21.15.112:0/3218717631 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6be80a2800 0x7f6be80a4cc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.198+0000 7f6c04309640 1 --2- 172.21.15.112:0/3218717631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bfc1022a0 0x7f6bfc19ef50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.198+0000 7f6c04309640 1 --2- 172.21.15.112:0/3218717631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6bfc1018e0 0x7f6bfc19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.198+0000 7f6c04309640 1 --2- 172.21.15.112:0/3218717631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bfc1006f0 0x7f6bfc06a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.198+0000 7f6c04309640 1 -- 172.21.15.112:0/3218717631 >> 172.21.15.112:0/3218717631 conn(0x7f6bfc0fbc80 msgr2=0x7f6bfc104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.198+0000 7f6c04309640 1 -- 172.21.15.112:0/3218717631 shutdown_connections 2024-03-16T09:15:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.198+0000 7f6c04309640 1 -- 172.21.15.112:0/3218717631 wait complete. 2024-03-16T09:15:28.210 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:28 smithi112 ceph-mon[32307]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:15:28.210 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:28 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2446529376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:28.210 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:28 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1905886471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:28.210 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:28 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1149089385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:28.211 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T09:15:28.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.314+0000 7f5656eb2640 1 -- 172.21.15.112:0/1923181581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56501022a0 msgr2=0x7f5650108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:28.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.314+0000 7f5656eb2640 1 --2- 172.21.15.112:0/1923181581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56501022a0 0x7f5650108b30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5648009ca0 tx=0x7f564802c0f0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.314+0000 7f5656eb2640 1 -- 172.21.15.112:0/1923181581 shutdown_connections 2024-03-16T09:15:28.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.314+0000 7f5656eb2640 1 --2- 172.21.15.112:0/1923181581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56501022a0 0x7f5650108b30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.314+0000 7f5656eb2640 1 --2- 172.21.15.112:0/1923181581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56501018e0 0x7f5650101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.314+0000 7f5656eb2640 1 --2- 172.21.15.112:0/1923181581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56501006f0 0x7f5650100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.314+0000 7f5656eb2640 1 -- 172.21.15.112:0/1923181581 >> 172.21.15.112:0/1923181581 conn(0x7f56500fbc80 msgr2=0x7f56500fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:28.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.314+0000 7f5656eb2640 1 -- 172.21.15.112:0/1923181581 shutdown_connections 2024-03-16T09:15:28.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.314+0000 7f5656eb2640 1 -- 172.21.15.112:0/1923181581 wait complete. 2024-03-16T09:15:28.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f5656eb2640 1 Processor -- start 2024-03-16T09:15:28.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f5656eb2640 1 -- start start 2024-03-16T09:15:28.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f5656eb2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56501006f0 0x7f565006a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:28.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f5656eb2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56501018e0 0x7f565019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:28.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f5656eb2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56501022a0 0x7f565019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:28.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f5656eb2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f565010c4a0 con 0x7f56501018e0 2024-03-16T09:15:28.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f5656eb2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f565010c320 con 0x7f56501006f0 2024-03-16T09:15:28.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f5656eb2640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f565010c620 con 0x7f56501022a0 2024-03-16T09:15:28.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f5647fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56501018e0 0x7f565019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:28.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f5647fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56501018e0 0x7f565019c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57594/0 (socket says 172.21.15.112:57594) 2024-03-16T09:15:28.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f5647fff640 1 -- 172.21.15.112:0/1802597449 learned_addr learned my addr 172.21.15.112:0/1802597449 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:28.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f5654c4e640 1 --2- 172.21.15.112:0/1802597449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56501006f0 0x7f565006a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:28.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f565544f640 1 --2- 172.21.15.112:0/1802597449 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56501022a0 0x7f565019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:28.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f5647fff640 1 -- 172.21.15.112:0/1802597449 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56501022a0 msgr2=0x7f565019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:28.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f5647fff640 1 --2- 172.21.15.112:0/1802597449 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56501022a0 0x7f565019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f5647fff640 1 -- 172.21.15.112:0/1802597449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56501006f0 msgr2=0x7f565006a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:28.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f5647fff640 1 --2- 172.21.15.112:0/1802597449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56501006f0 0x7f565006a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f5647fff640 1 -- 172.21.15.112:0/1802597449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565019f810 con 0x7f56501018e0 2024-03-16T09:15:28.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f5654c4e640 1 --2- 172.21.15.112:0/1802597449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56501006f0 0x7f565006a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:28.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.315+0000 7f5647fff640 1 --2- 172.21.15.112:0/1802597449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56501018e0 0x7f565019c8e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f564000d950 tx=0x7f564000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:28.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.316+0000 7f5645ffb640 1 -- 172.21.15.112:0/1802597449 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640017030 con 0x7f56501018e0 2024-03-16T09:15:28.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.316+0000 7f5656eb2640 1 -- 172.21.15.112:0/1802597449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f565019faa0 con 0x7f56501018e0 2024-03-16T09:15:28.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.316+0000 7f5645ffb640 1 -- 172.21.15.112:0/1802597449 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5640004020 con 0x7f56501018e0 2024-03-16T09:15:28.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.316+0000 7f5645ffb640 1 -- 172.21.15.112:0/1802597449 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640004a00 con 0x7f56501018e0 2024-03-16T09:15:28.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.316+0000 7f5656eb2640 1 -- 172.21.15.112:0/1802597449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f565019fff0 con 0x7f56501018e0 2024-03-16T09:15:28.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.317+0000 7f5656eb2640 1 -- 172.21.15.112:0/1802597449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5650107a10 con 0x7f56501018e0 2024-03-16T09:15:28.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.318+0000 7f5645ffb640 1 -- 172.21.15.112:0/1802597449 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f56400041d0 con 0x7f56501018e0 2024-03-16T09:15:28.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.318+0000 7f5645ffb640 1 --2- 172.21.15.112:0/1802597449 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f56380a2800 0x7f56380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:28.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.318+0000 7f5654c4e640 1 --2- 172.21.15.112:0/1802597449 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f56380a2800 0x7f56380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:28.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.318+0000 7f5645ffb640 1 -- 172.21.15.112:0/1802597449 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f56400ce3a0 con 0x7f56501018e0 2024-03-16T09:15:28.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.319+0000 7f5654c4e640 1 --2- 172.21.15.112:0/1802597449 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f56380a2800 0x7f56380a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5650101740 tx=0x7f564c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:28.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.323+0000 7f5645ffb640 1 -- 172.21.15.112:0/1802597449 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f564009b450 con 0x7f56501018e0 2024-03-16T09:15:28.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:28 smithi194 ceph-mon[36729]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:15:28.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:28 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2446529376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:28.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:28 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1905886471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:28.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:28 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1149089385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:28.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:28 smithi159 ceph-mon[36739]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:15:28.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:28 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2446529376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:28.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:28 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1905886471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:28.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:28 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1149089385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:28.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.499+0000 7f5656eb2640 1 -- 172.21.15.112:0/1802597449 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f56500623e0 con 0x7f56501018e0 2024-03-16T09:15:28.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.500+0000 7f5645ffb640 1 -- 172.21.15.112:0/1802597449 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f564009ad50 con 0x7f56501018e0 2024-03-16T09:15:28.502 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-03-16T09:15:28.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.502+0000 7f5656eb2640 1 -- 172.21.15.112:0/1802597449 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f56380a2800 msgr2=0x7f56380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:28.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.502+0000 7f5656eb2640 1 --2- 172.21.15.112:0/1802597449 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f56380a2800 0x7f56380a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5650101740 tx=0x7f564c005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:28.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.502+0000 7f5656eb2640 1 -- 172.21.15.112:0/1802597449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56501018e0 msgr2=0x7f565019c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:28.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.502+0000 7f5656eb2640 1 --2- 172.21.15.112:0/1802597449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56501018e0 0x7f565019c8e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f564000d950 tx=0x7f564000de10 comp rx=0 tx=0).stop 2024-03-16T09:15:28.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.502+0000 7f5656eb2640 1 -- 172.21.15.112:0/1802597449 shutdown_connections 2024-03-16T09:15:28.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.502+0000 7f5656eb2640 1 --2- 172.21.15.112:0/1802597449 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f56380a2800 0x7f56380a4cc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.502+0000 7f5656eb2640 1 --2- 172.21.15.112:0/1802597449 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56501022a0 0x7f565019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.502+0000 7f5656eb2640 1 --2- 172.21.15.112:0/1802597449 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56501018e0 0x7f565019c8e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.502+0000 7f5656eb2640 1 --2- 172.21.15.112:0/1802597449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56501006f0 0x7f565006a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.502+0000 7f5656eb2640 1 -- 172.21.15.112:0/1802597449 >> 172.21.15.112:0/1802597449 conn(0x7f56500fbc80 msgr2=0x7f5650104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:28.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.502+0000 7f5656eb2640 1 -- 172.21.15.112:0/1802597449 shutdown_connections 2024-03-16T09:15:28.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.502+0000 7f5656eb2640 1 -- 172.21.15.112:0/1802597449 wait complete. 2024-03-16T09:15:28.515 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:15:28.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.614+0000 7f8916790640 1 -- 172.21.15.112:0/3807215034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8910100fe0 msgr2=0x7f8910101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:28.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.614+0000 7f8916790640 1 --2- 172.21.15.112:0/3807215034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8910100fe0 0x7f8910101460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8904009900 tx=0x7f890402bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:28.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.614+0000 7f8916790640 1 -- 172.21.15.112:0/3807215034 shutdown_connections 2024-03-16T09:15:28.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.614+0000 7f8916790640 1 --2- 172.21.15.112:0/3807215034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89101019a0 0x7f89101082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.614+0000 7f8916790640 1 --2- 172.21.15.112:0/3807215034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8910100fe0 0x7f8910101460 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.614+0000 7f8916790640 1 --2- 172.21.15.112:0/3807215034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89100ffe90 0x7f89101002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.614+0000 7f8916790640 1 -- 172.21.15.112:0/3807215034 >> 172.21.15.112:0/3807215034 conn(0x7f89100fb400 msgr2=0x7f89100fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:28.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.614+0000 7f8916790640 1 -- 172.21.15.112:0/3807215034 shutdown_connections 2024-03-16T09:15:28.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.614+0000 7f8916790640 1 -- 172.21.15.112:0/3807215034 wait complete. 2024-03-16T09:15:28.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.615+0000 7f8916790640 1 Processor -- start 2024-03-16T09:15:28.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.615+0000 7f8916790640 1 -- start start 2024-03-16T09:15:28.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.615+0000 7f8916790640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89100ffe90 0x7f8910197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.615+0000 7f8916790640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8910100fe0 0x7f8910198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.615+0000 7f8916790640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89101019a0 0x7f891019f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.615+0000 7f8916790640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f891010b3c0 con 0x7f89101019a0 2024-03-16T09:15:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.615+0000 7f8916790640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f891010b240 con 0x7f8910100fe0 2024-03-16T09:15:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.615+0000 7f8916790640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f891010b540 con 0x7f89100ffe90 2024-03-16T09:15:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.615+0000 7f8914d2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89101019a0 0x7f891019f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.615+0000 7f8914d2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89101019a0 0x7f891019f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57600/0 (socket says 172.21.15.112:57600) 2024-03-16T09:15:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.615+0000 7f8914d2d640 1 -- 172.21.15.112:0/2881191262 learned_addr learned my addr 172.21.15.112:0/2881191262 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.615+0000 7f890f7fe640 1 --2- 172.21.15.112:0/2881191262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8910100fe0 0x7f8910198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.615+0000 7f890ffff640 1 --2- 172.21.15.112:0/2881191262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89100ffe90 0x7f8910197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.615+0000 7f8914d2d640 1 -- 172.21.15.112:0/2881191262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89100ffe90 msgr2=0x7f8910197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.615+0000 7f8914d2d640 1 --2- 172.21.15.112:0/2881191262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89100ffe90 0x7f8910197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.615+0000 7f8914d2d640 1 -- 172.21.15.112:0/2881191262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8910100fe0 msgr2=0x7f8910198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:28.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.615+0000 7f8914d2d640 1 --2- 172.21.15.112:0/2881191262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8910100fe0 0x7f8910198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.615+0000 7f8914d2d640 1 -- 172.21.15.112:0/2881191262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891019f850 con 0x7f89101019a0 2024-03-16T09:15:28.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.616+0000 7f8914d2d640 1 --2- 172.21.15.112:0/2881191262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89101019a0 0x7f891019f1b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f890800af90 tx=0x7f890800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:28.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.616+0000 7f890d7fa640 1 -- 172.21.15.112:0/2881191262 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8908003b70 con 0x7f89101019a0 2024-03-16T09:15:28.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.616+0000 7f8916790640 1 -- 172.21.15.112:0/2881191262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891019fb40 con 0x7f89101019a0 2024-03-16T09:15:28.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.616+0000 7f890d7fa640 1 -- 172.21.15.112:0/2881191262 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8908003d10 con 0x7f89101019a0 2024-03-16T09:15:28.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.616+0000 7f890d7fa640 1 -- 172.21.15.112:0/2881191262 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890800bc10 con 0x7f89101019a0 2024-03-16T09:15:28.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.616+0000 7f8916790640 1 -- 172.21.15.112:0/2881191262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89101a0040 con 0x7f89101019a0 2024-03-16T09:15:28.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.617+0000 7f8916790640 1 -- 172.21.15.112:0/2881191262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8910107240 con 0x7f89101019a0 2024-03-16T09:15:28.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.618+0000 7f890d7fa640 1 -- 172.21.15.112:0/2881191262 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f8908004000 con 0x7f89101019a0 2024-03-16T09:15:28.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.618+0000 7f890d7fa640 1 --2- 172.21.15.112:0/2881191262 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88f80a2800 0x7f88f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:28.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.619+0000 7f890ffff640 1 --2- 172.21.15.112:0/2881191262 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88f80a2800 0x7f88f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:28.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.619+0000 7f890d7fa640 1 -- 172.21.15.112:0/2881191262 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f89080cee70 con 0x7f89101019a0 2024-03-16T09:15:28.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.619+0000 7f890ffff640 1 --2- 172.21.15.112:0/2881191262 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88f80a2800 0x7f88f80a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8910199110 tx=0x7f8900005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:28.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.623+0000 7f890d7fa640 1 -- 172.21.15.112:0/2881191262 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f890809bf20 con 0x7f89101019a0 2024-03-16T09:15:28.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.835+0000 7f8916790640 1 -- 172.21.15.112:0/2881191262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8910062480 con 0x7f89101019a0 2024-03-16T09:15:28.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.836+0000 7f890d7fa640 1 -- 172.21.15.112:0/2881191262 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11881 (secure 0 0 0) 0x7f890809b820 con 0x7f89101019a0 2024-03-16T09:15:28.838 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:28.838 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3939748854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3939748854},{"type":"v1","addr":"172.21.15.112:6839","nonce":3939748854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3118523258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3118523258},{"type":"v1","addr":"172.21.15.112:6841","nonce":3118523258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T09:13:36.450216+0000","modified":"2024-03-16T09:13:40.527486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/1682076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":1682076},{"type":"v1","addr":"172.21.15.194:6837","nonce":1682076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.194:6839/2528943309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2528943309},{"type":"v1","addr":"172.21.15.194:6839","nonce":2528943309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/272895965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":272895965},{"type":"v1","addr":"172.21.15.159:6839","nonce":272895965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24452,24466,24494]},"id":1}]} 2024-03-16T09:15:28.838 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2024-03-16T09:15:28.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.838+0000 7f8916790640 1 -- 172.21.15.112:0/2881191262 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88f80a2800 msgr2=0x7f88f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:28.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.838+0000 7f8916790640 1 --2- 172.21.15.112:0/2881191262 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88f80a2800 0x7f88f80a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8910199110 tx=0x7f8900005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:28.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.838+0000 7f8916790640 1 -- 172.21.15.112:0/2881191262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89101019a0 msgr2=0x7f891019f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:28.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.838+0000 7f8916790640 1 --2- 172.21.15.112:0/2881191262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89101019a0 0x7f891019f1b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f890800af90 tx=0x7f890800c7a0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.838+0000 7f8916790640 1 -- 172.21.15.112:0/2881191262 shutdown_connections 2024-03-16T09:15:28.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.838+0000 7f8916790640 1 --2- 172.21.15.112:0/2881191262 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f88f80a2800 0x7f88f80a4cc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.838+0000 7f8916790640 1 --2- 172.21.15.112:0/2881191262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89101019a0 0x7f891019f1b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.838+0000 7f8916790640 1 --2- 172.21.15.112:0/2881191262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8910100fe0 0x7f8910198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.838+0000 7f8916790640 1 --2- 172.21.15.112:0/2881191262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89100ffe90 0x7f8910197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.838+0000 7f8916790640 1 -- 172.21.15.112:0/2881191262 >> 172.21.15.112:0/2881191262 conn(0x7f89100fb400 msgr2=0x7f8910104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:28.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.838+0000 7f8916790640 1 -- 172.21.15.112:0/2881191262 shutdown_connections 2024-03-16T09:15:28.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.838+0000 7f8916790640 1 -- 172.21.15.112:0/2881191262 wait complete. 2024-03-16T09:15:28.852 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:15:28.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.952+0000 7f59bd04b640 1 -- 172.21.15.112:0/3773238574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59b81018f0 msgr2=0x7f59b8101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:28.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.952+0000 7f59bd04b640 1 --2- 172.21.15.112:0/3773238574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59b81018f0 0x7f59b8101d70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009900 tx=0x7f59a802bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:28.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.952+0000 7f59bd04b640 1 -- 172.21.15.112:0/3773238574 shutdown_connections 2024-03-16T09:15:28.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.952+0000 7f59bd04b640 1 --2- 172.21.15.112:0/3773238574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59b81022b0 0x7f59b8108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.952+0000 7f59bd04b640 1 --2- 172.21.15.112:0/3773238574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59b81018f0 0x7f59b8101d70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.952+0000 7f59bd04b640 1 --2- 172.21.15.112:0/3773238574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59b8100700 0x7f59b8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.952+0000 7f59bd04b640 1 -- 172.21.15.112:0/3773238574 >> 172.21.15.112:0/3773238574 conn(0x7f59b80fbc70 msgr2=0x7f59b80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:28.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.952+0000 7f59bd04b640 1 -- 172.21.15.112:0/3773238574 shutdown_connections 2024-03-16T09:15:28.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.952+0000 7f59bd04b640 1 -- 172.21.15.112:0/3773238574 wait complete. 2024-03-16T09:15:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.953+0000 7f59bd04b640 1 Processor -- start 2024-03-16T09:15:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.953+0000 7f59bd04b640 1 -- start start 2024-03-16T09:15:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.953+0000 7f59bd04b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59b8100700 0x7f59b8197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.953+0000 7f59bd04b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59b81018f0 0x7f59b8198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.953+0000 7f59bd04b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59b81022b0 0x7f59b819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.953+0000 7f59bd04b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59b810c460 con 0x7f59b81022b0 2024-03-16T09:15:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.953+0000 7f59bd04b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f59b810c2e0 con 0x7f59b81018f0 2024-03-16T09:15:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.953+0000 7f59bd04b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f59b810c5e0 con 0x7f59b8100700 2024-03-16T09:15:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.953+0000 7f59b759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59b81022b0 0x7f59b819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.953+0000 7f59b759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59b81022b0 0x7f59b819f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57614/0 (socket says 172.21.15.112:57614) 2024-03-16T09:15:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.953+0000 7f59b759e640 1 -- 172.21.15.112:0/341840226 learned_addr learned my addr 172.21.15.112:0/341840226 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.953+0000 7f59b659c640 1 --2- 172.21.15.112:0/341840226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59b81018f0 0x7f59b8198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:28.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.953+0000 7f59b6d9d640 1 --2- 172.21.15.112:0/341840226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59b8100700 0x7f59b8197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.954+0000 7f59b759e640 1 -- 172.21.15.112:0/341840226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59b8100700 msgr2=0x7f59b8197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.954+0000 7f59b759e640 1 --2- 172.21.15.112:0/341840226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59b8100700 0x7f59b8197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.954+0000 7f59b759e640 1 -- 172.21.15.112:0/341840226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59b81018f0 msgr2=0x7f59b8198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.954+0000 7f59b759e640 1 --2- 172.21.15.112:0/341840226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59b81018f0 0x7f59b8198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.954+0000 7f59b759e640 1 -- 172.21.15.112:0/341840226 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b819f830 con 0x7f59b81022b0 2024-03-16T09:15:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.954+0000 7f59b659c640 1 --2- 172.21.15.112:0/341840226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59b81018f0 0x7f59b8198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.954+0000 7f59b759e640 1 --2- 172.21.15.112:0/341840226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59b81022b0 0x7f59b819f190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f59b000eb10 tx=0x7f59b000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.954+0000 7f599ffff640 1 -- 172.21.15.112:0/341840226 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0004b20 con 0x7f59b81022b0 2024-03-16T09:15:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.954+0000 7f59bd04b640 1 -- 172.21.15.112:0/341840226 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b819fb20 con 0x7f59b81022b0 2024-03-16T09:15:28.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.954+0000 7f599ffff640 1 -- 172.21.15.112:0/341840226 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f59b0004cc0 con 0x7f59b81022b0 2024-03-16T09:15:28.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.954+0000 7f599ffff640 1 -- 172.21.15.112:0/341840226 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b00107f0 con 0x7f59b81022b0 2024-03-16T09:15:28.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.954+0000 7f59bd04b640 1 -- 172.21.15.112:0/341840226 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b81a0020 con 0x7f59b81022b0 2024-03-16T09:15:28.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.955+0000 7f59bd04b640 1 -- 172.21.15.112:0/341840226 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b81a01f0 con 0x7f59b81022b0 2024-03-16T09:15:28.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.956+0000 7f599ffff640 1 -- 172.21.15.112:0/341840226 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f59b0005020 con 0x7f59b81022b0 2024-03-16T09:15:28.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.957+0000 7f599ffff640 1 --2- 172.21.15.112:0/341840226 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f59a00a2800 0x7f59a00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:28.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.957+0000 7f59b6d9d640 1 --2- 172.21.15.112:0/341840226 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f59a00a2800 0x7f59a00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:28.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.957+0000 7f599ffff640 1 -- 172.21.15.112:0/341840226 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f59b00cf090 con 0x7f59b81022b0 2024-03-16T09:15:28.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.958+0000 7f59b6d9d640 1 --2- 172.21.15.112:0/341840226 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f59a00a2800 0x7f59a00a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f59b81990f0 tx=0x7f59ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:28.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:28.962+0000 7f599ffff640 1 -- 172.21.15.112:0/341840226 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f59b009c140 con 0x7f59b81022b0 2024-03-16T09:15:29.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.145+0000 7f59bd04b640 1 -- 172.21.15.112:0/341840226 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59b81a03c0 con 0x7f59b81022b0 2024-03-16T09:15:29.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.147+0000 7f599ffff640 1 -- 172.21.15.112:0/341840226 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22725 (secure 0 0 0) 0x7f59b009ba40 con 0x7f59b81022b0 2024-03-16T09:15:29.149 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:29.149 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":87,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:14:31.991998+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T09:13:35.040628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T09:13:35.689419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:15:29.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.149+0000 7f59bd04b640 1 -- 172.21.15.112:0/341840226 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f59a00a2800 msgr2=0x7f59a00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.149+0000 7f59bd04b640 1 --2- 172.21.15.112:0/341840226 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f59a00a2800 0x7f59a00a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f59b81990f0 tx=0x7f59ac005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:29.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.149+0000 7f59bd04b640 1 -- 172.21.15.112:0/341840226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59b81022b0 msgr2=0x7f59b819f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.149+0000 7f59bd04b640 1 --2- 172.21.15.112:0/341840226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59b81022b0 0x7f59b819f190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f59b000eb10 tx=0x7f59b000c2d0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.149+0000 7f59bd04b640 1 -- 172.21.15.112:0/341840226 shutdown_connections 2024-03-16T09:15:29.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.149+0000 7f59bd04b640 1 --2- 172.21.15.112:0/341840226 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f59a00a2800 0x7f59a00a4cc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.149+0000 7f59bd04b640 1 --2- 172.21.15.112:0/341840226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59b81022b0 0x7f59b819f190 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.149+0000 7f59bd04b640 1 --2- 172.21.15.112:0/341840226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59b81018f0 0x7f59b8198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.149+0000 7f59bd04b640 1 --2- 172.21.15.112:0/341840226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59b8100700 0x7f59b8197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.149+0000 7f59bd04b640 1 -- 172.21.15.112:0/341840226 >> 172.21.15.112:0/341840226 conn(0x7f59b80fbc70 msgr2=0x7f59b80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:29.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.149+0000 7f59bd04b640 1 -- 172.21.15.112:0/341840226 shutdown_connections 2024-03-16T09:15:29.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.149+0000 7f59bd04b640 1 -- 172.21.15.112:0/341840226 wait complete. 2024-03-16T09:15:29.161 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:29 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3218717631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:29.161 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:29 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1802597449' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:29.161 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:29 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2881191262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:29.163 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-16T09:13:36.450216+0000', 'modified': '2024-03-16T09:13:40.527486+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24494, 'mds_1': 24466, 'mds_2': 24452}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24413': {'gid': 24413, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6835/567707106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 567707106}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 567707106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6835/2682964784', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6834', 'nonce': 2682964784}, {'type': 'v1', 'addr': '172.21.15.194:6835', 'nonce': 2682964784}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6837/1682076', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6836', 'nonce': 1682076}, {'type': 'v1', 'addr': '172.21.15.194:6837', 'nonce': 1682076}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.159:6837/4023942203', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 4023942203}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 4023942203}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.194:6839/2528943309', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6838', 'nonce': 2528943309}, {'type': 'v1', 'addr': '172.21.15.194:6839', 'nonce': 2528943309}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.159:6839/272895965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 272895965}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 272895965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24452, 24466, 24494]} 2024-03-16T09:15:29.163 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-16T09:15:29.163 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-16T09:15:29.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.264+0000 7f625ad9e640 1 -- 172.21.15.112:0/3792901121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6254101950 msgr2=0x7f6254101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.264+0000 7f625ad9e640 1 --2- 172.21.15.112:0/3792901121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6254101950 0x7f6254101dd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6248009900 tx=0x7f624802bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:29.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.264+0000 7f625ad9e640 1 -- 172.21.15.112:0/3792901121 shutdown_connections 2024-03-16T09:15:29.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.264+0000 7f625ad9e640 1 --2- 172.21.15.112:0/3792901121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6254102310 0x7f6254108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.264+0000 7f625ad9e640 1 --2- 172.21.15.112:0/3792901121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6254101950 0x7f6254101dd0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.264+0000 7f625ad9e640 1 --2- 172.21.15.112:0/3792901121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6254100760 0x7f6254100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.264+0000 7f625ad9e640 1 -- 172.21.15.112:0/3792901121 >> 172.21.15.112:0/3792901121 conn(0x7f62540fbd10 msgr2=0x7f62540fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:29.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.264+0000 7f625ad9e640 1 -- 172.21.15.112:0/3792901121 shutdown_connections 2024-03-16T09:15:29.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.264+0000 7f625ad9e640 1 -- 172.21.15.112:0/3792901121 wait complete. 2024-03-16T09:15:29.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.265+0000 7f625ad9e640 1 Processor -- start 2024-03-16T09:15:29.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.265+0000 7f625ad9e640 1 -- start start 2024-03-16T09:15:29.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.265+0000 7f625ad9e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6254100760 0x7f625410b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.265+0000 7f625ad9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6254101950 0x7f6254109720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.265+0000 7f625ad9e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6254102310 0x7f6254109c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.265+0000 7f625ad9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f625410d560 con 0x7f6254101950 2024-03-16T09:15:29.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.265+0000 7f625ad9e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f625410d3e0 con 0x7f6254100760 2024-03-16T09:15:29.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.265+0000 7f625ad9e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f625410d6e0 con 0x7f6254102310 2024-03-16T09:15:29.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.265+0000 7f6253fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6254101950 0x7f6254109720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.265+0000 7f6258b3a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6254100760 0x7f625410b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.265+0000 7f6253fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6254101950 0x7f6254109720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57642/0 (socket says 172.21.15.112:57642) 2024-03-16T09:15:29.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.265+0000 7f6253fff640 1 -- 172.21.15.112:0/1048861038 learned_addr learned my addr 172.21.15.112:0/1048861038 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:29.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.265+0000 7f625933b640 1 --2- 172.21.15.112:0/1048861038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6254102310 0x7f6254109c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.265+0000 7f6253fff640 1 -- 172.21.15.112:0/1048861038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6254102310 msgr2=0x7f6254109c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.265+0000 7f6253fff640 1 --2- 172.21.15.112:0/1048861038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6254102310 0x7f6254109c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.265+0000 7f6253fff640 1 -- 172.21.15.112:0/1048861038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6254100760 msgr2=0x7f625410b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.265+0000 7f6253fff640 1 --2- 172.21.15.112:0/1048861038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6254100760 0x7f625410b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.265+0000 7f6253fff640 1 -- 172.21.15.112:0/1048861038 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625410a4f0 con 0x7f6254101950 2024-03-16T09:15:29.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.265+0000 7f6258b3a640 1 --2- 172.21.15.112:0/1048861038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6254100760 0x7f625410b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:29.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.266+0000 7f6253fff640 1 --2- 172.21.15.112:0/1048861038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6254101950 0x7f6254109720 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6248009230 tx=0x7f6248004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:29.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.266+0000 7f6251ffb640 1 -- 172.21.15.112:0/1048861038 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6248043020 con 0x7f6254101950 2024-03-16T09:15:29.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.266+0000 7f625ad9e640 1 -- 172.21.15.112:0/1048861038 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62541a1cf0 con 0x7f6254101950 2024-03-16T09:15:29.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.266+0000 7f6251ffb640 1 -- 172.21.15.112:0/1048861038 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6248004250 con 0x7f6254101950 2024-03-16T09:15:29.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.266+0000 7f6251ffb640 1 -- 172.21.15.112:0/1048861038 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6248003b50 con 0x7f6254101950 2024-03-16T09:15:29.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.266+0000 7f625ad9e640 1 -- 172.21.15.112:0/1048861038 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62541a21d0 con 0x7f6254101950 2024-03-16T09:15:29.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.268+0000 7f6251ffb640 1 -- 172.21.15.112:0/1048861038 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f6248046020 con 0x7f6254101950 2024-03-16T09:15:29.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.268+0000 7f625d988740 1 Processor -- start 2024-03-16T09:15:29.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.268+0000 7f6251ffb640 1 --2- 172.21.15.112:0/1048861038 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f623c0a2800 0x7f623c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.268+0000 7f625d988740 1 -- start start 2024-03-16T09:15:29.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f625d988740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d0bd0 0x56493c2d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f625933b640 1 --2- 172.21.15.112:0/1048861038 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f623c0a2800 0x7f623c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f625d988740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d15c0 0x56493c2d60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f625d988740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d6730 0x56493c2d8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f625d988740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56493c234c80 con 0x56493c2d6730 2024-03-16T09:15:29.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f625d988740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56493c1ddaa0 con 0x56493c2d15c0 2024-03-16T09:15:29.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f6251ffb640 1 -- 172.21.15.112:0/1048861038 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f62480f0b40 con 0x7f6254101950 2024-03-16T09:15:29.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f625d988740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56493c230d20 con 0x56493c2d0bd0 2024-03-16T09:15:29.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f625933b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d6730 0x56493c2d8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f625933b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d6730 0x56493c2d8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57656/0 (socket says 172.21.15.112:57656) 2024-03-16T09:15:29.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f625933b640 1 -- 172.21.15.112:0/1451399437 learned_addr learned my addr 172.21.15.112:0/1451399437 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:29.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f6253fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d15c0 0x56493c2d60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f6258b3a640 1 --2- 172.21.15.112:0/1451399437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d0bd0 0x56493c2d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f625933b640 1 --2- 172.21.15.112:0/1048861038 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f623c0a2800 0x7f623c0a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f624c013830 tx=0x7f624c013c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:29.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f6258b3a640 1 -- 172.21.15.112:0/1451399437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d15c0 msgr2=0x56493c2d60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f6258b3a640 1 --2- 172.21.15.112:0/1451399437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d15c0 0x56493c2d60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f6258b3a640 1 -- 172.21.15.112:0/1451399437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d6730 msgr2=0x56493c2d8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f6258b3a640 1 --2- 172.21.15.112:0/1451399437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d6730 0x56493c2d8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f6258b3a640 1 -- 172.21.15.112:0/1451399437 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56493c1ffff0 con 0x56493c2d0bd0 2024-03-16T09:15:29.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f625933b640 1 --2- 172.21.15.112:0/1451399437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d6730 0x56493c2d8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:29.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f6258b3a640 1 --2- 172.21.15.112:0/1451399437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d0bd0 0x56493c2d0ff0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6244009fa0 tx=0x7f62440118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca30f66c0043df1b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:29.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f623affd640 1 -- 172.21.15.112:0/1451399437 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244004310 con 0x56493c2d0bd0 2024-03-16T09:15:29.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f623affd640 1 -- 172.21.15.112:0/1451399437 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f62440044b0 con 0x56493c2d0bd0 2024-03-16T09:15:29.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.269+0000 7f623affd640 1 -- 172.21.15.112:0/1451399437 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62440035f0 con 0x56493c2d0bd0 2024-03-16T09:15:29.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.270+0000 7f625d988740 1 -- 172.21.15.112:0/1451399437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d0bd0 msgr2=0x56493c2d0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.270+0000 7f625d988740 1 --2- 172.21.15.112:0/1451399437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d0bd0 0x56493c2d0ff0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6244009fa0 tx=0x7f62440118d0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.270+0000 7f625d988740 1 -- 172.21.15.112:0/1451399437 shutdown_connections 2024-03-16T09:15:29.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.270+0000 7f625d988740 1 --2- 172.21.15.112:0/1451399437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d6730 0x56493c2d8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.270+0000 7f625d988740 1 --2- 172.21.15.112:0/1451399437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d15c0 0x56493c2d60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.270+0000 7f625d988740 1 --2- 172.21.15.112:0/1451399437 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d0bd0 0x56493c2d0ff0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.270+0000 7f625d988740 1 -- 172.21.15.112:0/1451399437 >> 172.21.15.112:0/1451399437 conn(0x56493c2cc140 msgr2=0x56493c2cc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:29.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.270+0000 7f625d988740 1 -- 172.21.15.112:0/1451399437 shutdown_connections 2024-03-16T09:15:29.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.270+0000 7f625d988740 1 -- 172.21.15.112:0/1451399437 wait complete. 2024-03-16T09:15:29.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.270+0000 7f625d988740 1 -- start start 2024-03-16T09:15:29.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.270+0000 7f625d988740 1 Processor -- start 2024-03-16T09:15:29.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f625d988740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d0bd0 0x56493c2dad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f625d988740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d15c0 0x56493c2db2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f625d988740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d6730 0x56493c2e2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f625d988740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56493c2d96e0 con 0x56493c2d6730 2024-03-16T09:15:29.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f625d988740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56493c1f0ef0 con 0x56493c2d15c0 2024-03-16T09:15:29.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f625d988740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56493c2d9860 con 0x56493c2d0bd0 2024-03-16T09:15:29.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f625d988740 10 client.0 _ll_register_callbacks cb 0x56493c1ebdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:15:29.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f625933b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d6730 0x56493c2e2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f6253fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d15c0 0x56493c2db2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f6258b3a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d0bd0 0x56493c2dad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f625933b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d6730 0x56493c2e2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57670/0 (socket says 172.21.15.112:57670) 2024-03-16T09:15:29.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f625933b640 1 -- 172.21.15.112:0/3065805498 learned_addr learned my addr 172.21.15.112:0/3065805498 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:29.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f625933b640 1 -- 172.21.15.112:0/3065805498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d0bd0 msgr2=0x56493c2dad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f625933b640 1 --2- 172.21.15.112:0/3065805498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d0bd0 0x56493c2dad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f625933b640 1 -- 172.21.15.112:0/3065805498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d15c0 msgr2=0x56493c2db2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f625933b640 1 --2- 172.21.15.112:0/3065805498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d15c0 0x56493c2db2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f625933b640 1 -- 172.21.15.112:0/3065805498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56493c1fda30 con 0x56493c2d6730 2024-03-16T09:15:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f6258b3a640 1 --2- 172.21.15.112:0/3065805498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d0bd0 0x56493c2dad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f6253fff640 1 --2- 172.21.15.112:0/3065805498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d15c0 0x56493c2db2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f625933b640 1 --2- 172.21.15.112:0/3065805498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d6730 0x56493c2e2310 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f624c003ba0 tx=0x7f624c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f26ee74cd48249c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.271+0000 7f6233fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:15:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.272+0000 7f6233fff640 1 -- 172.21.15.112:0/3065805498 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c004600 con 0x56493c2d6730 2024-03-16T09:15:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.272+0000 7f625d988740 1 -- 172.21.15.112:0/3065805498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62440044b0 con 0x56493c2d6730 2024-03-16T09:15:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.272+0000 7f6233fff640 1 -- 172.21.15.112:0/3065805498 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f624c0047a0 con 0x56493c2d6730 2024-03-16T09:15:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.272+0000 7f6233fff640 1 -- 172.21.15.112:0/3065805498 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f624c010430 con 0x56493c2d6730 2024-03-16T09:15:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.272+0000 7f625d988740 10 client.15198 fetch_fsmap learned FSMap version 21 2024-03-16T09:15:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.272+0000 7f625d988740 1 -- 172.21.15.112:0/3065805498 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56493bf492d0 con 0x56493c2d6730 2024-03-16T09:15:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.272+0000 7f6233fff640 1 -- 172.21.15.112:0/3065805498 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f62440044b0 con 0x56493c2d6730 2024-03-16T09:15:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.272+0000 7f6233fff640 1 -- 172.21.15.112:0/3065805498 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f624c01f070 con 0x56493c2d6730 2024-03-16T09:15:29.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.272+0000 7f625d988740 10 client.15198 fetch_fsmap finished waiting for FSMap version 21 2024-03-16T09:15:29.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.272+0000 7f625d988740 10 client.15198 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-16T09:15:29.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.272+0000 7f625d988740 20 client.15198 populate_metadata read hostname 'smithi112' 2024-03-16T09:15:29.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.272+0000 7f625d988740 1 --2- 172.21.15.112:0/3065805498 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x56493c2e7970 0x56493c2e8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.272+0000 7f625d988740 4 client.15198 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:15:29.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.272+0000 7f625d988740 1 -- 172.21.15.112:0/3065805498 --> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56493c2421c0 con 0x56493c2e7970 2024-03-16T09:15:29.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.272+0000 7f6258b3a640 1 --2- 172.21.15.112:0/3065805498 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x56493c2e7970 0x56493c2e8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.273+0000 7f6233fff640 10 client.15198 ms_handle_connect on v2:172.21.15.159:6836/4023942203 2024-03-16T09:15:29.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.273+0000 7f6258b3a640 1 --2- 172.21.15.112:0/3065805498 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x56493c2e7970 0x56493c2e8de0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37084c1227543099 server_cookie=9e4c2f8ebe612c7 in_seq=0 out_seq=0 2024-03-16T09:15:29.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.273+0000 7f6233fff640 1 -- 172.21.15.112:0/3065805498 <== mds.2 v2:172.21.15.159:6836/4023942203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56493c2421c0 con 0x56493c2e7970 2024-03-16T09:15:29.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.273+0000 7f6233fff640 10 client.15198 handle_command_reply: tid=0 2024-03-16T09:15:29.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.273+0000 7f625d988740 1 client.15198 shutdown 2024-03-16T09:15:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.274+0000 7f625d988740 1 -- 172.21.15.112:0/3065805498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d6730 msgr2=0x56493c2e2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.274+0000 7f625d988740 1 --2- 172.21.15.112:0/3065805498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d6730 0x56493c2e2310 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f624c003ba0 tx=0x7f624c004ab0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.274+0000 7f625d988740 1 -- 172.21.15.112:0/3065805498 shutdown_connections 2024-03-16T09:15:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.274+0000 7f625d988740 1 --2- 172.21.15.112:0/3065805498 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x56493c2e7970 0x56493c2e8de0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.274+0000 7f625d988740 1 --2- 172.21.15.112:0/3065805498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d6730 0x56493c2e2310 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.274+0000 7f6233fff640 0 client.15198 ms_handle_reset on v2:172.21.15.159:6836/4023942203 2024-03-16T09:15:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.274+0000 7f625d988740 1 --2- 172.21.15.112:0/3065805498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d15c0 0x56493c2db2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.274+0000 7f625d988740 1 --2- 172.21.15.112:0/3065805498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d0bd0 0x56493c2dad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.274+0000 7f625d988740 1 -- 172.21.15.112:0/3065805498 >> 172.21.15.112:0/3065805498 conn(0x56493c2cc140 msgr2=0x56493c2cb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.274+0000 7f625d988740 1 -- 172.21.15.112:0/3065805498 shutdown_connections 2024-03-16T09:15:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.274+0000 7f625d988740 1 -- 172.21.15.112:0/3065805498 wait complete. 2024-03-16T09:15:29.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.274+0000 7f625d988740 20 client.15198 trim_cache size 0 max 16384 2024-03-16T09:15:29.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.291+0000 7f625d988740 1 Processor -- start 2024-03-16T09:15:29.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.291+0000 7f625d988740 1 -- start start 2024-03-16T09:15:29.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f625d988740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d0bd0 0x56493c2dad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f625d988740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d15c0 0x56493c2db2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f625d988740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d6730 0x56493c2e2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f625d988740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56493c2e7120 con 0x56493c2d15c0 2024-03-16T09:15:29.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f625d988740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56493c2e77f0 con 0x56493c2d0bd0 2024-03-16T09:15:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f625d988740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56493c2e7ec0 con 0x56493c2d6730 2024-03-16T09:15:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f6253fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d15c0 0x56493c2db2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f6253fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d15c0 0x56493c2db2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57678/0 (socket says 172.21.15.112:57678) 2024-03-16T09:15:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f6253fff640 1 -- 172.21.15.112:0/3848152730 learned_addr learned my addr 172.21.15.112:0/3848152730 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f6258b3a640 1 --2- 172.21.15.112:0/3848152730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d0bd0 0x56493c2dad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f625933b640 1 --2- 172.21.15.112:0/3848152730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d6730 0x56493c2e2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f6253fff640 1 -- 172.21.15.112:0/3848152730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d6730 msgr2=0x56493c2e2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f6253fff640 1 --2- 172.21.15.112:0/3848152730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d6730 0x56493c2e2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f6253fff640 1 -- 172.21.15.112:0/3848152730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d0bd0 msgr2=0x56493c2dad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f6253fff640 1 --2- 172.21.15.112:0/3848152730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d0bd0 0x56493c2dad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f6253fff640 1 -- 172.21.15.112:0/3848152730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56493c0136b0 con 0x56493c2d15c0 2024-03-16T09:15:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f6258b3a640 1 --2- 172.21.15.112:0/3848152730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d0bd0 0x56493c2dad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:15:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f6253fff640 1 --2- 172.21.15.112:0/3848152730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d15c0 0x56493c2db2d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f62480a80b0 tx=0x7f6248041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2823fea685b77c1a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:29.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f623a7fc640 1 -- 172.21.15.112:0/3848152730 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62480bd4f0 con 0x56493c2d15c0 2024-03-16T09:15:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f623a7fc640 1 -- 172.21.15.112:0/3848152730 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f62480c1640 con 0x56493c2d15c0 2024-03-16T09:15:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.292+0000 7f623a7fc640 1 -- 172.21.15.112:0/3848152730 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62480c5800 con 0x56493c2d15c0 2024-03-16T09:15:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.293+0000 7f625d988740 1 -- 172.21.15.112:0/3848152730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d15c0 msgr2=0x56493c2db2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.293+0000 7f625d988740 1 --2- 172.21.15.112:0/3848152730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d15c0 0x56493c2db2d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f62480a80b0 tx=0x7f6248041040 comp rx=0 tx=0).stop 2024-03-16T09:15:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.293+0000 7f625d988740 1 -- 172.21.15.112:0/3848152730 shutdown_connections 2024-03-16T09:15:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.293+0000 7f625d988740 1 --2- 172.21.15.112:0/3848152730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d6730 0x56493c2e2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.293+0000 7f625d988740 1 --2- 172.21.15.112:0/3848152730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d15c0 0x56493c2db2d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.293+0000 7f625d988740 1 --2- 172.21.15.112:0/3848152730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d0bd0 0x56493c2dad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.293+0000 7f625d988740 1 -- 172.21.15.112:0/3848152730 >> 172.21.15.112:0/3848152730 conn(0x56493c2cc140 msgr2=0x56493c2e66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.293+0000 7f625d988740 1 -- 172.21.15.112:0/3848152730 shutdown_connections 2024-03-16T09:15:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.293+0000 7f625d988740 1 -- 172.21.15.112:0/3848152730 wait complete. 2024-03-16T09:15:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.293+0000 7f625d988740 1 -- start start 2024-03-16T09:15:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.293+0000 7f625d988740 1 Processor -- start 2024-03-16T09:15:29.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.293+0000 7f625d988740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d0bd0 0x56493c2d4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.293+0000 7f625d988740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d15c0 0x56493c2e2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f625d988740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d6730 0x56493c2d46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f625d988740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56493c2e8c60 con 0x56493c2d0bd0 2024-03-16T09:15:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f625d988740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56493c2e8590 con 0x56493c2d15c0 2024-03-16T09:15:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f625d988740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56493c2e9330 con 0x56493c2d6730 2024-03-16T09:15:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f625d988740 10 client.0 _ll_register_callbacks cb 0x7f624c00fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:15:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f6258b3a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d0bd0 0x56493c2d4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f6258b3a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d0bd0 0x56493c2d4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57682/0 (socket says 172.21.15.112:57682) 2024-03-16T09:15:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f6258b3a640 1 -- 172.21.15.112:0/4092231309 learned_addr learned my addr 172.21.15.112:0/4092231309 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f6253fff640 1 --2- 172.21.15.112:0/4092231309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d15c0 0x56493c2e2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f625933b640 1 --2- 172.21.15.112:0/4092231309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d6730 0x56493c2d46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f6258b3a640 1 -- 172.21.15.112:0/4092231309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d6730 msgr2=0x56493c2d46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f6258b3a640 1 --2- 172.21.15.112:0/4092231309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d6730 0x56493c2d46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f6258b3a640 1 -- 172.21.15.112:0/4092231309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d15c0 msgr2=0x56493c2e2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f6258b3a640 1 --2- 172.21.15.112:0/4092231309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d15c0 0x56493c2e2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f6258b3a640 1 -- 172.21.15.112:0/4092231309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56493c34b6d0 con 0x56493c2d0bd0 2024-03-16T09:15:29.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f6258b3a640 1 --2- 172.21.15.112:0/4092231309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d0bd0 0x56493c2d4180 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f62440058c0 tx=0x7f6244009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1d6b0fac4b9b33 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:29.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f6233fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:15:29.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f6233fff640 1 -- 172.21.15.112:0/4092231309 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624401d040 con 0x56493c2d0bd0 2024-03-16T09:15:29.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f6233fff640 1 -- 172.21.15.112:0/4092231309 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f62440048e0 con 0x56493c2d0bd0 2024-03-16T09:15:29.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f625d988740 1 -- 172.21.15.112:0/4092231309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62480c1640 con 0x56493c2d0bd0 2024-03-16T09:15:29.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f6233fff640 1 -- 172.21.15.112:0/4092231309 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6244004ae0 con 0x56493c2d0bd0 2024-03-16T09:15:29.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f625d988740 10 client.15204 fetch_fsmap learned FSMap version 21 2024-03-16T09:15:29.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.294+0000 7f625d988740 1 -- 172.21.15.112:0/4092231309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56493c34c8b0 con 0x56493c2d0bd0 2024-03-16T09:15:29.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.295+0000 7f6233fff640 1 -- 172.21.15.112:0/4092231309 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f62480c1640 con 0x56493c2d0bd0 2024-03-16T09:15:29.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.295+0000 7f6233fff640 1 -- 172.21.15.112:0/4092231309 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f6244005000 con 0x56493c2d0bd0 2024-03-16T09:15:29.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.295+0000 7f625d988740 10 client.15204 fetch_fsmap finished waiting for FSMap version 21 2024-03-16T09:15:29.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.295+0000 7f625d988740 10 client.15204 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-16T09:15:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.295+0000 7f625d988740 20 client.15204 populate_metadata read hostname 'smithi112' 2024-03-16T09:15:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.295+0000 7f625d988740 1 --2- 172.21.15.112:0/4092231309 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x56493c351220 0x56493c34d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.295+0000 7f625d988740 4 client.15204 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T09:15:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.295+0000 7f625d988740 1 -- 172.21.15.112:0/4092231309 --> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] -- command(tid 0: {"prefix": "status"}) -- 0x56493c23c510 con 0x56493c351220 2024-03-16T09:15:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.295+0000 7f6253fff640 1 --2- 172.21.15.112:0/4092231309 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x56493c351220 0x56493c34d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.295+0000 7f6233fff640 10 client.15204 ms_handle_connect on v2:172.21.15.159:6836/4023942203 2024-03-16T09:15:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.295+0000 7f6253fff640 1 --2- 172.21.15.112:0/4092231309 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x56493c351220 0x56493c34d660 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d5b09bd624cef98a server_cookie=e5818e9aa37005b6 in_seq=0 out_seq=0 2024-03-16T09:15:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f6233fff640 1 -- 172.21.15.112:0/4092231309 <== mds.2 v2:172.21.15.159:6836/4023942203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56493c23c510 con 0x56493c351220 2024-03-16T09:15:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f6233fff640 10 client.15204 handle_command_reply: tid=0 2024-03-16T09:15:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625d988740 1 client.15204 shutdown 2024-03-16T09:15:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625d988740 1 -- 172.21.15.112:0/4092231309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d0bd0 msgr2=0x56493c2d4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625d988740 1 --2- 172.21.15.112:0/4092231309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d0bd0 0x56493c2d4180 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f62440058c0 tx=0x7f6244009310 comp rx=0 tx=0).stop 2024-03-16T09:15:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625d988740 1 -- 172.21.15.112:0/4092231309 shutdown_connections 2024-03-16T09:15:29.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625d988740 1 --2- 172.21.15.112:0/4092231309 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x56493c351220 0x56493c34d660 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625d988740 1 --2- 172.21.15.112:0/4092231309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56493c2d6730 0x56493c2d46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f6233fff640 0 client.15204 ms_handle_reset on v2:172.21.15.159:6836/4023942203 2024-03-16T09:15:29.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625d988740 1 --2- 172.21.15.112:0/4092231309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56493c2d15c0 0x56493c2e2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625d988740 1 --2- 172.21.15.112:0/4092231309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56493c2d0bd0 0x56493c2d4180 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625d988740 1 -- 172.21.15.112:0/4092231309 >> 172.21.15.112:0/4092231309 conn(0x56493c2cc140 msgr2=0x56493c2e66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:29.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625d988740 1 -- 172.21.15.112:0/4092231309 shutdown_connections 2024-03-16T09:15:29.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625d988740 1 -- 172.21.15.112:0/4092231309 wait complete. 2024-03-16T09:15:29.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625d988740 20 client.15204 trim_cache size 0 max 16384 2024-03-16T09:15:29.306 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:15:29.306 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "273be2a4-e374-11ee-95c9-87774f69a715", 2024-03-16T09:15:29.307 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 2, 2024-03-16T09:15:29.307 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24452, 2024-03-16T09:15:29.307 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-03-16T09:15:29.307 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-03-16T09:15:29.307 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-03-16T09:15:29.307 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 109.16584118900001, 2024-03-16T09:15:29.307 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 21, 2024-03-16T09:15:29.307 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 87, 2024-03-16T09:15:29.307 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 85, 2024-03-16T09:15:29.307 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 129.49158757699999 2024-03-16T09:15:29.307 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:15:29.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625ad9e640 1 -- 172.21.15.112:0/1048861038 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f623c0a2800 msgr2=0x7f623c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625ad9e640 1 --2- 172.21.15.112:0/1048861038 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f623c0a2800 0x7f623c0a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f624c013830 tx=0x7f624c013c30 comp rx=0 tx=0).stop 2024-03-16T09:15:29.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625ad9e640 1 -- 172.21.15.112:0/1048861038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6254101950 msgr2=0x7f6254109720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625ad9e640 1 --2- 172.21.15.112:0/1048861038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6254101950 0x7f6254109720 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6248009230 tx=0x7f6248004ab0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625ad9e640 1 -- 172.21.15.112:0/1048861038 shutdown_connections 2024-03-16T09:15:29.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625ad9e640 1 --2- 172.21.15.112:0/1048861038 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f623c0a2800 0x7f623c0a4cc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625ad9e640 1 --2- 172.21.15.112:0/1048861038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6254102310 0x7f6254109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625ad9e640 1 --2- 172.21.15.112:0/1048861038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6254101950 0x7f6254109720 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.296+0000 7f625ad9e640 1 --2- 172.21.15.112:0/1048861038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6254100760 0x7f625410b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.297+0000 7f625ad9e640 1 -- 172.21.15.112:0/1048861038 >> 172.21.15.112:0/1048861038 conn(0x7f62540fbd10 msgr2=0x7f6254104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:29.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.297+0000 7f625ad9e640 1 -- 172.21.15.112:0/1048861038 shutdown_connections 2024-03-16T09:15:29.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.297+0000 7f625ad9e640 1 -- 172.21.15.112:0/1048861038 wait complete. 2024-03-16T09:15:29.310 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-16T09:15:29.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:29 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3218717631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:29.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:29 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1802597449' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:29.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:29 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2881191262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:29.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.411+0000 7fd6f9fc4640 1 -- 172.21.15.112:0/416361539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f41006e0 msgr2=0x7fd6f4100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.411+0000 7fd6f9fc4640 1 --2- 172.21.15.112:0/416361539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f41006e0 0x7fd6f4100b00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009900 tx=0x7fd6e402bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:29.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.411+0000 7fd6f9fc4640 1 -- 172.21.15.112:0/416361539 shutdown_connections 2024-03-16T09:15:29.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.411+0000 7fd6f9fc4640 1 --2- 172.21.15.112:0/416361539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6f4102290 0x7fd6f4108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.411+0000 7fd6f9fc4640 1 --2- 172.21.15.112:0/416361539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6f41018d0 0x7fd6f4101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.411+0000 7fd6f9fc4640 1 --2- 172.21.15.112:0/416361539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f41006e0 0x7fd6f4100b00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.411+0000 7fd6f9fc4640 1 -- 172.21.15.112:0/416361539 >> 172.21.15.112:0/416361539 conn(0x7fd6f40fbc70 msgr2=0x7fd6f40fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:29.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.411+0000 7fd6f9fc4640 1 -- 172.21.15.112:0/416361539 shutdown_connections 2024-03-16T09:15:29.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.411+0000 7fd6f9fc4640 1 -- 172.21.15.112:0/416361539 wait complete. 2024-03-16T09:15:29.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f9fc4640 1 Processor -- start 2024-03-16T09:15:29.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f9fc4640 1 -- start start 2024-03-16T09:15:29.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f9fc4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f41006e0 0x7fd6f4197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f9fc4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6f41018d0 0x7fd6f4198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f9fc4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6f4102290 0x7fd6f419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f9fc4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f410c440 con 0x7fd6f41006e0 2024-03-16T09:15:29.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f9fc4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f410c2c0 con 0x7fd6f4102290 2024-03-16T09:15:29.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f9fc4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f410c5c0 con 0x7fd6f41018d0 2024-03-16T09:15:29.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f41006e0 0x7fd6f4197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f41006e0 0x7fd6f4197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57702/0 (socket says 172.21.15.112:57702) 2024-03-16T09:15:29.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f37fe640 1 -- 172.21.15.112:0/2298042809 learned_addr learned my addr 172.21.15.112:0/2298042809 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:29.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f3fff640 1 --2- 172.21.15.112:0/2298042809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6f4102290 0x7fd6f419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f2ffd640 1 --2- 172.21.15.112:0/2298042809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6f41018d0 0x7fd6f4198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f37fe640 1 -- 172.21.15.112:0/2298042809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6f41018d0 msgr2=0x7fd6f4198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f37fe640 1 --2- 172.21.15.112:0/2298042809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6f41018d0 0x7fd6f4198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f37fe640 1 -- 172.21.15.112:0/2298042809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6f4102290 msgr2=0x7fd6f419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f37fe640 1 --2- 172.21.15.112:0/2298042809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6f4102290 0x7fd6f419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f37fe640 1 -- 172.21.15.112:0/2298042809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f419f830 con 0x7fd6f41006e0 2024-03-16T09:15:29.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f3fff640 1 --2- 172.21.15.112:0/2298042809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6f4102290 0x7fd6f419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:29.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.412+0000 7fd6f37fe640 1 --2- 172.21.15.112:0/2298042809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f41006e0 0x7fd6f4197c20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4000c00 tx=0x7fd6e4004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:29.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.413+0000 7fd6f0ff9640 1 -- 172.21.15.112:0/2298042809 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4043020 con 0x7fd6f41006e0 2024-03-16T09:15:29.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.413+0000 7fd6f0ff9640 1 -- 172.21.15.112:0/2298042809 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd6e4004250 con 0x7fd6f41006e0 2024-03-16T09:15:29.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.413+0000 7fd6f9fc4640 1 -- 172.21.15.112:0/2298042809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f419fac0 con 0x7fd6f41006e0 2024-03-16T09:15:29.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.413+0000 7fd6f0ff9640 1 -- 172.21.15.112:0/2298042809 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4003b90 con 0x7fd6f41006e0 2024-03-16T09:15:29.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.413+0000 7fd6f9fc4640 1 -- 172.21.15.112:0/2298042809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f41a0010 con 0x7fd6f41006e0 2024-03-16T09:15:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.415+0000 7fd6f0ff9640 1 -- 172.21.15.112:0/2298042809 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7fd6e4046020 con 0x7fd6f41006e0 2024-03-16T09:15:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.415+0000 7fd6fcbac740 1 Processor -- start 2024-03-16T09:15:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.415+0000 7fd6fcbac740 1 -- start start 2024-03-16T09:15:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.415+0000 7fd6fcbac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78aaebd0 0x55bc78aaeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.415+0000 7fd6fcbac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaf5c0 0x55bc78ab40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.415+0000 7fd6fcbac740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78ab4730 0x55bc78ab6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.415+0000 7fd6f0ff9640 1 --2- 172.21.15.112:0/2298042809 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6dc0a2800 0x7fd6dc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.415+0000 7fd6fcbac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bc78a12c80 con 0x55bc78aaf5c0 2024-03-16T09:15:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.415+0000 7fd6fcbac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bc789bbaa0 con 0x55bc78aaebd0 2024-03-16T09:15:29.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.415+0000 7fd6fcbac740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bc78a0ed20 con 0x55bc78ab4730 2024-03-16T09:15:29.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.415+0000 7fd6f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaf5c0 0x55bc78ab40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.415+0000 7fd6f3fff640 1 --2- 172.21.15.112:0/2298042809 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6dc0a2800 0x7fd6dc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.415+0000 7fd6f3fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78aaebd0 0x55bc78aaeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.415+0000 7fd6f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaf5c0 0x55bc78ab40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57710/0 (socket says 172.21.15.112:57710) 2024-03-16T09:15:29.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.415+0000 7fd6f37fe640 1 -- 172.21.15.112:0/1137638531 learned_addr learned my addr 172.21.15.112:0/1137638531 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:29.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.415+0000 7fd6f2ffd640 1 --2- 172.21.15.112:0/1137638531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78ab4730 0x55bc78ab6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.415+0000 7fd6f0ff9640 1 -- 172.21.15.112:0/2298042809 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd6e40f0bb0 con 0x7fd6f41006e0 2024-03-16T09:15:29.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.416+0000 7fd6f3fff640 1 -- 172.21.15.112:0/1137638531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78ab4730 msgr2=0x55bc78ab6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.416+0000 7fd6f3fff640 1 --2- 172.21.15.112:0/1137638531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78ab4730 0x55bc78ab6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.416+0000 7fd6f3fff640 1 -- 172.21.15.112:0/1137638531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaf5c0 msgr2=0x55bc78ab40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.416+0000 7fd6f3fff640 1 --2- 172.21.15.112:0/1137638531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaf5c0 0x55bc78ab40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.416+0000 7fd6f3fff640 1 -- 172.21.15.112:0/1137638531 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc789ddff0 con 0x55bc78aaebd0 2024-03-16T09:15:29.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.416+0000 7fd6f3fff640 1 --2- 172.21.15.112:0/2298042809 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6dc0a2800 0x7fd6dc0a4cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00fe10 tx=0x7fd6ec01e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.416+0000 7fd6f3fff640 1 --2- 172.21.15.112:0/1137638531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78aaebd0 0x55bc78aaeff0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00f510 tx=0x7fd6ec00f910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a964ad997bf275d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.416+0000 7fd6d9ffb640 1 -- 172.21.15.112:0/1137638531 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec02b070 con 0x55bc78aaebd0 2024-03-16T09:15:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.416+0000 7fd6d9ffb640 1 -- 172.21.15.112:0/1137638531 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd6ec010550 con 0x55bc78aaebd0 2024-03-16T09:15:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.416+0000 7fd6d9ffb640 1 -- 172.21.15.112:0/1137638531 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec019040 con 0x55bc78aaebd0 2024-03-16T09:15:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.417+0000 7fd6fcbac740 1 -- 172.21.15.112:0/1137638531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78aaebd0 msgr2=0x55bc78aaeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.417+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/1137638531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78aaebd0 0x55bc78aaeff0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00f510 tx=0x7fd6ec00f910 comp rx=0 tx=0).stop 2024-03-16T09:15:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.417+0000 7fd6fcbac740 1 -- 172.21.15.112:0/1137638531 shutdown_connections 2024-03-16T09:15:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.417+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/1137638531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78ab4730 0x55bc78ab6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.417+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/1137638531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaf5c0 0x55bc78ab40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.417+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/1137638531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78aaebd0 0x55bc78aaeff0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.417+0000 7fd6fcbac740 1 -- 172.21.15.112:0/1137638531 >> 172.21.15.112:0/1137638531 conn(0x55bc78aaa140 msgr2=0x55bc78aaa540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.417+0000 7fd6fcbac740 1 -- 172.21.15.112:0/1137638531 shutdown_connections 2024-03-16T09:15:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.417+0000 7fd6fcbac740 1 -- 172.21.15.112:0/1137638531 wait complete. 2024-03-16T09:15:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.417+0000 7fd6fcbac740 1 -- start start 2024-03-16T09:15:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.417+0000 7fd6fcbac740 1 Processor -- start 2024-03-16T09:15:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.417+0000 7fd6fcbac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaebd0 0x55bc78ab8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.417+0000 7fd6fcbac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78aaf5c0 0x55bc78ab92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.417+0000 7fd6fcbac740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78ab4730 0x55bc78ac02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.417+0000 7fd6fcbac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bc78ab7680 con 0x55bc78aaebd0 2024-03-16T09:15:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.417+0000 7fd6fcbac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bc789ceef0 con 0x55bc78aaf5c0 2024-03-16T09:15:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.417+0000 7fd6fcbac740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bc78ab7800 con 0x55bc78ab4730 2024-03-16T09:15:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.417+0000 7fd6fcbac740 10 client.0 _ll_register_callbacks cb 0x55bc789c9dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:15:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaebd0 0x55bc78ab8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6f2ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78aaf5c0 0x55bc78ab92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaebd0 0x55bc78ab8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57712/0 (socket says 172.21.15.112:57712) 2024-03-16T09:15:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6f2ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78aaf5c0 0x55bc78ab92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:55322/0 (socket says 172.21.15.112:55322) 2024-03-16T09:15:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6f37fe640 1 -- 172.21.15.112:0/2676899902 learned_addr learned my addr 172.21.15.112:0/2676899902 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6f3fff640 1 --2- 172.21.15.112:0/2676899902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78ab4730 0x55bc78ac02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6f37fe640 1 -- 172.21.15.112:0/2676899902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78ab4730 msgr2=0x55bc78ac02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6f37fe640 1 --2- 172.21.15.112:0/2676899902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78ab4730 0x55bc78ac02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6f37fe640 1 -- 172.21.15.112:0/2676899902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78aaf5c0 msgr2=0x55bc78ab92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6f37fe640 1 --2- 172.21.15.112:0/2676899902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78aaf5c0 0x55bc78ab92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6f37fe640 1 -- 172.21.15.112:0/2676899902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc789dba30 con 0x55bc78aaebd0 2024-03-16T09:15:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6f2ffd640 1 --2- 172.21.15.112:0/2676899902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78aaf5c0 0x55bc78ab92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6f37fe640 1 --2- 172.21.15.112:0/2676899902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaebd0 0x55bc78ab8d60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40b9ea0 tx=0x7fd6e408c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800bbd23853025a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:15:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6d2ffd640 1 -- 172.21.15.112:0/2676899902 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e40bcec0 con 0x55bc78aaebd0 2024-03-16T09:15:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6d2ffd640 1 -- 172.21.15.112:0/2676899902 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd6e408c0e0 con 0x55bc78aaebd0 2024-03-16T09:15:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6fcbac740 1 -- 172.21.15.112:0/2676899902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6ec010550 con 0x55bc78aaebd0 2024-03-16T09:15:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6d2ffd640 1 -- 172.21.15.112:0/2676899902 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd6e40bb6a0 con 0x55bc78aaebd0 2024-03-16T09:15:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6fcbac740 10 client.15216 fetch_fsmap learned FSMap version 21 2024-03-16T09:15:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6fcbac740 1 -- 172.21.15.112:0/2676899902 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bc787272d0 con 0x55bc78aaebd0 2024-03-16T09:15:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.418+0000 7fd6d2ffd640 1 -- 172.21.15.112:0/2676899902 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd6ec010550 con 0x55bc78aaebd0 2024-03-16T09:15:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.419+0000 7fd6d2ffd640 1 -- 172.21.15.112:0/2676899902 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd6e40c5870 con 0x55bc78aaebd0 2024-03-16T09:15:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.419+0000 7fd6fcbac740 10 client.15216 fetch_fsmap finished waiting for FSMap version 21 2024-03-16T09:15:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.419+0000 7fd6fcbac740 10 client.15216 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-16T09:15:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.419+0000 7fd6fcbac740 20 client.15216 populate_metadata read hostname 'smithi112' 2024-03-16T09:15:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.419+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/2676899902 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55bc78ac5840 0x55bc78ac6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.419+0000 7fd6fcbac740 4 client.15216 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:15:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.419+0000 7fd6fcbac740 1 -- 172.21.15.112:0/2676899902 --> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc78a201c0 con 0x55bc78ac5840 2024-03-16T09:15:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.419+0000 7fd6f2ffd640 1 --2- 172.21.15.112:0/2676899902 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55bc78ac5840 0x55bc78ac6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.419+0000 7fd6f2ffd640 1 --2- 172.21.15.112:0/2676899902 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55bc78ac5840 0x55bc78ac6cb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9510b5d023e06292 server_cookie=1bb1fe8216b0b749 in_seq=0 out_seq=0 2024-03-16T09:15:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.419+0000 7fd6d2ffd640 10 client.15216 ms_handle_connect on v2:172.21.15.194:6838/2528943309 2024-03-16T09:15:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.420+0000 7fd6d2ffd640 1 -- 172.21.15.112:0/2676899902 <== mds.1 v2:172.21.15.194:6838/2528943309 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bc78a201c0 con 0x55bc78ac5840 2024-03-16T09:15:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.420+0000 7fd6d2ffd640 10 client.15216 handle_command_reply: tid=0 2024-03-16T09:15:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.420+0000 7fd6fcbac740 1 client.15216 shutdown 2024-03-16T09:15:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.420+0000 7fd6fcbac740 1 -- 172.21.15.112:0/2676899902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaebd0 msgr2=0x55bc78ab8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.420+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/2676899902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaebd0 0x55bc78ab8d60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40b9ea0 tx=0x7fd6e408c450 comp rx=0 tx=0).stop 2024-03-16T09:15:29.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.420+0000 7fd6fcbac740 1 -- 172.21.15.112:0/2676899902 shutdown_connections 2024-03-16T09:15:29.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.420+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/2676899902 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55bc78ac5840 0x55bc78ac6cb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.420+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/2676899902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78ab4730 0x55bc78ac02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.420+0000 7fd6d2ffd640 0 client.15216 ms_handle_reset on v2:172.21.15.194:6838/2528943309 2024-03-16T09:15:29.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.420+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/2676899902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78aaf5c0 0x55bc78ab92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.420+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/2676899902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaebd0 0x55bc78ab8d60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.420+0000 7fd6fcbac740 1 -- 172.21.15.112:0/2676899902 >> 172.21.15.112:0/2676899902 conn(0x55bc78aaa140 msgr2=0x55bc78aa9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:29.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.420+0000 7fd6fcbac740 1 -- 172.21.15.112:0/2676899902 shutdown_connections 2024-03-16T09:15:29.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.420+0000 7fd6fcbac740 1 -- 172.21.15.112:0/2676899902 wait complete. 2024-03-16T09:15:29.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.420+0000 7fd6fcbac740 20 client.15216 trim_cache size 0 max 16384 2024-03-16T09:15:29.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6fcbac740 1 Processor -- start 2024-03-16T09:15:29.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6fcbac740 1 -- start start 2024-03-16T09:15:29.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6fcbac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78aaebd0 0x55bc78ac4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6fcbac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaf5c0 0x55bc78ac02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6fcbac740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78ab4730 0x55bc78aaa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6fcbac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bc78ac4ff0 con 0x55bc78aaf5c0 2024-03-16T09:15:29.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6fcbac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bc78ac56c0 con 0x55bc78aaebd0 2024-03-16T09:15:29.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6fcbac740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bc78ac5d90 con 0x55bc78ab4730 2024-03-16T09:15:29.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaf5c0 0x55bc78ac02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaf5c0 0x55bc78ac02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57716/0 (socket says 172.21.15.112:57716) 2024-03-16T09:15:29.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6f2ffd640 1 -- 172.21.15.112:0/983342473 learned_addr learned my addr 172.21.15.112:0/983342473 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:29.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6f37fe640 1 --2- 172.21.15.112:0/983342473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78aaebd0 0x55bc78ac4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6f3fff640 1 --2- 172.21.15.112:0/983342473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78ab4730 0x55bc78aaa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6f2ffd640 1 -- 172.21.15.112:0/983342473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78ab4730 msgr2=0x55bc78aaa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6f2ffd640 1 --2- 172.21.15.112:0/983342473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78ab4730 0x55bc78aaa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6f2ffd640 1 -- 172.21.15.112:0/983342473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78aaebd0 msgr2=0x55bc78ac4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6f2ffd640 1 --2- 172.21.15.112:0/983342473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78aaebd0 0x55bc78ac4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6f2ffd640 1 -- 172.21.15.112:0/983342473 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc787f16b0 con 0x55bc78aaf5c0 2024-03-16T09:15:29.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6f3fff640 1 --2- 172.21.15.112:0/983342473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78ab4730 0x55bc78aaa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:15:29.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6f2ffd640 1 --2- 172.21.15.112:0/983342473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaf5c0 0x55bc78ac02a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55bc78ac73f0 tx=0x7fd6e8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57ccb8cd23fcbb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:29.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.438+0000 7fd6d97fa640 1 -- 172.21.15.112:0/983342473 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8028070 con 0x55bc78aaf5c0 2024-03-16T09:15:29.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.439+0000 7fd6d97fa640 1 -- 172.21.15.112:0/983342473 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd6e800f870 con 0x55bc78aaf5c0 2024-03-16T09:15:29.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.439+0000 7fd6d97fa640 1 -- 172.21.15.112:0/983342473 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8022610 con 0x55bc78aaf5c0 2024-03-16T09:15:29.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.439+0000 7fd6fcbac740 1 -- 172.21.15.112:0/983342473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaf5c0 msgr2=0x55bc78ac02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.439+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/983342473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaf5c0 0x55bc78ac02a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55bc78ac73f0 tx=0x7fd6e8005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:29.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.439+0000 7fd6fcbac740 1 -- 172.21.15.112:0/983342473 shutdown_connections 2024-03-16T09:15:29.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.439+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/983342473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78ab4730 0x55bc78aaa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.439+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/983342473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaf5c0 0x55bc78ac02a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.439+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/983342473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78aaebd0 0x55bc78ac4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.439+0000 7fd6fcbac740 1 -- 172.21.15.112:0/983342473 >> 172.21.15.112:0/983342473 conn(0x55bc78aaa140 msgr2=0x55bc78aad3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:29.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.439+0000 7fd6fcbac740 1 -- 172.21.15.112:0/983342473 shutdown_connections 2024-03-16T09:15:29.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.439+0000 7fd6fcbac740 1 -- 172.21.15.112:0/983342473 wait complete. 2024-03-16T09:15:29.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.439+0000 7fd6fcbac740 1 -- start start 2024-03-16T09:15:29.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6fcbac740 1 Processor -- start 2024-03-16T09:15:29.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6fcbac740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaebd0 0x55bc78abbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6fcbac740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78aaf5c0 0x55bc78abc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6fcbac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78ab4730 0x55bc78abf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6fcbac740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bc78ac6b30 con 0x55bc78aaebd0 2024-03-16T09:15:29.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6fcbac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bc78ac6460 con 0x55bc78ab4730 2024-03-16T09:15:29.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6fcbac740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bc78ac7200 con 0x55bc78aaf5c0 2024-03-16T09:15:29.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6fcbac740 10 client.0 _ll_register_callbacks cb 0x7fd6e40057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:15:29.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaebd0 0x55bc78abbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaebd0 0x55bc78abbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57724/0 (socket says 172.21.15.112:57724) 2024-03-16T09:15:29.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6f37fe640 1 -- 172.21.15.112:0/1825336432 learned_addr learned my addr 172.21.15.112:0/1825336432 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:29.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6f2ffd640 1 --2- 172.21.15.112:0/1825336432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78aaf5c0 0x55bc78abc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6f3fff640 1 --2- 172.21.15.112:0/1825336432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78ab4730 0x55bc78abf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6f37fe640 1 -- 172.21.15.112:0/1825336432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78aaf5c0 msgr2=0x55bc78abc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6f37fe640 1 --2- 172.21.15.112:0/1825336432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78aaf5c0 0x55bc78abc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6f37fe640 1 -- 172.21.15.112:0/1825336432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78ab4730 msgr2=0x55bc78abf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6f37fe640 1 --2- 172.21.15.112:0/1825336432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78ab4730 0x55bc78abf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6f37fe640 1 -- 172.21.15.112:0/1825336432 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc78abfac0 con 0x55bc78aaebd0 2024-03-16T09:15:29.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6f37fe640 1 --2- 172.21.15.112:0/1825336432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaebd0 0x55bc78abbf70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40ee6c0 tx=0x7fd6e40095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7303a0be1a7d57 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:29.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:15:29.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6d2ffd640 1 -- 172.21.15.112:0/1825336432 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e40bd330 con 0x55bc78aaebd0 2024-03-16T09:15:29.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6d2ffd640 1 -- 172.21.15.112:0/1825336432 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd6e40eed80 con 0x55bc78aaebd0 2024-03-16T09:15:29.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.440+0000 7fd6fcbac740 1 -- 172.21.15.112:0/1825336432 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6e800f870 con 0x55bc78aaebd0 2024-03-16T09:15:29.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.441+0000 7fd6d2ffd640 1 -- 172.21.15.112:0/1825336432 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd6e40bdc60 con 0x55bc78aaebd0 2024-03-16T09:15:29.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.441+0000 7fd6fcbac740 10 client.15222 fetch_fsmap learned FSMap version 21 2024-03-16T09:15:29.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.441+0000 7fd6fcbac740 1 -- 172.21.15.112:0/1825336432 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55bc78b2bb70 con 0x55bc78aaebd0 2024-03-16T09:15:29.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.441+0000 7fd6d2ffd640 1 -- 172.21.15.112:0/1825336432 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd6e800f870 con 0x55bc78aaebd0 2024-03-16T09:15:29.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.441+0000 7fd6d2ffd640 1 -- 172.21.15.112:0/1825336432 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fd6e40bc9b0 con 0x55bc78aaebd0 2024-03-16T09:15:29.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.441+0000 7fd6fcbac740 10 client.15222 fetch_fsmap finished waiting for FSMap version 21 2024-03-16T09:15:29.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.441+0000 7fd6fcbac740 10 client.15222 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-16T09:15:29.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.441+0000 7fd6fcbac740 20 client.15222 populate_metadata read hostname 'smithi112' 2024-03-16T09:15:29.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.441+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/1825336432 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55bc78b305b0 0x55bc78b2f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.441+0000 7fd6fcbac740 4 client.15222 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T09:15:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.441+0000 7fd6fcbac740 1 -- 172.21.15.112:0/1825336432 --> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] -- command(tid 0: {"prefix": "status"}) -- 0x55bc78a1a510 con 0x55bc78b305b0 2024-03-16T09:15:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.441+0000 7fd6f2ffd640 1 --2- 172.21.15.112:0/1825336432 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55bc78b305b0 0x55bc78b2f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.441+0000 7fd6d2ffd640 10 client.15222 ms_handle_connect on v2:172.21.15.194:6838/2528943309 2024-03-16T09:15:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.441+0000 7fd6f2ffd640 1 --2- 172.21.15.112:0/1825336432 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55bc78b305b0 0x55bc78b2f2c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10093d6d5d1b1c41 server_cookie=b572fdc7cb1d7b20 in_seq=0 out_seq=0 2024-03-16T09:15:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.442+0000 7fd6d2ffd640 1 -- 172.21.15.112:0/1825336432 <== mds.1 v2:172.21.15.194:6838/2528943309 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bc78a1a510 con 0x55bc78b305b0 2024-03-16T09:15:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.442+0000 7fd6d2ffd640 10 client.15222 handle_command_reply: tid=0 2024-03-16T09:15:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.442+0000 7fd6fcbac740 1 client.15222 shutdown 2024-03-16T09:15:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.442+0000 7fd6fcbac740 1 -- 172.21.15.112:0/1825336432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaebd0 msgr2=0x55bc78abbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.442+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/1825336432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaebd0 0x55bc78abbf70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40ee6c0 tx=0x7fd6e40095d0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.442+0000 7fd6fcbac740 1 -- 172.21.15.112:0/1825336432 shutdown_connections 2024-03-16T09:15:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.442+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/1825336432 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55bc78b305b0 0x55bc78b2f2c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.442+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/1825336432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bc78ab4730 0x55bc78abf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.442+0000 7fd6d2ffd640 0 client.15222 ms_handle_reset on v2:172.21.15.194:6838/2528943309 2024-03-16T09:15:29.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.442+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/1825336432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bc78aaf5c0 0x55bc78abc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.442+0000 7fd6fcbac740 1 --2- 172.21.15.112:0/1825336432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bc78aaebd0 0x55bc78abbf70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.442+0000 7fd6fcbac740 1 -- 172.21.15.112:0/1825336432 >> 172.21.15.112:0/1825336432 conn(0x55bc78aaa140 msgr2=0x55bc78aadc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:29.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.442+0000 7fd6fcbac740 1 -- 172.21.15.112:0/1825336432 shutdown_connections 2024-03-16T09:15:29.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.442+0000 7fd6fcbac740 1 -- 172.21.15.112:0/1825336432 wait complete. 2024-03-16T09:15:29.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.442+0000 7fd6fcbac740 20 client.15222 trim_cache size 0 max 16384 2024-03-16T09:15:29.453 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:15:29.454 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "273be2a4-e374-11ee-95c9-87774f69a715", 2024-03-16T09:15:29.454 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 1, 2024-03-16T09:15:29.454 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24466, 2024-03-16T09:15:29.454 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-03-16T09:15:29.454 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-03-16T09:15:29.454 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-03-16T09:15:29.454 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 110.320247546, 2024-03-16T09:15:29.454 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 21, 2024-03-16T09:15:29.454 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 87, 2024-03-16T09:15:29.454 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 85, 2024-03-16T09:15:29.455 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 126.78682383899999 2024-03-16T09:15:29.455 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:15:29.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.442+0000 7fd6f9fc4640 1 -- 172.21.15.112:0/2298042809 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6dc0a2800 msgr2=0x7fd6dc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.442+0000 7fd6f9fc4640 1 --2- 172.21.15.112:0/2298042809 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6dc0a2800 0x7fd6dc0a4cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00fe10 tx=0x7fd6ec01e000 comp rx=0 tx=0).stop 2024-03-16T09:15:29.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.442+0000 7fd6f9fc4640 1 -- 172.21.15.112:0/2298042809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f41006e0 msgr2=0x7fd6f4197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.442+0000 7fd6f9fc4640 1 --2- 172.21.15.112:0/2298042809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f41006e0 0x7fd6f4197c20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4000c00 tx=0x7fd6e4004a30 comp rx=0 tx=0).stop 2024-03-16T09:15:29.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.443+0000 7fd6f9fc4640 1 -- 172.21.15.112:0/2298042809 shutdown_connections 2024-03-16T09:15:29.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.443+0000 7fd6f9fc4640 1 --2- 172.21.15.112:0/2298042809 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6dc0a2800 0x7fd6dc0a4cc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.443+0000 7fd6f9fc4640 1 --2- 172.21.15.112:0/2298042809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6f4102290 0x7fd6f419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.443+0000 7fd6f9fc4640 1 --2- 172.21.15.112:0/2298042809 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6f41018d0 0x7fd6f4198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.443+0000 7fd6f9fc4640 1 --2- 172.21.15.112:0/2298042809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f41006e0 0x7fd6f4197c20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.443+0000 7fd6f9fc4640 1 -- 172.21.15.112:0/2298042809 >> 172.21.15.112:0/2298042809 conn(0x7fd6f40fbc70 msgr2=0x7fd6f40fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:29.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.443+0000 7fd6f9fc4640 1 -- 172.21.15.112:0/2298042809 shutdown_connections 2024-03-16T09:15:29.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.443+0000 7fd6f9fc4640 1 -- 172.21.15.112:0/2298042809 wait complete. 2024-03-16T09:15:29.458 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-16T09:15:29.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:29 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3218717631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:29.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:29 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1802597449' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:29.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:29 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2881191262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:29.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.558+0000 7ff24907e640 1 -- 172.21.15.112:0/3264084125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2441033a0 msgr2=0x7ff244109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.558+0000 7ff24907e640 1 --2- 172.21.15.112:0/3264084125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2441033a0 0x7ff244109c30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009ca0 tx=0x7ff23c02c0f0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.558+0000 7ff24907e640 1 -- 172.21.15.112:0/3264084125 shutdown_connections 2024-03-16T09:15:29.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.558+0000 7ff24907e640 1 --2- 172.21.15.112:0/3264084125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2441033a0 0x7ff244109c30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.558+0000 7ff24907e640 1 --2- 172.21.15.112:0/3264084125 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2441029e0 0x7ff244102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.558+0000 7ff24907e640 1 --2- 172.21.15.112:0/3264084125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2441017f0 0x7ff244101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.558+0000 7ff24907e640 1 -- 172.21.15.112:0/3264084125 >> 172.21.15.112:0/3264084125 conn(0x7ff2440fcd80 msgr2=0x7ff2440ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:29.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.558+0000 7ff24907e640 1 -- 172.21.15.112:0/3264084125 shutdown_connections 2024-03-16T09:15:29.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.558+0000 7ff24907e640 1 -- 172.21.15.112:0/3264084125 wait complete. 2024-03-16T09:15:29.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.558+0000 7ff24907e640 1 Processor -- start 2024-03-16T09:15:29.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.558+0000 7ff24907e640 1 -- start start 2024-03-16T09:15:29.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff24907e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2441017f0 0x7ff244197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff24907e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2441029e0 0x7ff2441980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff24907e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2441033a0 0x7ff24419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff24907e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff24410d590 con 0x7ff2441033a0 2024-03-16T09:15:29.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff24907e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff24410d410 con 0x7ff2441017f0 2024-03-16T09:15:29.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff24907e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff24410d710 con 0x7ff2441029e0 2024-03-16T09:15:29.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff24359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2441033a0 0x7ff24419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff242d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2441017f0 0x7ff244197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff24359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2441033a0 0x7ff24419f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57738/0 (socket says 172.21.15.112:57738) 2024-03-16T09:15:29.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff24359e640 1 -- 172.21.15.112:0/3549964107 learned_addr learned my addr 172.21.15.112:0/3549964107 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:29.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff24259c640 1 --2- 172.21.15.112:0/3549964107 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2441029e0 0x7ff2441980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff24359e640 1 -- 172.21.15.112:0/3549964107 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2441029e0 msgr2=0x7ff2441980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff24359e640 1 --2- 172.21.15.112:0/3549964107 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2441029e0 0x7ff2441980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff24359e640 1 -- 172.21.15.112:0/3549964107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2441017f0 msgr2=0x7ff244197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff24359e640 1 --2- 172.21.15.112:0/3549964107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2441017f0 0x7ff244197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff24359e640 1 -- 172.21.15.112:0/3549964107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24419f790 con 0x7ff2441033a0 2024-03-16T09:15:29.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff24359e640 1 --2- 172.21.15.112:0/3549964107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2441033a0 0x7ff24419f120 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff23c02c630 tx=0x7ff23c034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:29.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff22bfff640 1 -- 172.21.15.112:0/3549964107 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c039040 con 0x7ff2441033a0 2024-03-16T09:15:29.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff24907e640 1 -- 172.21.15.112:0/3549964107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff24419fa20 con 0x7ff2441033a0 2024-03-16T09:15:29.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff22bfff640 1 -- 172.21.15.112:0/3549964107 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff23c044070 con 0x7ff2441033a0 2024-03-16T09:15:29.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff22bfff640 1 -- 172.21.15.112:0/3549964107 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c004430 con 0x7ff2441033a0 2024-03-16T09:15:29.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.559+0000 7ff24907e640 1 -- 172.21.15.112:0/3549964107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff24419ff10 con 0x7ff2441033a0 2024-03-16T09:15:29.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.561+0000 7ff22bfff640 1 -- 172.21.15.112:0/3549964107 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7ff23c0045d0 con 0x7ff2441033a0 2024-03-16T09:15:29.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff24bc68740 1 Processor -- start 2024-03-16T09:15:29.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff24bc68740 1 -- start start 2024-03-16T09:15:29.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff22bfff640 1 --2- 172.21.15.112:0/3549964107 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff22c0a2800 0x7ff22c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff24bc68740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd63bd0 0x55ae6cd63ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff24bc68740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd645c0 0x55ae6cd690c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff24bc68740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd69730 0x55ae6cd6bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff24bc68740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ae6ccc7c80 con 0x55ae6cd645c0 2024-03-16T09:15:29.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff24bc68740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ae6cc70aa0 con 0x55ae6cd63bd0 2024-03-16T09:15:29.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff24bc68740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ae6ccc3d20 con 0x55ae6cd69730 2024-03-16T09:15:29.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff24259c640 1 --2- 172.21.15.112:0/3549964107 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff22c0a2800 0x7ff22c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff24259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd645c0 0x55ae6cd690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff242d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd63bd0 0x55ae6cd63ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff24359e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd69730 0x55ae6cd6bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff242d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd63bd0 0x55ae6cd63ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:55372/0 (socket says 172.21.15.112:55372) 2024-03-16T09:15:29.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff22bfff640 1 -- 172.21.15.112:0/3549964107 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff23c0f1000 con 0x7ff2441033a0 2024-03-16T09:15:29.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff24259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd645c0 0x55ae6cd690c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57754/0 (socket says 172.21.15.112:57754) 2024-03-16T09:15:29.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff242d9d640 1 -- 172.21.15.112:0/1707994257 learned_addr learned my addr 172.21.15.112:0/1707994257 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:29.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff24359e640 1 -- 172.21.15.112:0/1707994257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd63bd0 msgr2=0x55ae6cd63ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff24359e640 1 --2- 172.21.15.112:0/1707994257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd63bd0 0x55ae6cd63ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff24359e640 1 -- 172.21.15.112:0/1707994257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd645c0 msgr2=0x55ae6cd690c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff24359e640 1 --2- 172.21.15.112:0/1707994257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd645c0 0x55ae6cd690c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.562+0000 7ff24359e640 1 -- 172.21.15.112:0/1707994257 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae6cc92ff0 con 0x55ae6cd69730 2024-03-16T09:15:29.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.563+0000 7ff24259c640 1 --2- 172.21.15.112:0/3549964107 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff22c0a2800 0x7ff22c0a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff2380058c0 tx=0x7ff238005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:29.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.563+0000 7ff24359e640 1 --2- 172.21.15.112:0/1707994257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd69730 0x55ae6cd6bbc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0eeb60 tx=0x7ff23c02c0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2e9687a27f165f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:29.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.563+0000 7ff2297fa640 1 -- 172.21.15.112:0/1707994257 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c0bd4f0 con 0x55ae6cd69730 2024-03-16T09:15:29.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.563+0000 7ff2297fa640 1 -- 172.21.15.112:0/1707994257 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff23c032340 con 0x55ae6cd69730 2024-03-16T09:15:29.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.563+0000 7ff2297fa640 1 -- 172.21.15.112:0/1707994257 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c0957e0 con 0x55ae6cd69730 2024-03-16T09:15:29.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.563+0000 7ff24bc68740 1 -- 172.21.15.112:0/1707994257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd69730 msgr2=0x55ae6cd6bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.563+0000 7ff24bc68740 1 --2- 172.21.15.112:0/1707994257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd69730 0x55ae6cd6bbc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0eeb60 tx=0x7ff23c02c0f0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.563+0000 7ff24bc68740 1 -- 172.21.15.112:0/1707994257 shutdown_connections 2024-03-16T09:15:29.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.563+0000 7ff24bc68740 1 --2- 172.21.15.112:0/1707994257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd69730 0x55ae6cd6bbc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.563+0000 7ff24bc68740 1 --2- 172.21.15.112:0/1707994257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd645c0 0x55ae6cd690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.563+0000 7ff24bc68740 1 --2- 172.21.15.112:0/1707994257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd63bd0 0x55ae6cd63ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.563+0000 7ff24bc68740 1 -- 172.21.15.112:0/1707994257 >> 172.21.15.112:0/1707994257 conn(0x55ae6cd5f140 msgr2=0x55ae6cd5f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:29.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.563+0000 7ff24bc68740 1 -- 172.21.15.112:0/1707994257 shutdown_connections 2024-03-16T09:15:29.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.563+0000 7ff24bc68740 1 -- 172.21.15.112:0/1707994257 wait complete. 2024-03-16T09:15:29.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff24bc68740 1 -- start start 2024-03-16T09:15:29.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff24bc68740 1 Processor -- start 2024-03-16T09:15:29.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff24bc68740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd63bd0 0x55ae6cd6db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff24bc68740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd645c0 0x55ae6cd6e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff24bc68740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd69730 0x55ae6cd750c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff24bc68740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ae6cd6c4a0 con 0x55ae6cd69730 2024-03-16T09:15:29.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff24bc68740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ae6cc83ef0 con 0x55ae6cd63bd0 2024-03-16T09:15:29.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff24bc68740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ae6cd6c620 con 0x55ae6cd645c0 2024-03-16T09:15:29.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff24bc68740 10 client.0 _ll_register_callbacks cb 0x55ae6cc7edc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:15:29.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff24359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd69730 0x55ae6cd750c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff242d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd63bd0 0x55ae6cd6db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff24259c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd645c0 0x55ae6cd6e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff24359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd69730 0x55ae6cd750c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57758/0 (socket says 172.21.15.112:57758) 2024-03-16T09:15:29.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff24359e640 1 -- 172.21.15.112:0/96383790 learned_addr learned my addr 172.21.15.112:0/96383790 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:29.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff24359e640 1 -- 172.21.15.112:0/96383790 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd645c0 msgr2=0x55ae6cd6e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff24359e640 1 --2- 172.21.15.112:0/96383790 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd645c0 0x55ae6cd6e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff24359e640 1 -- 172.21.15.112:0/96383790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd63bd0 msgr2=0x55ae6cd6db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff24359e640 1 --2- 172.21.15.112:0/96383790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd63bd0 0x55ae6cd6db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff24359e640 1 -- 172.21.15.112:0/96383790 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae6cc90a30 con 0x55ae6cd69730 2024-03-16T09:15:29.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.564+0000 7ff24259c640 1 --2- 172.21.15.112:0/96383790 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd645c0 0x55ae6cd6e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:29.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.565+0000 7ff24359e640 1 --2- 172.21.15.112:0/96383790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd69730 0x55ae6cd750c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0b9690 tx=0x7ff23c032970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b759208b9b73fd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:29.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.565+0000 7ff226ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:15:29.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.565+0000 7ff226ff5640 1 -- 172.21.15.112:0/96383790 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c0bd310 con 0x55ae6cd69730 2024-03-16T09:15:29.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.565+0000 7ff226ff5640 1 -- 172.21.15.112:0/96383790 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff23c0bde10 con 0x55ae6cd69730 2024-03-16T09:15:29.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.565+0000 7ff24bc68740 1 -- 172.21.15.112:0/96383790 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ae6ccd51c0 con 0x55ae6cd69730 2024-03-16T09:15:29.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.565+0000 7ff226ff5640 1 -- 172.21.15.112:0/96383790 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff23c0be0b0 con 0x55ae6cd69730 2024-03-16T09:15:29.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.565+0000 7ff24bc68740 10 client.15234 fetch_fsmap learned FSMap version 21 2024-03-16T09:15:29.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.565+0000 7ff24bc68740 1 -- 172.21.15.112:0/96383790 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ae6c9dc2d0 con 0x55ae6cd69730 2024-03-16T09:15:29.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.565+0000 7ff226ff5640 1 -- 172.21.15.112:0/96383790 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x55ae6ccd51c0 con 0x55ae6cd69730 2024-03-16T09:15:29.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.565+0000 7ff226ff5640 1 -- 172.21.15.112:0/96383790 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff23c0bece0 con 0x55ae6cd69730 2024-03-16T09:15:29.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.565+0000 7ff24bc68740 10 client.15234 fetch_fsmap finished waiting for FSMap version 21 2024-03-16T09:15:29.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.565+0000 7ff24bc68740 10 client.15234 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T09:15:29.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.565+0000 7ff24bc68740 20 client.15234 populate_metadata read hostname 'smithi112' 2024-03-16T09:15:29.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.565+0000 7ff24bc68740 1 --2- 172.21.15.112:0/96383790 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55ae6cd7a2c0 0x55ae6cd7b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.565+0000 7ff24bc68740 4 client.15234 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:15:29.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.565+0000 7ff24bc68740 1 -- 172.21.15.112:0/96383790 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae6cccf510 con 0x55ae6cd7a2c0 2024-03-16T09:15:29.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.565+0000 7ff242d9d640 1 --2- 172.21.15.112:0/96383790 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55ae6cd7a2c0 0x55ae6cd7b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.566+0000 7ff226ff5640 10 client.15234 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:29.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.566+0000 7ff242d9d640 1 --2- 172.21.15.112:0/96383790 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55ae6cd7a2c0 0x55ae6cd7b730 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33cc19f66a495d2f server_cookie=559e6f98d2eb7c3 in_seq=0 out_seq=0 2024-03-16T09:15:29.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.566+0000 7ff226ff5640 1 -- 172.21.15.112:0/96383790 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ae6cccf510 con 0x55ae6cd7a2c0 2024-03-16T09:15:29.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.566+0000 7ff226ff5640 10 client.15234 handle_command_reply: tid=0 2024-03-16T09:15:29.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.566+0000 7ff24bc68740 1 client.15234 shutdown 2024-03-16T09:15:29.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.566+0000 7ff24bc68740 1 -- 172.21.15.112:0/96383790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd69730 msgr2=0x55ae6cd750c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.566+0000 7ff24bc68740 1 --2- 172.21.15.112:0/96383790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd69730 0x55ae6cd750c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0b9690 tx=0x7ff23c032970 comp rx=0 tx=0).stop 2024-03-16T09:15:29.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.566+0000 7ff24bc68740 1 -- 172.21.15.112:0/96383790 shutdown_connections 2024-03-16T09:15:29.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.566+0000 7ff24bc68740 1 --2- 172.21.15.112:0/96383790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd69730 0x55ae6cd750c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.566+0000 7ff24bc68740 1 --2- 172.21.15.112:0/96383790 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd645c0 0x55ae6cd6e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.566+0000 7ff24bc68740 1 --2- 172.21.15.112:0/96383790 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55ae6cd7a2c0 0x55ae6cd7b730 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.566+0000 7ff24bc68740 1 --2- 172.21.15.112:0/96383790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd63bd0 0x55ae6cd6db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.567+0000 7ff226ff5640 0 client.15234 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:29.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.567+0000 7ff24bc68740 1 -- 172.21.15.112:0/96383790 >> 172.21.15.112:0/96383790 conn(0x55ae6cd5f140 msgr2=0x55ae6cd5e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:29.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.567+0000 7ff24bc68740 1 -- 172.21.15.112:0/96383790 shutdown_connections 2024-03-16T09:15:29.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.567+0000 7ff24bc68740 1 -- 172.21.15.112:0/96383790 wait complete. 2024-03-16T09:15:29.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.567+0000 7ff24bc68740 20 client.15234 trim_cache size 0 max 16384 2024-03-16T09:15:29.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.584+0000 7ff24bc68740 1 Processor -- start 2024-03-16T09:15:29.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.584+0000 7ff24bc68740 1 -- start start 2024-03-16T09:15:29.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.584+0000 7ff24bc68740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd5f140 0x55ae6cd62ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.584+0000 7ff24bc68740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd63bd0 0x55ae6cd75080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.584+0000 7ff24bc68740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd645c0 0x55ae6cd65060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.584+0000 7ff24bc68740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ae6cd79a70 con 0x55ae6cd5f140 2024-03-16T09:15:29.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.584+0000 7ff24bc68740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ae6cd7a140 con 0x55ae6cd645c0 2024-03-16T09:15:29.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.584+0000 7ff24bc68740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ae6cd7a810 con 0x55ae6cd63bd0 2024-03-16T09:15:29.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.584+0000 7ff242d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd5f140 0x55ae6cd62ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.584+0000 7ff242d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd5f140 0x55ae6cd62ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57764/0 (socket says 172.21.15.112:57764) 2024-03-16T09:15:29.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.584+0000 7ff242d9d640 1 -- 172.21.15.112:0/4121669259 learned_addr learned my addr 172.21.15.112:0/4121669259 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:29.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.584+0000 7ff24259c640 1 --2- 172.21.15.112:0/4121669259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd63bd0 0x55ae6cd75080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.584+0000 7ff24359e640 1 --2- 172.21.15.112:0/4121669259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd645c0 0x55ae6cd65060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.584+0000 7ff242d9d640 1 -- 172.21.15.112:0/4121669259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd63bd0 msgr2=0x55ae6cd75080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.584+0000 7ff242d9d640 1 --2- 172.21.15.112:0/4121669259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd63bd0 0x55ae6cd75080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.584+0000 7ff242d9d640 1 -- 172.21.15.112:0/4121669259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd645c0 msgr2=0x55ae6cd65060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.584+0000 7ff242d9d640 1 --2- 172.21.15.112:0/4121669259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd645c0 0x55ae6cd65060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.584+0000 7ff242d9d640 1 -- 172.21.15.112:0/4121669259 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae6caa66b0 con 0x55ae6cd5f140 2024-03-16T09:15:29.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.584+0000 7ff24259c640 1 --2- 172.21.15.112:0/4121669259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd63bd0 0x55ae6cd75080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:15:29.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.585+0000 7ff242d9d640 1 --2- 172.21.15.112:0/4121669259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd5f140 0x55ae6cd62ab0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55ae6cd7be70 tx=0x7ff234005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a424afea3b0087 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:29.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.585+0000 7ff228ff9640 1 -- 172.21.15.112:0/4121669259 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234024020 con 0x55ae6cd5f140 2024-03-16T09:15:29.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.585+0000 7ff228ff9640 1 -- 172.21.15.112:0/4121669259 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff2340206b0 con 0x55ae6cd5f140 2024-03-16T09:15:29.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.585+0000 7ff228ff9640 1 -- 172.21.15.112:0/4121669259 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234003ca0 con 0x55ae6cd5f140 2024-03-16T09:15:29.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.585+0000 7ff24bc68740 1 -- 172.21.15.112:0/4121669259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd5f140 msgr2=0x55ae6cd62ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.585+0000 7ff24bc68740 1 --2- 172.21.15.112:0/4121669259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd5f140 0x55ae6cd62ab0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55ae6cd7be70 tx=0x7ff234005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:29.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.585+0000 7ff24bc68740 1 -- 172.21.15.112:0/4121669259 shutdown_connections 2024-03-16T09:15:29.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.585+0000 7ff24bc68740 1 --2- 172.21.15.112:0/4121669259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd645c0 0x55ae6cd65060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.585+0000 7ff24bc68740 1 --2- 172.21.15.112:0/4121669259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd63bd0 0x55ae6cd75080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.585+0000 7ff24bc68740 1 --2- 172.21.15.112:0/4121669259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd5f140 0x55ae6cd62ab0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.585+0000 7ff24bc68740 1 -- 172.21.15.112:0/4121669259 >> 172.21.15.112:0/4121669259 conn(0x55ae6cd79590 msgr2=0x55ae6cd623a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:29.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.585+0000 7ff24bc68740 1 -- 172.21.15.112:0/4121669259 shutdown_connections 2024-03-16T09:15:29.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.585+0000 7ff24bc68740 1 -- 172.21.15.112:0/4121669259 wait complete. 2024-03-16T09:15:29.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.585+0000 7ff24bc68740 1 -- start start 2024-03-16T09:15:29.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff24bc68740 1 Processor -- start 2024-03-16T09:15:29.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff24bc68740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd5f140 0x55ae6cd73070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff24bc68740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd63bd0 0x55ae6cddc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff24bc68740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd645c0 0x55ae6cdde560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff24bc68740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ae6cd7b5b0 con 0x55ae6cd63bd0 2024-03-16T09:15:29.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff24bc68740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ae6cd7aee0 con 0x55ae6cd5f140 2024-03-16T09:15:29.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff24bc68740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ae6cd7bc80 con 0x55ae6cd645c0 2024-03-16T09:15:29.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff24bc68740 10 client.0 _ll_register_callbacks cb 0x7ff2340097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:15:29.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff24259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd63bd0 0x55ae6cddc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff24259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd63bd0 0x55ae6cddc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57772/0 (socket says 172.21.15.112:57772) 2024-03-16T09:15:29.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff24259c640 1 -- 172.21.15.112:0/2445600766 learned_addr learned my addr 172.21.15.112:0/2445600766 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:29.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff242d9d640 1 --2- 172.21.15.112:0/2445600766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd5f140 0x55ae6cd73070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff24359e640 1 --2- 172.21.15.112:0/2445600766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd645c0 0x55ae6cdde560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff24259c640 1 -- 172.21.15.112:0/2445600766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd645c0 msgr2=0x55ae6cdde560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff24259c640 1 --2- 172.21.15.112:0/2445600766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd645c0 0x55ae6cdde560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff24259c640 1 -- 172.21.15.112:0/2445600766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd5f140 msgr2=0x55ae6cd73070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:29.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff24259c640 1 --2- 172.21.15.112:0/2445600766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd5f140 0x55ae6cd73070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff24259c640 1 -- 172.21.15.112:0/2445600766 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae6cdded70 con 0x55ae6cd63bd0 2024-03-16T09:15:29.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff24259c640 1 --2- 172.21.15.112:0/2445600766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd63bd0 0x55ae6cddc010 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55ae6cd65030 tx=0x7ff238005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e443ff98c776a7dc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:29.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff226ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:15:29.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff226ff5640 1 -- 172.21.15.112:0/2445600766 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23802f060 con 0x55ae6cd63bd0 2024-03-16T09:15:29.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff226ff5640 1 -- 172.21.15.112:0/2445600766 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff238003bd0 con 0x55ae6cd63bd0 2024-03-16T09:15:29.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.586+0000 7ff24bc68740 1 -- 172.21.15.112:0/2445600766 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2340206b0 con 0x55ae6cd63bd0 2024-03-16T09:15:29.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.587+0000 7ff226ff5640 1 -- 172.21.15.112:0/2445600766 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff238019040 con 0x55ae6cd63bd0 2024-03-16T09:15:29.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.587+0000 7ff24bc68740 10 client.15240 fetch_fsmap learned FSMap version 21 2024-03-16T09:15:29.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.587+0000 7ff24bc68740 1 -- 172.21.15.112:0/2445600766 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55ae6cde00e0 con 0x55ae6cd63bd0 2024-03-16T09:15:29.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.587+0000 7ff226ff5640 1 -- 172.21.15.112:0/2445600766 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff2340206b0 con 0x55ae6cd63bd0 2024-03-16T09:15:29.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.587+0000 7ff226ff5640 1 -- 172.21.15.112:0/2445600766 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff23802a450 con 0x55ae6cd63bd0 2024-03-16T09:15:29.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.587+0000 7ff24bc68740 10 client.15240 fetch_fsmap finished waiting for FSMap version 21 2024-03-16T09:15:29.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.587+0000 7ff24bc68740 10 client.15240 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T09:15:29.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.587+0000 7ff24bc68740 20 client.15240 populate_metadata read hostname 'smithi112' 2024-03-16T09:15:29.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.587+0000 7ff24bc68740 1 --2- 172.21.15.112:0/2445600766 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55ae6cd69730 0x55ae6cde3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:29.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.587+0000 7ff24bc68740 4 client.15240 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T09:15:29.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.587+0000 7ff24bc68740 1 -- 172.21.15.112:0/2445600766 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "status"}) -- 0x55ae6cc4da00 con 0x55ae6cd69730 2024-03-16T09:15:29.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.587+0000 7ff242d9d640 1 --2- 172.21.15.112:0/2445600766 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55ae6cd69730 0x55ae6cde3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:29.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.587+0000 7ff242d9d640 1 --2- 172.21.15.112:0/2445600766 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55ae6cd69730 0x55ae6cde3a40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d32da51066ac9160 server_cookie=9125dce2d4fc630c in_seq=0 out_seq=0 2024-03-16T09:15:29.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.587+0000 7ff226ff5640 10 client.15240 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:29.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff226ff5640 1 -- 172.21.15.112:0/2445600766 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55ae6cc4da00 con 0x55ae6cd69730 2024-03-16T09:15:29.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff226ff5640 10 client.15240 handle_command_reply: tid=0 2024-03-16T09:15:29.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24bc68740 1 client.15240 shutdown 2024-03-16T09:15:29.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24bc68740 1 -- 172.21.15.112:0/2445600766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd63bd0 msgr2=0x55ae6cddc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24bc68740 1 --2- 172.21.15.112:0/2445600766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd63bd0 0x55ae6cddc010 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55ae6cd65030 tx=0x7ff238005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:29.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24bc68740 1 -- 172.21.15.112:0/2445600766 shutdown_connections 2024-03-16T09:15:29.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24bc68740 1 --2- 172.21.15.112:0/2445600766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ae6cd645c0 0x55ae6cdde560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24bc68740 1 --2- 172.21.15.112:0/2445600766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ae6cd63bd0 0x55ae6cddc010 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24bc68740 1 --2- 172.21.15.112:0/2445600766 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55ae6cd69730 0x55ae6cde3a40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24bc68740 1 --2- 172.21.15.112:0/2445600766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae6cd5f140 0x55ae6cd73070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24bc68740 1 -- 172.21.15.112:0/2445600766 >> 172.21.15.112:0/2445600766 conn(0x55ae6cd79590 msgr2=0x55ae6cd62c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:29.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff226ff5640 0 client.15240 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:29.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24bc68740 1 -- 172.21.15.112:0/2445600766 shutdown_connections 2024-03-16T09:15:29.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24bc68740 1 -- 172.21.15.112:0/2445600766 wait complete. 2024-03-16T09:15:29.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24bc68740 20 client.15240 trim_cache size 0 max 16384 2024-03-16T09:15:29.599 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:15:29.600 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "273be2a4-e374-11ee-95c9-87774f69a715", 2024-03-16T09:15:29.600 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-03-16T09:15:29.600 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24494, 2024-03-16T09:15:29.600 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-03-16T09:15:29.600 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-03-16T09:15:29.600 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-03-16T09:15:29.600 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 112.47793256, 2024-03-16T09:15:29.600 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 21, 2024-03-16T09:15:29.600 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 87, 2024-03-16T09:15:29.600 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 85, 2024-03-16T09:15:29.600 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 118.150896782 2024-03-16T09:15:29.600 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:15:29.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24907e640 1 -- 172.21.15.112:0/3549964107 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff22c0a2800 msgr2=0x7ff22c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24907e640 1 --2- 172.21.15.112:0/3549964107 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff22c0a2800 0x7ff22c0a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff2380058c0 tx=0x7ff238005850 comp rx=0 tx=0).stop 2024-03-16T09:15:29.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24907e640 1 -- 172.21.15.112:0/3549964107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2441033a0 msgr2=0x7ff24419f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:29.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24907e640 1 --2- 172.21.15.112:0/3549964107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2441033a0 0x7ff24419f120 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff23c02c630 tx=0x7ff23c034670 comp rx=0 tx=0).stop 2024-03-16T09:15:29.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24907e640 1 -- 172.21.15.112:0/3549964107 shutdown_connections 2024-03-16T09:15:29.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24907e640 1 --2- 172.21.15.112:0/3549964107 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff22c0a2800 0x7ff22c0a4cc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24907e640 1 --2- 172.21.15.112:0/3549964107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2441033a0 0x7ff24419f120 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24907e640 1 --2- 172.21.15.112:0/3549964107 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2441029e0 0x7ff2441980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24907e640 1 --2- 172.21.15.112:0/3549964107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2441017f0 0x7ff244197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:29.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.588+0000 7ff24907e640 1 -- 172.21.15.112:0/3549964107 >> 172.21.15.112:0/3549964107 conn(0x7ff2440fcd80 msgr2=0x7ff244105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:29.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.589+0000 7ff24907e640 1 -- 172.21.15.112:0/3549964107 shutdown_connections 2024-03-16T09:15:29.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:29.589+0000 7ff24907e640 1 -- 172.21.15.112:0/3549964107 wait complete. 2024-03-16T09:15:29.603 INFO:teuthology.run_tasks:Running task kclient... 2024-03-16T09:15:29.628 INFO:tasks.kclient:Mounting kernel clients... 2024-03-16T09:15:29.628 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-16T09:15:29.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:15:29.629 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-03-16T09:15:29.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:15:29.645 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-03-16T09:15:29.715 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2024-03-16T09:15:29.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:15:29.716 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'rbytes': False, 'syntax': 'v1', 'mount_subvol_num': 1} 2024-03-16T09:15:29.716 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-16T09:15:29.717 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-16T09:15:29.717 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-16T09:15:29.995 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:15:30.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:30 smithi112 ceph-mon[32307]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T09:15:30.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:30 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/341840226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:30.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:30 smithi194 ceph-mon[36729]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T09:15:30.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:30 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/341840226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:30.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:30 smithi159 ceph-mon[36739]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-16T09:15:30.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:30 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/341840226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:31.449 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:31 smithi112 ceph-mon[32307]: pgmap v347: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T09:15:31.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:31 smithi194 ceph-mon[36729]: pgmap v347: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T09:15:31.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:31 smithi159 ceph-mon[36739]: pgmap v347: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-16T09:15:31.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.852+0000 7f720a793700 1 -- 172.21.15.112:0/270171096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7204101c30 msgr2=0x7f7204102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:31.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.852+0000 7f720a793700 1 --2- 172.21.15.112:0/270171096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7204101c30 0x7f7204102010 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f71f40099c0 tx=0x7f71f4009cd0 comp rx=0 tx=0).stop 2024-03-16T09:15:31.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.852+0000 7f720a793700 1 -- 172.21.15.112:0/270171096 shutdown_connections 2024-03-16T09:15:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.852+0000 7f720a793700 1 --2- 172.21.15.112:0/270171096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7204106550 0x7f720410e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.852+0000 7f720a793700 1 --2- 172.21.15.112:0/270171096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7204101c30 0x7f7204102010 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.852+0000 7f720a793700 1 --2- 172.21.15.112:0/270171096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72041025e0 0x7f7204105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.852+0000 7f720a793700 1 -- 172.21.15.112:0/270171096 >> 172.21.15.112:0/270171096 conn(0x7f72040777d0 msgr2=0x7f72040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.852+0000 7f720a793700 1 -- 172.21.15.112:0/270171096 shutdown_connections 2024-03-16T09:15:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.852+0000 7f720a793700 1 -- 172.21.15.112:0/270171096 wait complete. 2024-03-16T09:15:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.852+0000 7f720a793700 1 Processor -- start 2024-03-16T09:15:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.852+0000 7f720a793700 1 -- start start 2024-03-16T09:15:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f720a793700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7204101c30 0x7f720419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f720a793700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72041025e0 0x7f720419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f720a793700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7204106550 0x7f7204198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f720a793700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7204111640 con 0x7f7204106550 2024-03-16T09:15:31.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f720a793700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f72041114c0 con 0x7f72041025e0 2024-03-16T09:15:31.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f720a793700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f72041117c0 con 0x7f7204101c30 2024-03-16T09:15:31.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f7208d30700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7204106550 0x7f7204198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:31.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f7208d30700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7204106550 0x7f7204198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56998/0 (socket says 172.21.15.112:56998) 2024-03-16T09:15:31.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f7208d30700 1 -- 172.21.15.112:0/1041125922 learned_addr learned my addr 172.21.15.112:0/1041125922 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:31.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f7203fff700 1 --2- 172.21.15.112:0/1041125922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7204101c30 0x7f720419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:31.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f72037fe700 1 --2- 172.21.15.112:0/1041125922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72041025e0 0x7f720419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:31.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f7208d30700 1 -- 172.21.15.112:0/1041125922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7204101c30 msgr2=0x7f720419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:31.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f7208d30700 1 --2- 172.21.15.112:0/1041125922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7204101c30 0x7f720419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:31.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f7208d30700 1 -- 172.21.15.112:0/1041125922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72041025e0 msgr2=0x7f720419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:31.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f7208d30700 1 --2- 172.21.15.112:0/1041125922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72041025e0 0x7f720419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:31.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f7208d30700 1 -- 172.21.15.112:0/1041125922 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72041991f0 con 0x7f7204106550 2024-03-16T09:15:31.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f7208d30700 1 --2- 172.21.15.112:0/1041125922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7204106550 0x7f7204198a90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f71fc007ee0 tx=0x7f71fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:31.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f72017fa700 1 -- 172.21.15.112:0/1041125922 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc00dd20 con 0x7f7204106550 2024-03-16T09:15:31.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f72017fa700 1 -- 172.21.15.112:0/1041125922 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f71fc004020 con 0x7f7204106550 2024-03-16T09:15:31.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f720a793700 1 -- 172.21.15.112:0/1041125922 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72041994e0 con 0x7f7204106550 2024-03-16T09:15:31.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f72017fa700 1 -- 172.21.15.112:0/1041125922 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc015660 con 0x7f7204106550 2024-03-16T09:15:31.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.853+0000 7f720a793700 1 -- 172.21.15.112:0/1041125922 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72041a57a0 con 0x7f7204106550 2024-03-16T09:15:31.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.854+0000 7f720a793700 1 -- 172.21.15.112:0/1041125922 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7204199fc0 con 0x7f7204106550 2024-03-16T09:15:31.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.856+0000 7f72017fa700 1 -- 172.21.15.112:0/1041125922 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f71fc0041d0 con 0x7f7204106550 2024-03-16T09:15:31.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.856+0000 7f72017fa700 1 --2- 172.21.15.112:0/1041125922 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f71ec0a2890 0x7f71ec0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:31.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.856+0000 7f7203fff700 1 --2- 172.21.15.112:0/1041125922 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f71ec0a2890 0x7f71ec0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:31.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.857+0000 7f72017fa700 1 -- 172.21.15.112:0/1041125922 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f71fc0ceba0 con 0x7f7204106550 2024-03-16T09:15:31.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.857+0000 7f7203fff700 1 --2- 172.21.15.112:0/1041125922 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f71ec0a2890 0x7f71ec0a4d50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f71f4000c00 tx=0x7f71f4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:31.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:31.860+0000 7f72017fa700 1 -- 172.21.15.112:0/1041125922 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f71fc010070 con 0x7f7204106550 2024-03-16T09:15:32.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:32.106+0000 7f720a793700 1 -- 172.21.15.112:0/1041125922 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f720402db70 con 0x7f7204106550 2024-03-16T09:15:32.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:32.106+0000 7f72017fa700 1 -- 172.21.15.112:0/1041125922 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) ==== 53+0+73 (secure 0 0 0) 0x7f71fc09bc50 con 0x7f7204106550 2024-03-16T09:15:32.108 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-16T09:15:32.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:32.108+0000 7f720a793700 1 -- 172.21.15.112:0/1041125922 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f71ec0a2890 msgr2=0x7f71ec0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:32.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:32.108+0000 7f720a793700 1 --2- 172.21.15.112:0/1041125922 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f71ec0a2890 0x7f71ec0a4d50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f71f4000c00 tx=0x7f71f4005fb0 comp rx=0 tx=0).stop 2024-03-16T09:15:32.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:32.108+0000 7f720a793700 1 -- 172.21.15.112:0/1041125922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7204106550 msgr2=0x7f7204198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:32.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:32.108+0000 7f720a793700 1 --2- 172.21.15.112:0/1041125922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7204106550 0x7f7204198a90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f71fc007ee0 tx=0x7f71fc00d3b0 comp rx=0 tx=0).stop 2024-03-16T09:15:32.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:32.109+0000 7f720a793700 1 -- 172.21.15.112:0/1041125922 shutdown_connections 2024-03-16T09:15:32.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:32.109+0000 7f720a793700 1 --2- 172.21.15.112:0/1041125922 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f71ec0a2890 0x7f71ec0a4d50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:32.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:32.109+0000 7f720a793700 1 --2- 172.21.15.112:0/1041125922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7204106550 0x7f7204198a90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:32.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:32.109+0000 7f720a793700 1 --2- 172.21.15.112:0/1041125922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7204101c30 0x7f720419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:32.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:32.109+0000 7f720a793700 1 --2- 172.21.15.112:0/1041125922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72041025e0 0x7f720419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:32.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:32.109+0000 7f720a793700 1 -- 172.21.15.112:0/1041125922 >> 172.21.15.112:0/1041125922 conn(0x7f72040777d0 msgr2=0x7f7204100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:32.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:32.109+0000 7f720a793700 1 -- 172.21.15.112:0/1041125922 shutdown_connections 2024-03-16T09:15:32.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:32.109+0000 7f720a793700 1 -- 172.21.15.112:0/1041125922 wait complete. 2024-03-16T09:15:32.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:32 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1041125922' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T09:15:32.723 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:32 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1041125922' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T09:15:32.736 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:32 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1041125922' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-16T09:15:32.929 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-16T09:15:32.930 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-16T09:15:32.930 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-03-16T09:15:32.930 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-16T09:15:32.930 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-16T09:15:32.930 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-16T09:15:32.930 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-16T09:15:32.930 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-16T09:15:32.930 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-16T09:15:32.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:15:32.930 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-03-16T09:15:32.955 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T09:15:32.955 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T09:15:32.955 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T09:15:32.955 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-03-16T09:15:32.955 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-03-16T09:15:32.955 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-03-16T09:15:32.955 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T09:15:32.956 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-03-16T09:15:32.956 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-03-16T09:15:32.956 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T09:15:32.956 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-03-16T09:15:32.956 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-03-16T09:15:32.956 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T09:15:32.956 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-03-16T09:15:32.956 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-16T09:15:32.956 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-03-16T09:15:32.956 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-16T09:15:32.956 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42084sec preferred_lft 42084sec 2024-03-16T09:15:32.956 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link 2024-03-16T09:15:32.956 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-03-16T09:15:32.957 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-16T09:15:32.957 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T09:15:32.957 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-03-16T09:15:32.957 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add name ceph-brx type bridge 2024-03-16T09:15:32.957 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr flush dev ceph-brx 2024-03-16T09:15:32.958 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set ceph-brx up 2024-03-16T09:15:32.958 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-16T09:15:32.958 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-03-16T09:15:33.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T09:15:33.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T09:15:33.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:15:33.166 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-16T09:15:33.197 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-03-16T09:15:33.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:15:33.198 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-03-16T09:15:33.277 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-03-16T09:15:33.277 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-16T09:15:33.277 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-16T09:15:33.277 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-16T09:15:33.277 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-16T09:15:33.277 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T09:15:33.278 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-03-16T09:15:33.278 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-16T09:15:33.278 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-16T09:15:33.278 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-16T09:15:33.278 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-03-16T09:15:33.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T09:15:33.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T09:15:33.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:15:33.584 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-03-16T09:15:33.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:15:33.600 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2024-03-16T09:15:33.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:33 smithi194 ceph-mon[36729]: pgmap v348: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:15:33.655 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T09:15:33.655 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-03-16T09:15:33.655 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T09:15:33.655 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-16T09:15:33.655 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-03-16T09:15:33.738 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:33 smithi159 ceph-mon[36739]: pgmap v348: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:15:33.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T09:15:33.753 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:33 smithi112 ceph-mon[32307]: pgmap v348: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:15:33.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T09:15:33.777 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-16T09:15:33.777 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T09:15:33.777 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-03-16T09:15:33.777 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-16T09:15:33.777 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-16T09:15:33.777 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-16T09:15:33.778 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-16T09:15:33.778 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-16T09:15:33.778 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-03-16T09:15:33.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T09:15:33.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T09:15:33.923 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T09:15:33.923 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-03-16T09:15:33.923 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-03-16T09:15:33.923 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.0 master ceph-brx 2024-03-16T09:15:33.924 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-03-16T09:15:33.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T09:15:33.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T09:15:34.005 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T09:15:34.005 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-16T09:15:34.019 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-16T09:15:34.019 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T09:15:34.020 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-16T09:15:34.074 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-16T09:15:34.074 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-16T09:15:34.074 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff 2024-03-16T09:15:34.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:15:34.075 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-16T09:15:34.160 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-16T09:15:34.160 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-03-16T09:15:34.161 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-03-16T09:15:34.275 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-16T09:15:34.275 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-03-16T09:15:34.275 INFO:teuthology.orchestra.run.smithi112.stdout:invalid new device string format 2024-03-16T09:15:34.275 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:6789,172.21.15.159:6789,172.21.15.194:6789" 2024-03-16T09:15:34.276 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.112:6789,172.21.15.159:6789,172.21.15.194:6789:/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff 2024-03-16T09:15:34.276 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=273be2a4-e374-11ee-95c9-87774f69a715" will pass to kernel 2024-03-16T09:15:34.276 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-16T09:15:34.276 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-03-16T09:15:34.276 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-16T09:15:34.276 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-03-16T09:15:34.277 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-16T09:15:34.278 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-16T09:15:34.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:15:34.278 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-16T09:15:34.307 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T09:15:34.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-16T09:15:34.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T09:15:34.406 INFO:teuthology.orchestra.run.smithi112.stdout:client25111 2024-03-16T09:15:34.406 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T09:15:34.406 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T09:15:34.426 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T09:15:34.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-16T09:15:34.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T09:15:34.523 INFO:teuthology.orchestra.run.smithi112.stdout:client25111 2024-03-16T09:15:34.523 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T09:15:34.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-16T09:15:34.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T09:15:34.579 INFO:teuthology.orchestra.run.smithi112.stdout:client25111 2024-03-16T09:15:34.579 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/273be2a4-e374-11ee-95c9-87774f69a715.client25111/status') 2024-03-16T09:15:34.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T09:15:34.640 INFO:teuthology.orchestra.run.smithi112.stdout:instance: client.25111 (3)172.21.15.112:0/776038281 2024-03-16T09:15:34.640 INFO:teuthology.orchestra.run.smithi112.stdout:blocklisted: false 2024-03-16T09:15:34.640 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records in 2024-03-16T09:15:34.640 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records out 2024-03-16T09:15:34.640 INFO:teuthology.orchestra.run.smithi112.stderr:71 bytes copied, 2.1205e-05 s, 3.3 MB/s 2024-03-16T09:15:34.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T09:15:34.647 INFO:teuthology.run_tasks:Running task exec... 2024-03-16T09:15:34.657 INFO:teuthology.task.exec:Executing custom commands... 2024-03-16T09:15:34.657 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-03-16T09:15:34.657 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-16T09:15:34.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.759+0000 7ff8a32e8640 1 -- 172.21.15.112:0/1052217510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff89c1022b0 msgr2=0x7ff89c108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:34.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.759+0000 7ff8a32e8640 1 --2- 172.21.15.112:0/1052217510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff89c1022b0 0x7ff89c108b40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff898009900 tx=0x7ff89802bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:34.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.759+0000 7ff8a32e8640 1 -- 172.21.15.112:0/1052217510 shutdown_connections 2024-03-16T09:15:34.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.759+0000 7ff8a32e8640 1 --2- 172.21.15.112:0/1052217510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff89c1022b0 0x7ff89c108b40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:34.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.759+0000 7ff8a32e8640 1 --2- 172.21.15.112:0/1052217510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff89c1018f0 0x7ff89c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:34.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.759+0000 7ff8a32e8640 1 --2- 172.21.15.112:0/1052217510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff89c100700 0x7ff89c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:34.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.759+0000 7ff8a32e8640 1 -- 172.21.15.112:0/1052217510 >> 172.21.15.112:0/1052217510 conn(0x7ff89c0fbc70 msgr2=0x7ff89c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:34.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.759+0000 7ff8a32e8640 1 -- 172.21.15.112:0/1052217510 shutdown_connections 2024-03-16T09:15:34.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.759+0000 7ff8a32e8640 1 -- 172.21.15.112:0/1052217510 wait complete. 2024-03-16T09:15:34.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.760+0000 7ff8a32e8640 1 Processor -- start 2024-03-16T09:15:34.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.760+0000 7ff8a32e8640 1 -- start start 2024-03-16T09:15:34.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.760+0000 7ff8a32e8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff89c100700 0x7ff89c1969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:34.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.760+0000 7ff8a32e8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff89c1018f0 0x7ff89c196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:34.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.760+0000 7ff8a32e8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff89c1022b0 0x7ff89c19df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:34.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.760+0000 7ff8a32e8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c10c460 con 0x7ff89c100700 2024-03-16T09:15:34.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.760+0000 7ff8a32e8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c10c2e0 con 0x7ff89c1022b0 2024-03-16T09:15:34.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.760+0000 7ff8a32e8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c10c5e0 con 0x7ff89c1018f0 2024-03-16T09:15:34.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.760+0000 7ff8a1084640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff89c100700 0x7ff89c1969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:34.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.760+0000 7ff8a0883640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff89c1018f0 0x7ff89c196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:34.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.760+0000 7ff8a1084640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff89c100700 0x7ff89c1969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57020/0 (socket says 172.21.15.112:57020) 2024-03-16T09:15:34.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.760+0000 7ff8a1084640 1 -- 172.21.15.112:0/2782226889 learned_addr learned my addr 172.21.15.112:0/2782226889 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:34.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.760+0000 7ff8a1885640 1 --2- 172.21.15.112:0/2782226889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff89c1022b0 0x7ff89c19df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:34.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.760+0000 7ff8a1084640 1 -- 172.21.15.112:0/2782226889 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff89c1018f0 msgr2=0x7ff89c196ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:34.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.760+0000 7ff8a1084640 1 --2- 172.21.15.112:0/2782226889 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff89c1018f0 0x7ff89c196ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:34.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.760+0000 7ff8a1084640 1 -- 172.21.15.112:0/2782226889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff89c1022b0 msgr2=0x7ff89c19df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:34.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.760+0000 7ff8a1084640 1 --2- 172.21.15.112:0/2782226889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff89c1022b0 0x7ff89c19df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:34.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.760+0000 7ff8a1084640 1 -- 172.21.15.112:0/2782226889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89c19e5d0 con 0x7ff89c100700 2024-03-16T09:15:34.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.760+0000 7ff8a1084640 1 --2- 172.21.15.112:0/2782226889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff89c100700 0x7ff89c1969b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff89400eb70 tx=0x7ff89400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:34.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.761+0000 7ff8927fc640 1 -- 172.21.15.112:0/2782226889 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894003b70 con 0x7ff89c100700 2024-03-16T09:15:34.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.761+0000 7ff8927fc640 1 -- 172.21.15.112:0/2782226889 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff89400ee70 con 0x7ff89c100700 2024-03-16T09:15:34.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.761+0000 7ff8a32e8640 1 -- 172.21.15.112:0/2782226889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89c19e8c0 con 0x7ff89c100700 2024-03-16T09:15:34.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.761+0000 7ff8927fc640 1 -- 172.21.15.112:0/2782226889 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8940108d0 con 0x7ff89c100700 2024-03-16T09:15:34.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.761+0000 7ff8a32e8640 1 -- 172.21.15.112:0/2782226889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff89c19ee10 con 0x7ff89c100700 2024-03-16T09:15:34.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.762+0000 7ff8a32e8640 1 -- 172.21.15.112:0/2782226889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff89c107950 con 0x7ff89c100700 2024-03-16T09:15:34.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.763+0000 7ff8927fc640 1 -- 172.21.15.112:0/2782226889 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7ff894003f30 con 0x7ff89c100700 2024-03-16T09:15:34.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.763+0000 7ff8927fc640 1 --2- 172.21.15.112:0/2782226889 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff8880a2920 0x7ff8880a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:34.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.763+0000 7ff8a0883640 1 --2- 172.21.15.112:0/2782226889 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff8880a2920 0x7ff8880a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:34.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.763+0000 7ff8927fc640 1 -- 172.21.15.112:0/2782226889 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff8940cefc0 con 0x7ff89c100700 2024-03-16T09:15:34.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.764+0000 7ff8a0883640 1 --2- 172.21.15.112:0/2782226889 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff8880a2920 0x7ff8880a4de0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff89c197ec0 tx=0x7ff88c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:34.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.768+0000 7ff8927fc640 1 -- 172.21.15.112:0/2782226889 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff89409c070 con 0x7ff89c100700 2024-03-16T09:15:34.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:34.981+0000 7ff8a32e8640 1 -- 172.21.15.112:0/2782226889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7ff89c0623e0 con 0x7ff89c100700 2024-03-16T09:15:35.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:35.963+0000 7ff8927fc640 1 -- 172.21.15.112:0/2782226889 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v22) ==== 118+0+0 (secure 0 0 0) 0x7ff89409b970 con 0x7ff89c100700 2024-03-16T09:15:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:35.966+0000 7ff8a32e8640 1 -- 172.21.15.112:0/2782226889 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff8880a2920 msgr2=0x7ff8880a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:35.966+0000 7ff8a32e8640 1 --2- 172.21.15.112:0/2782226889 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff8880a2920 0x7ff8880a4de0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff89c197ec0 tx=0x7ff88c005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:35.966+0000 7ff8a32e8640 1 -- 172.21.15.112:0/2782226889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff89c100700 msgr2=0x7ff89c1969b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:35.966+0000 7ff8a32e8640 1 --2- 172.21.15.112:0/2782226889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff89c100700 0x7ff89c1969b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff89400eb70 tx=0x7ff89400c6a0 comp rx=0 tx=0).stop 2024-03-16T09:15:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:35.966+0000 7ff8a32e8640 1 -- 172.21.15.112:0/2782226889 shutdown_connections 2024-03-16T09:15:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:35.966+0000 7ff8a32e8640 1 --2- 172.21.15.112:0/2782226889 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff8880a2920 0x7ff8880a4de0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:35.966+0000 7ff8a32e8640 1 --2- 172.21.15.112:0/2782226889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff89c1022b0 0x7ff89c19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:35.966+0000 7ff8a32e8640 1 --2- 172.21.15.112:0/2782226889 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff89c1018f0 0x7ff89c196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:35.966+0000 7ff8a32e8640 1 --2- 172.21.15.112:0/2782226889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff89c100700 0x7ff89c1969b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:35.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:35.966+0000 7ff8a32e8640 1 -- 172.21.15.112:0/2782226889 >> 172.21.15.112:0/2782226889 conn(0x7ff89c0fbc70 msgr2=0x7ff89c104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:35.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:35.966+0000 7ff8a32e8640 1 -- 172.21.15.112:0/2782226889 shutdown_connections 2024-03-16T09:15:35.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:35.966+0000 7ff8a32e8640 1 -- 172.21.15.112:0/2782226889 wait complete. 2024-03-16T09:15:35.981 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-16T09:15:36.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.090+0000 7f3344176640 1 -- 172.21.15.112:0/3956790050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f333c100070 msgr2=0x7f333c100490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:36.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.090+0000 7f3344176640 1 --2- 172.21.15.112:0/3956790050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f333c100070 0x7f333c100490 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3338009900 tx=0x7f333802bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:36.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.090+0000 7f3344176640 1 -- 172.21.15.112:0/3956790050 shutdown_connections 2024-03-16T09:15:36.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.090+0000 7f3344176640 1 --2- 172.21.15.112:0/3956790050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f333c101370 0x7f333c1017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.090+0000 7f3344176640 1 --2- 172.21.15.112:0/3956790050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f333c1009d0 0x7f333c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.090+0000 7f3344176640 1 --2- 172.21.15.112:0/3956790050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f333c100070 0x7f333c100490 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.090+0000 7f3344176640 1 -- 172.21.15.112:0/3956790050 >> 172.21.15.112:0/3956790050 conn(0x7f333c0fbc70 msgr2=0x7f333c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:36.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.090+0000 7f3344176640 1 -- 172.21.15.112:0/3956790050 shutdown_connections 2024-03-16T09:15:36.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.090+0000 7f3344176640 1 -- 172.21.15.112:0/3956790050 wait complete. 2024-03-16T09:15:36.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3344176640 1 Processor -- start 2024-03-16T09:15:36.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3344176640 1 -- start start 2024-03-16T09:15:36.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3344176640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f333c100070 0x7f333c199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:36.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3344176640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f333c1009d0 0x7f333c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:36.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3344176640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f333c101370 0x7f333c1a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:36.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3344176640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f333c10f770 con 0x7f333c100070 2024-03-16T09:15:36.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3344176640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f333c10f5f0 con 0x7f333c101370 2024-03-16T09:15:36.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3344176640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f333c10f8f0 con 0x7f333c1009d0 2024-03-16T09:15:36.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3341f12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f333c100070 0x7f333c199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:36.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3341f12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f333c100070 0x7f333c199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57038/0 (socket says 172.21.15.112:57038) 2024-03-16T09:15:36.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3341f12640 1 -- 172.21.15.112:0/3362809820 learned_addr learned my addr 172.21.15.112:0/3362809820 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:36.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3341711640 1 --2- 172.21.15.112:0/3362809820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f333c1009d0 0x7f333c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:36.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3342713640 1 --2- 172.21.15.112:0/3362809820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f333c101370 0x7f333c1a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:36.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3341f12640 1 -- 172.21.15.112:0/3362809820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f333c1009d0 msgr2=0x7f333c19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:36.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3341f12640 1 --2- 172.21.15.112:0/3362809820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f333c1009d0 0x7f333c19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3341f12640 1 -- 172.21.15.112:0/3362809820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f333c101370 msgr2=0x7f333c1a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:36.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3341f12640 1 --2- 172.21.15.112:0/3362809820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f333c101370 0x7f333c1a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3341f12640 1 -- 172.21.15.112:0/3362809820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333c1a1960 con 0x7f333c100070 2024-03-16T09:15:36.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3341f12640 1 --2- 172.21.15.112:0/3362809820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f333c100070 0x7f333c199d40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3338009230 tx=0x7f3338004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:36.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.091+0000 7f3332ffd640 1 -- 172.21.15.112:0/3362809820 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3338043020 con 0x7f333c100070 2024-03-16T09:15:36.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.092+0000 7f3344176640 1 -- 172.21.15.112:0/3362809820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f333c1a1bf0 con 0x7f333c100070 2024-03-16T09:15:36.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.092+0000 7f3332ffd640 1 -- 172.21.15.112:0/3362809820 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3338004250 con 0x7f333c100070 2024-03-16T09:15:36.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.092+0000 7f3332ffd640 1 -- 172.21.15.112:0/3362809820 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3338003b50 con 0x7f333c100070 2024-03-16T09:15:36.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.092+0000 7f3344176640 1 -- 172.21.15.112:0/3362809820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f333c1a2140 con 0x7f333c100070 2024-03-16T09:15:36.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.093+0000 7f3344176640 1 -- 172.21.15.112:0/3362809820 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f333c10adc0 con 0x7f333c100070 2024-03-16T09:15:36.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.094+0000 7f3332ffd640 1 -- 172.21.15.112:0/3362809820 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f3338046020 con 0x7f333c100070 2024-03-16T09:15:36.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.094+0000 7f3332ffd640 1 --2- 172.21.15.112:0/3362809820 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f33280a28a0 0x7f33280a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:36.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.094+0000 7f3341711640 1 --2- 172.21.15.112:0/3362809820 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f33280a28a0 0x7f33280a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:36.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.094+0000 7f3332ffd640 1 -- 172.21.15.112:0/3362809820 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f33380f0f90 con 0x7f333c100070 2024-03-16T09:15:36.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.095+0000 7f3341711640 1 --2- 172.21.15.112:0/3362809820 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f33280a28a0 0x7f33280a4d60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f333c19b250 tx=0x7f332c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:36.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.098+0000 7f3332ffd640 1 -- 172.21.15.112:0/3362809820 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f333804d020 con 0x7f333c100070 2024-03-16T09:15:36.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:35 smithi159 ceph-mon[36739]: pgmap v349: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T09:15:36.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:35 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2782226889' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-16T09:15:36.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.305+0000 7f3344176640 1 -- 172.21.15.112:0/3362809820 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f333c061d80 con 0x7f33280a28a0 2024-03-16T09:15:36.312 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:35 smithi112 ceph-mon[32307]: pgmap v349: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T09:15:36.312 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:35 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2782226889' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-16T09:15:36.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.321+0000 7f3332ffd640 1 -- 172.21.15.112:0/3362809820 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f333c061d80 con 0x7f33280a28a0 2024-03-16T09:15:36.323 INFO:teuthology.orchestra.run.smithi112.stdout:{} 2024-03-16T09:15:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.323+0000 7f3344176640 1 -- 172.21.15.112:0/3362809820 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f33280a28a0 msgr2=0x7f33280a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.323+0000 7f3344176640 1 --2- 172.21.15.112:0/3362809820 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f33280a28a0 0x7f33280a4d60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f333c19b250 tx=0x7f332c005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.323+0000 7f3344176640 1 -- 172.21.15.112:0/3362809820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f333c100070 msgr2=0x7f333c199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:36.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.323+0000 7f3344176640 1 --2- 172.21.15.112:0/3362809820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f333c100070 0x7f333c199d40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3338009230 tx=0x7f3338004ab0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.323+0000 7f3344176640 1 -- 172.21.15.112:0/3362809820 shutdown_connections 2024-03-16T09:15:36.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.323+0000 7f3344176640 1 --2- 172.21.15.112:0/3362809820 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f33280a28a0 0x7f33280a4d60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.323+0000 7f3344176640 1 --2- 172.21.15.112:0/3362809820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f333c101370 0x7f333c1a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.323+0000 7f3344176640 1 --2- 172.21.15.112:0/3362809820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f333c1009d0 0x7f333c19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.323+0000 7f3344176640 1 --2- 172.21.15.112:0/3362809820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f333c100070 0x7f333c199d40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.323+0000 7f3344176640 1 -- 172.21.15.112:0/3362809820 >> 172.21.15.112:0/3362809820 conn(0x7f333c0fbc70 msgr2=0x7f333c0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:36.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.323+0000 7f3344176640 1 -- 172.21.15.112:0/3362809820 shutdown_connections 2024-03-16T09:15:36.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.323+0000 7f3344176640 1 -- 172.21.15.112:0/3362809820 wait complete. 2024-03-16T09:15:36.336 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-16T09:15:36.346 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-16T09:15:36.380 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:15:36.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:35 smithi194 ceph-mon[36729]: pgmap v349: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-16T09:15:36.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:35 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2782226889' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-16T09:15:36.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.481+0000 7feb37f13640 1 -- 172.21.15.112:0/1638400025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb30103500 msgr2=0x7feb30109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:36.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.481+0000 7feb37f13640 1 --2- 172.21.15.112:0/1638400025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb30103500 0x7feb30109d90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7feb28009ca0 tx=0x7feb2802c140 comp rx=0 tx=0).stop 2024-03-16T09:15:36.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.481+0000 7feb37f13640 1 -- 172.21.15.112:0/1638400025 shutdown_connections 2024-03-16T09:15:36.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.481+0000 7feb37f13640 1 --2- 172.21.15.112:0/1638400025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb30103500 0x7feb30109d90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.481+0000 7feb37f13640 1 --2- 172.21.15.112:0/1638400025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb30102b40 0x7feb30102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.481+0000 7feb37f13640 1 --2- 172.21.15.112:0/1638400025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb300ffad0 0x7feb300ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.481+0000 7feb37f13640 1 -- 172.21.15.112:0/1638400025 >> 172.21.15.112:0/1638400025 conn(0x7feb300fb400 msgr2=0x7feb300fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:36.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.481+0000 7feb37f13640 1 -- 172.21.15.112:0/1638400025 shutdown_connections 2024-03-16T09:15:36.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.481+0000 7feb37f13640 1 -- 172.21.15.112:0/1638400025 wait complete. 2024-03-16T09:15:36.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.481+0000 7feb37f13640 1 Processor -- start 2024-03-16T09:15:36.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.481+0000 7feb37f13640 1 -- start start 2024-03-16T09:15:36.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb37f13640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb300ffad0 0x7feb30197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:36.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb37f13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb30102b40 0x7feb301980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:36.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb37f13640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb30103500 0x7feb3019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:36.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb37f13640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb3010d5e0 con 0x7feb30102b40 2024-03-16T09:15:36.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb37f13640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feb3010d460 con 0x7feb30103500 2024-03-16T09:15:36.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb37f13640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feb3010d760 con 0x7feb300ffad0 2024-03-16T09:15:36.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb354ae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb30102b40 0x7feb301980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:36.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb354ae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb30102b40 0x7feb301980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57076/0 (socket says 172.21.15.112:57076) 2024-03-16T09:15:36.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb354ae640 1 -- 172.21.15.112:0/40523400 learned_addr learned my addr 172.21.15.112:0/40523400 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:36.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb35caf640 1 --2- 172.21.15.112:0/40523400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb300ffad0 0x7feb30197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:36.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb364b0640 1 --2- 172.21.15.112:0/40523400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb30103500 0x7feb3019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:36.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb354ae640 1 -- 172.21.15.112:0/40523400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb300ffad0 msgr2=0x7feb30197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:36.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb354ae640 1 --2- 172.21.15.112:0/40523400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb300ffad0 0x7feb30197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb354ae640 1 -- 172.21.15.112:0/40523400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb30103500 msgr2=0x7feb3019f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:36.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb354ae640 1 --2- 172.21.15.112:0/40523400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb30103500 0x7feb3019f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb354ae640 1 -- 172.21.15.112:0/40523400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3019f7c0 con 0x7feb30102b40 2024-03-16T09:15:36.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb354ae640 1 --2- 172.21.15.112:0/40523400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb30102b40 0x7feb301980e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00ea60 tx=0x7feb2c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:36.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb26ffd640 1 -- 172.21.15.112:0/40523400 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c00cde0 con 0x7feb30102b40 2024-03-16T09:15:36.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb26ffd640 1 -- 172.21.15.112:0/40523400 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feb2c0035f0 con 0x7feb30102b40 2024-03-16T09:15:36.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb37f13640 1 -- 172.21.15.112:0/40523400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb3019fab0 con 0x7feb30102b40 2024-03-16T09:15:36.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb26ffd640 1 -- 172.21.15.112:0/40523400 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c010720 con 0x7feb30102b40 2024-03-16T09:15:36.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.482+0000 7feb37f13640 1 -- 172.21.15.112:0/40523400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb301a0000 con 0x7feb30102b40 2024-03-16T09:15:36.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.483+0000 7feb37f13640 1 -- 172.21.15.112:0/40523400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb30108ba0 con 0x7feb30102b40 2024-03-16T09:15:36.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.484+0000 7feb26ffd640 1 -- 172.21.15.112:0/40523400 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7feb2c0037a0 con 0x7feb30102b40 2024-03-16T09:15:36.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.485+0000 7feb26ffd640 1 --2- 172.21.15.112:0/40523400 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feb1c0a2800 0x7feb1c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:36.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.485+0000 7feb35caf640 1 --2- 172.21.15.112:0/40523400 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feb1c0a2800 0x7feb1c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:36.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.485+0000 7feb26ffd640 1 -- 172.21.15.112:0/40523400 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7feb2c0ced70 con 0x7feb30102b40 2024-03-16T09:15:36.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.485+0000 7feb35caf640 1 --2- 172.21.15.112:0/40523400 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feb1c0a2800 0x7feb1c0a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7feb30069b70 tx=0x7feb20005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:36.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.489+0000 7feb26ffd640 1 -- 172.21.15.112:0/40523400 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7feb2c09be20 con 0x7feb30102b40 2024-03-16T09:15:36.592 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:15:36 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:15:36.309+0000 7f4f8b2fb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T09:15:36.592 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:15:36 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:15:36.309+0000 7f4f8b2fb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T09:15:36.593 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:15:36 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:15:36.309+0000 7f4f8b2fb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T09:15:36.593 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:15:36 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:15:36.309+0000 7f4f8b2fb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T09:15:36.593 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:15:36 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:15:36.309+0000 7f4f8b2fb700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-16T09:15:36.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.667+0000 7feb37f13640 1 -- 172.21.15.112:0/40523400 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb30062480 con 0x7feb30102b40 2024-03-16T09:15:36.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.668+0000 7feb26ffd640 1 -- 172.21.15.112:0/40523400 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22725 (secure 0 0 0) 0x7feb2c09b720 con 0x7feb30102b40 2024-03-16T09:15:36.670 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:36.671 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":87,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:14:31.991998+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T09:13:35.040628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T09:13:35.689419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:15:36.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.670+0000 7feb37f13640 1 -- 172.21.15.112:0/40523400 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feb1c0a2800 msgr2=0x7feb1c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:36.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.670+0000 7feb37f13640 1 --2- 172.21.15.112:0/40523400 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feb1c0a2800 0x7feb1c0a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7feb30069b70 tx=0x7feb20005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:36.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.670+0000 7feb37f13640 1 -- 172.21.15.112:0/40523400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb30102b40 msgr2=0x7feb301980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:36.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.670+0000 7feb37f13640 1 --2- 172.21.15.112:0/40523400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb30102b40 0x7feb301980e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00ea60 tx=0x7feb2c00ef20 comp rx=0 tx=0).stop 2024-03-16T09:15:36.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.670+0000 7feb37f13640 1 -- 172.21.15.112:0/40523400 shutdown_connections 2024-03-16T09:15:36.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.670+0000 7feb37f13640 1 --2- 172.21.15.112:0/40523400 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feb1c0a2800 0x7feb1c0a4cc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.670+0000 7feb37f13640 1 --2- 172.21.15.112:0/40523400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb30103500 0x7feb3019f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.670+0000 7feb37f13640 1 --2- 172.21.15.112:0/40523400 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb30102b40 0x7feb301980e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.670+0000 7feb37f13640 1 --2- 172.21.15.112:0/40523400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb300ffad0 0x7feb30197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.670+0000 7feb37f13640 1 -- 172.21.15.112:0/40523400 >> 172.21.15.112:0/40523400 conn(0x7feb300fb400 msgr2=0x7feb30105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:36.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.670+0000 7feb37f13640 1 -- 172.21.15.112:0/40523400 shutdown_connections 2024-03-16T09:15:36.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.670+0000 7feb37f13640 1 -- 172.21.15.112:0/40523400 wait complete. 2024-03-16T09:15:36.685 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T09:08:57.135312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T09:13:35.040628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T09:13:35.689419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T09:15:36.686 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T09:15:36.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.789+0000 7f9f49dd4640 1 -- 172.21.15.112:0/2150800433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f441006e0 msgr2=0x7f9f44100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:36.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.789+0000 7f9f49dd4640 1 --2- 172.21.15.112:0/2150800433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f441006e0 0x7f9f44100b00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009900 tx=0x7f9f3402bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:36.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.789+0000 7f9f49dd4640 1 -- 172.21.15.112:0/2150800433 shutdown_connections 2024-03-16T09:15:36.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.789+0000 7f9f49dd4640 1 --2- 172.21.15.112:0/2150800433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f44102290 0x7f9f44108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.789+0000 7f9f49dd4640 1 --2- 172.21.15.112:0/2150800433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f441018d0 0x7f9f44101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.789+0000 7f9f49dd4640 1 --2- 172.21.15.112:0/2150800433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f441006e0 0x7f9f44100b00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.789+0000 7f9f49dd4640 1 -- 172.21.15.112:0/2150800433 >> 172.21.15.112:0/2150800433 conn(0x7f9f440fbc70 msgr2=0x7f9f440fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:36.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.789+0000 7f9f49dd4640 1 -- 172.21.15.112:0/2150800433 shutdown_connections 2024-03-16T09:15:36.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.789+0000 7f9f49dd4640 1 -- 172.21.15.112:0/2150800433 wait complete. 2024-03-16T09:15:36.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.790+0000 7f9f49dd4640 1 Processor -- start 2024-03-16T09:15:36.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.790+0000 7f9f49dd4640 1 -- start start 2024-03-16T09:15:36.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.790+0000 7f9f49dd4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f441006e0 0x7f9f44197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:36.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.790+0000 7f9f49dd4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f441018d0 0x7f9f441981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:36.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.790+0000 7f9f49dd4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f44102290 0x7f9f4419f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:36.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.790+0000 7f9f49dd4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4410c440 con 0x7f9f441006e0 2024-03-16T09:15:36.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.790+0000 7f9f49dd4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4410c2c0 con 0x7f9f44102290 2024-03-16T09:15:36.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.790+0000 7f9f49dd4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4410c5c0 con 0x7f9f441018d0 2024-03-16T09:15:36.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.790+0000 7f9f437fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f441006e0 0x7f9f44197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:36.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.790+0000 7f9f437fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f441006e0 0x7f9f44197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57096/0 (socket says 172.21.15.112:57096) 2024-03-16T09:15:36.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.790+0000 7f9f437fe640 1 -- 172.21.15.112:0/1857454653 learned_addr learned my addr 172.21.15.112:0/1857454653 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:36.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.790+0000 7f9f42ffd640 1 --2- 172.21.15.112:0/1857454653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f441018d0 0x7f9f441981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:36.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.790+0000 7f9f43fff640 1 --2- 172.21.15.112:0/1857454653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f44102290 0x7f9f4419f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:36.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.791+0000 7f9f437fe640 1 -- 172.21.15.112:0/1857454653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f441018d0 msgr2=0x7f9f441981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:36.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.791+0000 7f9f437fe640 1 --2- 172.21.15.112:0/1857454653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f441018d0 0x7f9f441981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.791+0000 7f9f437fe640 1 -- 172.21.15.112:0/1857454653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f44102290 msgr2=0x7f9f4419f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:36.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.791+0000 7f9f437fe640 1 --2- 172.21.15.112:0/1857454653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f44102290 0x7f9f4419f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.791+0000 7f9f437fe640 1 -- 172.21.15.112:0/1857454653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4419f870 con 0x7f9f441006e0 2024-03-16T09:15:36.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.791+0000 7f9f43fff640 1 --2- 172.21.15.112:0/1857454653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f44102290 0x7f9f4419f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:36.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.791+0000 7f9f437fe640 1 --2- 172.21.15.112:0/1857454653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f441006e0 0x7f9f44197c60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9f34000c00 tx=0x7f9f34004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:36.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.791+0000 7f9f40ff9640 1 -- 172.21.15.112:0/1857454653 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f34043020 con 0x7f9f441006e0 2024-03-16T09:15:36.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.791+0000 7f9f40ff9640 1 -- 172.21.15.112:0/1857454653 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9f34004250 con 0x7f9f441006e0 2024-03-16T09:15:36.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.791+0000 7f9f40ff9640 1 -- 172.21.15.112:0/1857454653 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f34003b90 con 0x7f9f441006e0 2024-03-16T09:15:36.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.791+0000 7f9f49dd4640 1 -- 172.21.15.112:0/1857454653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4419fb00 con 0x7f9f441006e0 2024-03-16T09:15:36.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.791+0000 7f9f49dd4640 1 -- 172.21.15.112:0/1857454653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f441a0050 con 0x7f9f441006e0 2024-03-16T09:15:36.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.792+0000 7f9f49dd4640 1 -- 172.21.15.112:0/1857454653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f441a0220 con 0x7f9f441006e0 2024-03-16T09:15:36.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.793+0000 7f9f40ff9640 1 -- 172.21.15.112:0/1857454653 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f9f34046020 con 0x7f9f441006e0 2024-03-16T09:15:36.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.793+0000 7f9f40ff9640 1 --2- 172.21.15.112:0/1857454653 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9f300a2800 0x7f9f300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:36.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.794+0000 7f9f42ffd640 1 --2- 172.21.15.112:0/1857454653 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9f300a2800 0x7f9f300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:36.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.794+0000 7f9f40ff9640 1 -- 172.21.15.112:0/1857454653 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f9f340f0bb0 con 0x7f9f441006e0 2024-03-16T09:15:36.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.794+0000 7f9f42ffd640 1 --2- 172.21.15.112:0/1857454653 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9f300a2800 0x7f9f300a4cc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009bd0 tx=0x7f9f2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:36.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.798+0000 7f9f40ff9640 1 -- 172.21.15.112:0/1857454653 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9f3404d020 con 0x7f9f441006e0 2024-03-16T09:15:36.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.973+0000 7f9f49dd4640 1 -- 172.21.15.112:0/1857454653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9f441a03f0 con 0x7f9f441006e0 2024-03-16T09:15:36.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.973+0000 7f9f40ff9640 1 -- 172.21.15.112:0/1857454653 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f9f340bdc60 con 0x7f9f441006e0 2024-03-16T09:15:36.975 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-03-16T09:15:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.975+0000 7f9f49dd4640 1 -- 172.21.15.112:0/1857454653 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9f300a2800 msgr2=0x7f9f300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.975+0000 7f9f49dd4640 1 --2- 172.21.15.112:0/1857454653 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9f300a2800 0x7f9f300a4cc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009bd0 tx=0x7f9f2c005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.975+0000 7f9f49dd4640 1 -- 172.21.15.112:0/1857454653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f441006e0 msgr2=0x7f9f44197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.975+0000 7f9f49dd4640 1 --2- 172.21.15.112:0/1857454653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f441006e0 0x7f9f44197c60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9f34000c00 tx=0x7f9f34004a30 comp rx=0 tx=0).stop 2024-03-16T09:15:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.975+0000 7f9f49dd4640 1 -- 172.21.15.112:0/1857454653 shutdown_connections 2024-03-16T09:15:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.975+0000 7f9f49dd4640 1 --2- 172.21.15.112:0/1857454653 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9f300a2800 0x7f9f300a4cc0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.975+0000 7f9f49dd4640 1 --2- 172.21.15.112:0/1857454653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f44102290 0x7f9f4419f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.975+0000 7f9f49dd4640 1 --2- 172.21.15.112:0/1857454653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f441018d0 0x7f9f441981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.975+0000 7f9f49dd4640 1 --2- 172.21.15.112:0/1857454653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f441006e0 0x7f9f44197c60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.975+0000 7f9f49dd4640 1 -- 172.21.15.112:0/1857454653 >> 172.21.15.112:0/1857454653 conn(0x7f9f440fbc70 msgr2=0x7f9f44068940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:36.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.976+0000 7f9f49dd4640 1 -- 172.21.15.112:0/1857454653 shutdown_connections 2024-03-16T09:15:36.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:36.976+0000 7f9f49dd4640 1 -- 172.21.15.112:0/1857454653 wait complete. 2024-03-16T09:15:36.989 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T09:15:37.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.091+0000 7fe6d104d640 1 -- 172.21.15.112:0/3394178888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6cc100fe0 msgr2=0x7fe6cc101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:37.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.091+0000 7fe6d104d640 1 --2- 172.21.15.112:0/3394178888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6cc100fe0 0x7fe6cc101460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009900 tx=0x7fe6c002bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:37.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.091+0000 7fe6d104d640 1 -- 172.21.15.112:0/3394178888 shutdown_connections 2024-03-16T09:15:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.091+0000 7fe6d104d640 1 --2- 172.21.15.112:0/3394178888 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6cc1019a0 0x7fe6cc1082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.091+0000 7fe6d104d640 1 --2- 172.21.15.112:0/3394178888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6cc100fe0 0x7fe6cc101460 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.091+0000 7fe6d104d640 1 --2- 172.21.15.112:0/3394178888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6cc0ffe90 0x7fe6cc1002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.091+0000 7fe6d104d640 1 -- 172.21.15.112:0/3394178888 >> 172.21.15.112:0/3394178888 conn(0x7fe6cc0fb3e0 msgr2=0x7fe6cc0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.091+0000 7fe6d104d640 1 -- 172.21.15.112:0/3394178888 shutdown_connections 2024-03-16T09:15:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.091+0000 7fe6d104d640 1 -- 172.21.15.112:0/3394178888 wait complete. 2024-03-16T09:15:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.091+0000 7fe6d104d640 1 Processor -- start 2024-03-16T09:15:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.092+0000 7fe6d104d640 1 -- start start 2024-03-16T09:15:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.092+0000 7fe6d104d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6cc0ffe90 0x7fe6cc196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.092+0000 7fe6d104d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6cc100fe0 0x7fe6cc196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.092+0000 7fe6d104d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6cc1019a0 0x7fe6cc19e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.092+0000 7fe6d104d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc10bc20 con 0x7fe6cc100fe0 2024-03-16T09:15:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.092+0000 7fe6d104d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc10baa0 con 0x7fe6cc1019a0 2024-03-16T09:15:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.092+0000 7fe6d104d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc10bda0 con 0x7fe6cc0ffe90 2024-03-16T09:15:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.092+0000 7fe6ca59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6cc100fe0 0x7fe6cc196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.092+0000 7fe6ca59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6cc100fe0 0x7fe6cc196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57112/0 (socket says 172.21.15.112:57112) 2024-03-16T09:15:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.092+0000 7fe6ca59c640 1 -- 172.21.15.112:0/4124757243 learned_addr learned my addr 172.21.15.112:0/4124757243 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.092+0000 7fe6cb59e640 1 --2- 172.21.15.112:0/4124757243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6cc1019a0 0x7fe6cc19e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.092+0000 7fe6cad9d640 1 --2- 172.21.15.112:0/4124757243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6cc0ffe90 0x7fe6cc196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.092+0000 7fe6ca59c640 1 -- 172.21.15.112:0/4124757243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6cc0ffe90 msgr2=0x7fe6cc196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.092+0000 7fe6ca59c640 1 --2- 172.21.15.112:0/4124757243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6cc0ffe90 0x7fe6cc196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.092+0000 7fe6ca59c640 1 -- 172.21.15.112:0/4124757243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6cc1019a0 msgr2=0x7fe6cc19e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.092+0000 7fe6ca59c640 1 --2- 172.21.15.112:0/4124757243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6cc1019a0 0x7fe6cc19e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.092+0000 7fe6ca59c640 1 -- 172.21.15.112:0/4124757243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6cc19e6d0 con 0x7fe6cc100fe0 2024-03-16T09:15:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.092+0000 7fe6cb59e640 1 --2- 172.21.15.112:0/4124757243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6cc1019a0 0x7fe6cc19e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.092+0000 7fe6ca59c640 1 --2- 172.21.15.112:0/4124757243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6cc100fe0 0x7fe6cc196ff0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0000c00 tx=0x7fe6c0004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.093+0000 7fe6b3fff640 1 -- 172.21.15.112:0/4124757243 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0043020 con 0x7fe6cc100fe0 2024-03-16T09:15:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.093+0000 7fe6b3fff640 1 -- 172.21.15.112:0/4124757243 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe6c0004250 con 0x7fe6cc100fe0 2024-03-16T09:15:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.093+0000 7fe6d104d640 1 -- 172.21.15.112:0/4124757243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6cc19e960 con 0x7fe6cc100fe0 2024-03-16T09:15:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.093+0000 7fe6b3fff640 1 -- 172.21.15.112:0/4124757243 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0003b90 con 0x7fe6cc100fe0 2024-03-16T09:15:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.093+0000 7fe6d104d640 1 -- 172.21.15.112:0/4124757243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6cc19eeb0 con 0x7fe6cc100fe0 2024-03-16T09:15:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.094+0000 7fe6d104d640 1 -- 172.21.15.112:0/4124757243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6cc107510 con 0x7fe6cc100fe0 2024-03-16T09:15:37.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.095+0000 7fe6b3fff640 1 -- 172.21.15.112:0/4124757243 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7fe6c0046020 con 0x7fe6cc100fe0 2024-03-16T09:15:37.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.095+0000 7fe6b3fff640 1 --2- 172.21.15.112:0/4124757243 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe6b40a2800 0x7fe6b40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:37.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.095+0000 7fe6cad9d640 1 --2- 172.21.15.112:0/4124757243 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe6b40a2800 0x7fe6b40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:37.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.096+0000 7fe6b3fff640 1 -- 172.21.15.112:0/4124757243 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fe6c00f0b00 con 0x7fe6cc100fe0 2024-03-16T09:15:37.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.096+0000 7fe6cad9d640 1 --2- 172.21.15.112:0/4124757243 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe6b40a2800 0x7fe6b40a4cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0095f0 tx=0x7fe6bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:37.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.100+0000 7fe6b3fff640 1 -- 172.21.15.112:0/4124757243 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe6c004d020 con 0x7fe6cc100fe0 2024-03-16T09:15:37.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:36 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2782226889' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-16T09:15:37.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:36 smithi159 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:15:37.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:36 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/40523400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:37.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.278+0000 7fe6d104d640 1 -- 172.21.15.112:0/4124757243 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe6cc0623e0 con 0x7fe6cc100fe0 2024-03-16T09:15:37.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.278+0000 7fe6b3fff640 1 -- 172.21.15.112:0/4124757243 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fe6c00bdbb0 con 0x7fe6cc100fe0 2024-03-16T09:15:37.281 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-03-16T09:15:37.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.281+0000 7fe6d104d640 1 -- 172.21.15.112:0/4124757243 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe6b40a2800 msgr2=0x7fe6b40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:37.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.281+0000 7fe6d104d640 1 --2- 172.21.15.112:0/4124757243 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe6b40a2800 0x7fe6b40a4cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0095f0 tx=0x7fe6bc005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:37.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.281+0000 7fe6d104d640 1 -- 172.21.15.112:0/4124757243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6cc100fe0 msgr2=0x7fe6cc196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:37.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.281+0000 7fe6d104d640 1 --2- 172.21.15.112:0/4124757243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6cc100fe0 0x7fe6cc196ff0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0000c00 tx=0x7fe6c0004a30 comp rx=0 tx=0).stop 2024-03-16T09:15:37.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.281+0000 7fe6d104d640 1 -- 172.21.15.112:0/4124757243 shutdown_connections 2024-03-16T09:15:37.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.281+0000 7fe6d104d640 1 --2- 172.21.15.112:0/4124757243 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fe6b40a2800 0x7fe6b40a4cc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.281+0000 7fe6d104d640 1 --2- 172.21.15.112:0/4124757243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6cc1019a0 0x7fe6cc19e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.281+0000 7fe6d104d640 1 --2- 172.21.15.112:0/4124757243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6cc100fe0 0x7fe6cc196ff0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.281+0000 7fe6d104d640 1 --2- 172.21.15.112:0/4124757243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6cc0ffe90 0x7fe6cc196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.281+0000 7fe6d104d640 1 -- 172.21.15.112:0/4124757243 >> 172.21.15.112:0/4124757243 conn(0x7fe6cc0fb3e0 msgr2=0x7fe6cc1046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:37.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.281+0000 7fe6d104d640 1 -- 172.21.15.112:0/4124757243 shutdown_connections 2024-03-16T09:15:37.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.281+0000 7fe6d104d640 1 -- 172.21.15.112:0/4124757243 wait complete. 2024-03-16T09:15:37.293 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:36 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2782226889' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-16T09:15:37.293 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:36 smithi112 ceph-mon[32307]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:15:37.293 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:36 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/40523400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:37.294 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T09:15:37.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.397+0000 7f2ef00d1640 1 -- 172.21.15.112:0/1372465044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee81018d0 msgr2=0x7f2ee8101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:37.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.397+0000 7f2ef00d1640 1 --2- 172.21.15.112:0/1372465044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee81018d0 0x7f2ee8101d50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4009900 tx=0x7f2ee402bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:37.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.397+0000 7f2ef00d1640 1 -- 172.21.15.112:0/1372465044 shutdown_connections 2024-03-16T09:15:37.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.397+0000 7f2ef00d1640 1 --2- 172.21.15.112:0/1372465044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ee8102290 0x7f2ee8108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.397+0000 7f2ef00d1640 1 --2- 172.21.15.112:0/1372465044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee81018d0 0x7f2ee8101d50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.397+0000 7f2ef00d1640 1 --2- 172.21.15.112:0/1372465044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ee81006e0 0x7f2ee8100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.397+0000 7f2ef00d1640 1 -- 172.21.15.112:0/1372465044 >> 172.21.15.112:0/1372465044 conn(0x7f2ee80fbc70 msgr2=0x7f2ee80fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:37.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.397+0000 7f2ef00d1640 1 -- 172.21.15.112:0/1372465044 shutdown_connections 2024-03-16T09:15:37.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.397+0000 7f2ef00d1640 1 -- 172.21.15.112:0/1372465044 wait complete. 2024-03-16T09:15:37.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.397+0000 7f2ef00d1640 1 Processor -- start 2024-03-16T09:15:37.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.397+0000 7f2ef00d1640 1 -- start start 2024-03-16T09:15:37.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.397+0000 7f2ef00d1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ee81006e0 0x7f2ee8104600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:37.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.397+0000 7f2ef00d1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee81018d0 0x7f2ee8102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:37.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.397+0000 7f2ef00d1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ee8102290 0x7f2ee81031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:37.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.397+0000 7f2ef00d1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee810c440 con 0x7f2ee81018d0 2024-03-16T09:15:37.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.397+0000 7f2ef00d1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee810c2c0 con 0x7f2ee81006e0 2024-03-16T09:15:37.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.397+0000 7f2ef00d1640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee810c5c0 con 0x7f2ee8102290 2024-03-16T09:15:37.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.397+0000 7f2eed66c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee81018d0 0x7f2ee8102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:37.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.398+0000 7f2eed66c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee81018d0 0x7f2ee8102cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57130/0 (socket says 172.21.15.112:57130) 2024-03-16T09:15:37.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.398+0000 7f2eede6d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ee81006e0 0x7f2ee8104600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:37.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.398+0000 7f2eed66c640 1 -- 172.21.15.112:0/562275246 learned_addr learned my addr 172.21.15.112:0/562275246 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:37.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.398+0000 7f2eee66e640 1 --2- 172.21.15.112:0/562275246 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ee8102290 0x7f2ee81031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:37.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.398+0000 7f2eed66c640 1 -- 172.21.15.112:0/562275246 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ee8102290 msgr2=0x7f2ee81031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:37.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.398+0000 7f2eed66c640 1 --2- 172.21.15.112:0/562275246 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ee8102290 0x7f2ee81031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.398+0000 7f2eed66c640 1 -- 172.21.15.112:0/562275246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ee81006e0 msgr2=0x7f2ee8104600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:37.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.398+0000 7f2eed66c640 1 --2- 172.21.15.112:0/562275246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ee81006e0 0x7f2ee8104600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.398+0000 7f2eed66c640 1 -- 172.21.15.112:0/562275246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee8103a10 con 0x7f2ee81018d0 2024-03-16T09:15:37.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.398+0000 7f2eed66c640 1 --2- 172.21.15.112:0/562275246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee81018d0 0x7f2ee8102cb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4009230 tx=0x7f2ee4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:37.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.398+0000 7f2edeffd640 1 -- 172.21.15.112:0/562275246 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4043020 con 0x7f2ee81018d0 2024-03-16T09:15:37.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.398+0000 7f2edeffd640 1 -- 172.21.15.112:0/562275246 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2ee4004250 con 0x7f2ee81018d0 2024-03-16T09:15:37.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.398+0000 7f2ef00d1640 1 -- 172.21.15.112:0/562275246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee81a1c90 con 0x7f2ee81018d0 2024-03-16T09:15:37.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.398+0000 7f2edeffd640 1 -- 172.21.15.112:0/562275246 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4003b50 con 0x7f2ee81018d0 2024-03-16T09:15:37.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.398+0000 7f2ef00d1640 1 -- 172.21.15.112:0/562275246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee81a2190 con 0x7f2ee81018d0 2024-03-16T09:15:37.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.399+0000 7f2ef00d1640 1 -- 172.21.15.112:0/562275246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee81a2360 con 0x7f2ee81018d0 2024-03-16T09:15:37.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:36 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2782226889' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-16T09:15:37.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:36 smithi194 ceph-mon[36729]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:15:37.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:36 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/40523400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:37.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.400+0000 7f2edeffd640 1 -- 172.21.15.112:0/562275246 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f2ee4046020 con 0x7f2ee81018d0 2024-03-16T09:15:37.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.401+0000 7f2edeffd640 1 --2- 172.21.15.112:0/562275246 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2ed40a2800 0x7f2ed40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:37.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.401+0000 7f2eede6d640 1 --2- 172.21.15.112:0/562275246 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2ed40a2800 0x7f2ed40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:37.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.401+0000 7f2edeffd640 1 -- 172.21.15.112:0/562275246 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f2ee40f0f10 con 0x7f2ee81018d0 2024-03-16T09:15:37.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.401+0000 7f2eede6d640 1 --2- 172.21.15.112:0/562275246 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2ed40a2800 0x7f2ed40a4cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8101730 tx=0x7f2ed8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:37.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.404+0000 7f2edeffd640 1 -- 172.21.15.112:0/562275246 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2ee404d020 con 0x7f2ee81018d0 2024-03-16T09:15:37.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.578+0000 7f2ef00d1640 1 -- 172.21.15.112:0/562275246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2ee81a2530 con 0x7f2ee81018d0 2024-03-16T09:15:37.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.578+0000 7f2edeffd640 1 -- 172.21.15.112:0/562275246 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f2ee40bdfc0 con 0x7f2ee81018d0 2024-03-16T09:15:37.580 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-03-16T09:15:37.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.580+0000 7f2ef00d1640 1 -- 172.21.15.112:0/562275246 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2ed40a2800 msgr2=0x7f2ed40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:37.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.580+0000 7f2ef00d1640 1 --2- 172.21.15.112:0/562275246 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2ed40a2800 0x7f2ed40a4cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8101730 tx=0x7f2ed8005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:37.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.580+0000 7f2ef00d1640 1 -- 172.21.15.112:0/562275246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee81018d0 msgr2=0x7f2ee8102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:37.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.580+0000 7f2ef00d1640 1 --2- 172.21.15.112:0/562275246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee81018d0 0x7f2ee8102cb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4009230 tx=0x7f2ee4004ab0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.580+0000 7f2ef00d1640 1 -- 172.21.15.112:0/562275246 shutdown_connections 2024-03-16T09:15:37.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.580+0000 7f2ef00d1640 1 --2- 172.21.15.112:0/562275246 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2ed40a2800 0x7f2ed40a4cc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.580+0000 7f2ef00d1640 1 --2- 172.21.15.112:0/562275246 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ee8102290 0x7f2ee81031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.580+0000 7f2ef00d1640 1 --2- 172.21.15.112:0/562275246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ee81018d0 0x7f2ee8102cb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.580+0000 7f2ef00d1640 1 --2- 172.21.15.112:0/562275246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ee81006e0 0x7f2ee8104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.580+0000 7f2ef00d1640 1 -- 172.21.15.112:0/562275246 >> 172.21.15.112:0/562275246 conn(0x7f2ee80fbc70 msgr2=0x7f2ee80fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:37.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.580+0000 7f2ef00d1640 1 -- 172.21.15.112:0/562275246 shutdown_connections 2024-03-16T09:15:37.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.580+0000 7f2ef00d1640 1 -- 172.21.15.112:0/562275246 wait complete. 2024-03-16T09:15:37.593 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:15:37.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.692+0000 7fc066b3b640 1 -- 172.21.15.112:0/1214963012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc060100700 msgr2=0x7fc060100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:37.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.692+0000 7fc066b3b640 1 --2- 172.21.15.112:0/1214963012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc060100700 0x7fc060100b20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc050009900 tx=0x7fc05002bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:37.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.692+0000 7fc066b3b640 1 -- 172.21.15.112:0/1214963012 shutdown_connections 2024-03-16T09:15:37.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.692+0000 7fc066b3b640 1 --2- 172.21.15.112:0/1214963012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0601022b0 0x7fc060108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.692+0000 7fc066b3b640 1 --2- 172.21.15.112:0/1214963012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0601018f0 0x7fc060101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.692+0000 7fc066b3b640 1 --2- 172.21.15.112:0/1214963012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc060100700 0x7fc060100b20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.692+0000 7fc066b3b640 1 -- 172.21.15.112:0/1214963012 >> 172.21.15.112:0/1214963012 conn(0x7fc0600fbc70 msgr2=0x7fc0600fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:37.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.692+0000 7fc066b3b640 1 -- 172.21.15.112:0/1214963012 shutdown_connections 2024-03-16T09:15:37.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.692+0000 7fc066b3b640 1 -- 172.21.15.112:0/1214963012 wait complete. 2024-03-16T09:15:37.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.693+0000 7fc066b3b640 1 Processor -- start 2024-03-16T09:15:37.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.693+0000 7fc066b3b640 1 -- start start 2024-03-16T09:15:37.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.693+0000 7fc066b3b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc060100700 0x7fc060069510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:37.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.693+0000 7fc066b3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0601018f0 0x7fc06019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:37.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.693+0000 7fc066b3b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0601022b0 0x7fc06019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:37.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.693+0000 7fc066b3b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc06010c450 con 0x7fc0601018f0 2024-03-16T09:15:37.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.693+0000 7fc066b3b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc06010c2d0 con 0x7fc060100700 2024-03-16T09:15:37.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.693+0000 7fc066b3b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc06010c5d0 con 0x7fc0601022b0 2024-03-16T09:15:37.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.693+0000 7fc05ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0601018f0 0x7fc06019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:37.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.693+0000 7fc05ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0601018f0 0x7fc06019c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57140/0 (socket says 172.21.15.112:57140) 2024-03-16T09:15:37.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.693+0000 7fc05ffff640 1 -- 172.21.15.112:0/568234433 learned_addr learned my addr 172.21.15.112:0/568234433 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:37.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.693+0000 7fc0648d7640 1 --2- 172.21.15.112:0/568234433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc060100700 0x7fc060069510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:37.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.693+0000 7fc0650d8640 1 --2- 172.21.15.112:0/568234433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0601022b0 0x7fc06019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:37.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.693+0000 7fc05ffff640 1 -- 172.21.15.112:0/568234433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0601022b0 msgr2=0x7fc06019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:37.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.693+0000 7fc05ffff640 1 --2- 172.21.15.112:0/568234433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0601022b0 0x7fc06019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.693+0000 7fc05ffff640 1 -- 172.21.15.112:0/568234433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc060100700 msgr2=0x7fc060069510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:37.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.693+0000 7fc05ffff640 1 --2- 172.21.15.112:0/568234433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc060100700 0x7fc060069510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.694+0000 7fc05ffff640 1 -- 172.21.15.112:0/568234433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06019f810 con 0x7fc0601018f0 2024-03-16T09:15:37.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.694+0000 7fc05ffff640 1 --2- 172.21.15.112:0/568234433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0601018f0 0x7fc06019c8e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc05400e9f0 tx=0x7fc05400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:37.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.694+0000 7fc05dffb640 1 -- 172.21.15.112:0/568234433 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05400cd80 con 0x7fc0601018f0 2024-03-16T09:15:37.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.694+0000 7fc05dffb640 1 -- 172.21.15.112:0/568234433 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc054010930 con 0x7fc0601018f0 2024-03-16T09:15:37.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.694+0000 7fc066b3b640 1 -- 172.21.15.112:0/568234433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc06019fb00 con 0x7fc0601018f0 2024-03-16T09:15:37.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.694+0000 7fc05dffb640 1 -- 172.21.15.112:0/568234433 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc054018b00 con 0x7fc0601018f0 2024-03-16T09:15:37.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.694+0000 7fc066b3b640 1 -- 172.21.15.112:0/568234433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0601a0050 con 0x7fc0601018f0 2024-03-16T09:15:37.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.695+0000 7fc066b3b640 1 -- 172.21.15.112:0/568234433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc060107a20 con 0x7fc0601018f0 2024-03-16T09:15:37.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.697+0000 7fc05dffb640 1 -- 172.21.15.112:0/568234433 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7fc054018ca0 con 0x7fc0601018f0 2024-03-16T09:15:37.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.697+0000 7fc05dffb640 1 --2- 172.21.15.112:0/568234433 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc0480a2800 0x7fc0480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:37.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.697+0000 7fc0648d7640 1 --2- 172.21.15.112:0/568234433 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc0480a2800 0x7fc0480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:37.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.697+0000 7fc05dffb640 1 -- 172.21.15.112:0/568234433 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc0540cf0b0 con 0x7fc0601018f0 2024-03-16T09:15:37.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.698+0000 7fc0648d7640 1 --2- 172.21.15.112:0/568234433 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc0480a2800 0x7fc0480a4cc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc050000c00 tx=0x7fc050005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:37.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.701+0000 7fc05dffb640 1 -- 172.21.15.112:0/568234433 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc05409bb10 con 0x7fc0601018f0 2024-03-16T09:15:37.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.876+0000 7fc066b3b640 1 -- 172.21.15.112:0/568234433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc06002d380 con 0x7fc0601018f0 2024-03-16T09:15:37.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.877+0000 7fc05dffb640 1 -- 172.21.15.112:0/568234433 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22725 (secure 0 0 0) 0x7fc05409fbb0 con 0x7fc0601018f0 2024-03-16T09:15:37.880 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:37.880 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":87,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:14:31.991998+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T09:13:35.040628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T09:13:35.689419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:15:37.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.879+0000 7fc066b3b640 1 -- 172.21.15.112:0/568234433 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc0480a2800 msgr2=0x7fc0480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:37.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.879+0000 7fc066b3b640 1 --2- 172.21.15.112:0/568234433 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc0480a2800 0x7fc0480a4cc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc050000c00 tx=0x7fc050005fd0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.879+0000 7fc066b3b640 1 -- 172.21.15.112:0/568234433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0601018f0 msgr2=0x7fc06019c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:37.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.879+0000 7fc066b3b640 1 --2- 172.21.15.112:0/568234433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0601018f0 0x7fc06019c8e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc05400e9f0 tx=0x7fc05400eeb0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.879+0000 7fc066b3b640 1 -- 172.21.15.112:0/568234433 shutdown_connections 2024-03-16T09:15:37.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.879+0000 7fc066b3b640 1 --2- 172.21.15.112:0/568234433 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc0480a2800 0x7fc0480a4cc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.879+0000 7fc066b3b640 1 --2- 172.21.15.112:0/568234433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc0601022b0 0x7fc06019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.879+0000 7fc066b3b640 1 --2- 172.21.15.112:0/568234433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0601018f0 0x7fc06019c8e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.879+0000 7fc066b3b640 1 --2- 172.21.15.112:0/568234433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc060100700 0x7fc060069510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:37.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.879+0000 7fc066b3b640 1 -- 172.21.15.112:0/568234433 >> 172.21.15.112:0/568234433 conn(0x7fc0600fbc70 msgr2=0x7fc060104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:37.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.880+0000 7fc066b3b640 1 -- 172.21.15.112:0/568234433 shutdown_connections 2024-03-16T09:15:37.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.880+0000 7fc066b3b640 1 -- 172.21.15.112:0/568234433 wait complete. 2024-03-16T09:15:37.893 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T09:08:57.135312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T09:13:35.040628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T09:13:35.689419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T09:15:37.893 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T09:15:38.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.997+0000 7fa20fe6c640 1 -- 172.21.15.112:0/3954800579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2081018f0 msgr2=0x7fa208101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:38.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.997+0000 7fa20fe6c640 1 --2- 172.21.15.112:0/3954800579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2081018f0 0x7fa208101d70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009900 tx=0x7fa1f802bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:38.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.997+0000 7fa20fe6c640 1 -- 172.21.15.112:0/3954800579 shutdown_connections 2024-03-16T09:15:38.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.997+0000 7fa20fe6c640 1 --2- 172.21.15.112:0/3954800579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2081022b0 0x7fa208108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.997+0000 7fa20fe6c640 1 --2- 172.21.15.112:0/3954800579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2081018f0 0x7fa208101d70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.997+0000 7fa20fe6c640 1 --2- 172.21.15.112:0/3954800579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa208100700 0x7fa208100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.997+0000 7fa20fe6c640 1 -- 172.21.15.112:0/3954800579 >> 172.21.15.112:0/3954800579 conn(0x7fa2080fbc90 msgr2=0x7fa2080fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:38.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.997+0000 7fa20fe6c640 1 -- 172.21.15.112:0/3954800579 shutdown_connections 2024-03-16T09:15:38.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.997+0000 7fa20fe6c640 1 -- 172.21.15.112:0/3954800579 wait complete. 2024-03-16T09:15:38.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.997+0000 7fa20fe6c640 1 Processor -- start 2024-03-16T09:15:38.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa20fe6c640 1 -- start start 2024-03-16T09:15:38.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa20fe6c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa208100700 0x7fa208197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:38.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa20fe6c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2081018f0 0x7fa208198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:38.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa20fe6c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2081022b0 0x7fa20819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:38.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa20fe6c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa20810d570 con 0x7fa2081022b0 2024-03-16T09:15:38.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa20fe6c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa20810d3f0 con 0x7fa208100700 2024-03-16T09:15:38.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa20fe6c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa20810d6f0 con 0x7fa2081018f0 2024-03-16T09:15:38.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa20e409640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2081022b0 0x7fa20819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:38.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa20e409640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2081022b0 0x7fa20819f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57160/0 (socket says 172.21.15.112:57160) 2024-03-16T09:15:38.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa20e409640 1 -- 172.21.15.112:0/1433566425 learned_addr learned my addr 172.21.15.112:0/1433566425 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:38.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa20dc08640 1 --2- 172.21.15.112:0/1433566425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa208100700 0x7fa208197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:38.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa20d407640 1 --2- 172.21.15.112:0/1433566425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2081018f0 0x7fa208198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:38.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa20e409640 1 -- 172.21.15.112:0/1433566425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2081018f0 msgr2=0x7fa208198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:38.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa20e409640 1 --2- 172.21.15.112:0/1433566425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2081018f0 0x7fa208198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa20e409640 1 -- 172.21.15.112:0/1433566425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa208100700 msgr2=0x7fa208197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:38.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa20e409640 1 --2- 172.21.15.112:0/1433566425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa208100700 0x7fa208197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa20e409640 1 -- 172.21.15.112:0/1433566425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20819f810 con 0x7fa2081022b0 2024-03-16T09:15:38.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa20e409640 1 --2- 172.21.15.112:0/1433566425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2081022b0 0x7fa20819f1a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa20000efd0 tx=0x7fa20000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:38.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa1feffd640 1 -- 172.21.15.112:0/1433566425 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200003b70 con 0x7fa2081022b0 2024-03-16T09:15:38.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa1feffd640 1 -- 172.21.15.112:0/1433566425 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa200003d10 con 0x7fa2081022b0 2024-03-16T09:15:38.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa20fe6c640 1 -- 172.21.15.112:0/1433566425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20819fb00 con 0x7fa2081022b0 2024-03-16T09:15:38.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.998+0000 7fa1feffd640 1 -- 172.21.15.112:0/1433566425 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200010a90 con 0x7fa2081022b0 2024-03-16T09:15:38.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.999+0000 7fa20fe6c640 1 -- 172.21.15.112:0/1433566425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa20819ffd0 con 0x7fa2081022b0 2024-03-16T09:15:38.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:37.999+0000 7fa20fe6c640 1 -- 172.21.15.112:0/1433566425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa208107aa0 con 0x7fa2081022b0 2024-03-16T09:15:38.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.001+0000 7fa1feffd640 1 -- 172.21.15.112:0/1433566425 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7fa200004b40 con 0x7fa2081022b0 2024-03-16T09:15:38.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.001+0000 7fa1feffd640 1 --2- 172.21.15.112:0/1433566425 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa1f40a2800 0x7fa1f40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:38.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.001+0000 7fa20dc08640 1 --2- 172.21.15.112:0/1433566425 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa1f40a2800 0x7fa1f40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:38.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.001+0000 7fa1feffd640 1 -- 172.21.15.112:0/1433566425 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa2000cf690 con 0x7fa2081022b0 2024-03-16T09:15:38.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.002+0000 7fa20dc08640 1 --2- 172.21.15.112:0/1433566425 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa1f40a2800 0x7fa1f40a4cc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa2040095f0 tx=0x7fa204005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:38.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.005+0000 7fa1feffd640 1 -- 172.21.15.112:0/1433566425 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa20009c740 con 0x7fa2081022b0 2024-03-16T09:15:38.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.181+0000 7fa20fe6c640 1 -- 172.21.15.112:0/1433566425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa20802d1b0 con 0x7fa2081022b0 2024-03-16T09:15:38.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.182+0000 7fa1feffd640 1 -- 172.21.15.112:0/1433566425 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fa20009c040 con 0x7fa2081022b0 2024-03-16T09:15:38.184 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-03-16T09:15:38.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.184+0000 7fa20fe6c640 1 -- 172.21.15.112:0/1433566425 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa1f40a2800 msgr2=0x7fa1f40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:38.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.184+0000 7fa20fe6c640 1 --2- 172.21.15.112:0/1433566425 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa1f40a2800 0x7fa1f40a4cc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa2040095f0 tx=0x7fa204005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:38.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.184+0000 7fa20fe6c640 1 -- 172.21.15.112:0/1433566425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2081022b0 msgr2=0x7fa20819f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:38.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.184+0000 7fa20fe6c640 1 --2- 172.21.15.112:0/1433566425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2081022b0 0x7fa20819f1a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa20000efd0 tx=0x7fa20000c6a0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.184+0000 7fa20fe6c640 1 -- 172.21.15.112:0/1433566425 shutdown_connections 2024-03-16T09:15:38.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.184+0000 7fa20fe6c640 1 --2- 172.21.15.112:0/1433566425 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa1f40a2800 0x7fa1f40a4cc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.184+0000 7fa20fe6c640 1 --2- 172.21.15.112:0/1433566425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2081022b0 0x7fa20819f1a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.184+0000 7fa20fe6c640 1 --2- 172.21.15.112:0/1433566425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2081018f0 0x7fa208198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.184+0000 7fa20fe6c640 1 --2- 172.21.15.112:0/1433566425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa208100700 0x7fa208197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.184+0000 7fa20fe6c640 1 -- 172.21.15.112:0/1433566425 >> 172.21.15.112:0/1433566425 conn(0x7fa2080fbc90 msgr2=0x7fa208104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.184+0000 7fa20fe6c640 1 -- 172.21.15.112:0/1433566425 shutdown_connections 2024-03-16T09:15:38.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.184+0000 7fa20fe6c640 1 -- 172.21.15.112:0/1433566425 wait complete. 2024-03-16T09:15:38.196 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:37 smithi112 ceph-mon[32307]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:15:38.196 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:37 smithi112 ceph-mon[32307]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:15:38.196 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:37 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1857454653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:38.196 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:37 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/4124757243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:38.196 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:37 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/562275246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:38.196 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:37 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/568234433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:38.197 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T09:15:38.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:37 smithi159 ceph-mon[36739]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:15:38.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:37 smithi159 ceph-mon[36739]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:15:38.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:37 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1857454653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:38.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:37 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/4124757243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:38.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:37 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/562275246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:38.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:37 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/568234433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:38.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.299+0000 7f67844e9640 1 -- 172.21.15.112:0/3063760624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f677c102290 msgr2=0x7f677c108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:38.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.299+0000 7f67844e9640 1 --2- 172.21.15.112:0/3063760624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f677c102290 0x7f677c108b20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6774009ca0 tx=0x7f677402c0f0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f67844e9640 1 -- 172.21.15.112:0/3063760624 shutdown_connections 2024-03-16T09:15:38.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f67844e9640 1 --2- 172.21.15.112:0/3063760624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f677c102290 0x7f677c108b20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f67844e9640 1 --2- 172.21.15.112:0/3063760624 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f677c1018d0 0x7f677c101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f67844e9640 1 --2- 172.21.15.112:0/3063760624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f677c1006e0 0x7f677c100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f67844e9640 1 -- 172.21.15.112:0/3063760624 >> 172.21.15.112:0/3063760624 conn(0x7f677c0fbc70 msgr2=0x7f677c0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f67844e9640 1 -- 172.21.15.112:0/3063760624 shutdown_connections 2024-03-16T09:15:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f67844e9640 1 -- 172.21.15.112:0/3063760624 wait complete. 2024-03-16T09:15:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f67844e9640 1 Processor -- start 2024-03-16T09:15:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f67844e9640 1 -- start start 2024-03-16T09:15:38.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f67844e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f677c1006e0 0x7f677c10b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:38.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f67844e9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f677c1018d0 0x7f677c1096a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:38.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f67844e9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f677c102290 0x7f677c109be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:38.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f67844e9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f677c10d5a0 con 0x7f677c1006e0 2024-03-16T09:15:38.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f67844e9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f677c10d420 con 0x7f677c102290 2024-03-16T09:15:38.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f67844e9640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f677c10d720 con 0x7f677c1018d0 2024-03-16T09:15:38.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f6782285640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f677c1006e0 0x7f677c10b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:38.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f6782285640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f677c1006e0 0x7f677c10b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57190/0 (socket says 172.21.15.112:57190) 2024-03-16T09:15:38.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f6782285640 1 -- 172.21.15.112:0/3014110900 learned_addr learned my addr 172.21.15.112:0/3014110900 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:38.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f6781a84640 1 --2- 172.21.15.112:0/3014110900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f677c1018d0 0x7f677c1096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:38.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.300+0000 7f6782a86640 1 --2- 172.21.15.112:0/3014110900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f677c102290 0x7f677c109be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:38.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.301+0000 7f6782285640 1 -- 172.21.15.112:0/3014110900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f677c1018d0 msgr2=0x7f677c1096a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:38.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.301+0000 7f6782285640 1 --2- 172.21.15.112:0/3014110900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f677c1018d0 0x7f677c1096a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.301+0000 7f6782285640 1 -- 172.21.15.112:0/3014110900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f677c102290 msgr2=0x7f677c109be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:38.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.301+0000 7f6782285640 1 --2- 172.21.15.112:0/3014110900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f677c102290 0x7f677c109be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.301+0000 7f6782285640 1 -- 172.21.15.112:0/3014110900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677c10a470 con 0x7f677c1006e0 2024-03-16T09:15:38.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.301+0000 7f6782285640 1 --2- 172.21.15.112:0/3014110900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f677c1006e0 0x7f677c10b060 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f677800e9c0 tx=0x7f677800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:38.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.301+0000 7f67737fe640 1 -- 172.21.15.112:0/3014110900 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677800cd80 con 0x7f677c1006e0 2024-03-16T09:15:38.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.301+0000 7f67844e9640 1 -- 172.21.15.112:0/3014110900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677c1a1d00 con 0x7f677c1006e0 2024-03-16T09:15:38.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.301+0000 7f67737fe640 1 -- 172.21.15.112:0/3014110900 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f67780035f0 con 0x7f677c1006e0 2024-03-16T09:15:38.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.301+0000 7f67737fe640 1 -- 172.21.15.112:0/3014110900 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778010690 con 0x7f677c1006e0 2024-03-16T09:15:38.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.301+0000 7f67844e9640 1 -- 172.21.15.112:0/3014110900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677c1a2190 con 0x7f677c1006e0 2024-03-16T09:15:38.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.302+0000 7f67844e9640 1 -- 172.21.15.112:0/3014110900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f677c1079a0 con 0x7f677c1006e0 2024-03-16T09:15:38.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.303+0000 7f67737fe640 1 -- 172.21.15.112:0/3014110900 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f67780037a0 con 0x7f677c1006e0 2024-03-16T09:15:38.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.304+0000 7f67737fe640 1 --2- 172.21.15.112:0/3014110900 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f67680a2800 0x7f67680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:38.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.304+0000 7f67737fe640 1 -- 172.21.15.112:0/3014110900 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f67780cee70 con 0x7f677c1006e0 2024-03-16T09:15:38.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.304+0000 7f6781a84640 1 --2- 172.21.15.112:0/3014110900 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f67680a2800 0x7f67680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:38.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.304+0000 7f6781a84640 1 --2- 172.21.15.112:0/3014110900 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f67680a2800 0x7f67680a4cc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f677c10add0 tx=0x7f676c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:38.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.308+0000 7f67737fe640 1 -- 172.21.15.112:0/3014110900 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f677809bf20 con 0x7f677c1006e0 2024-03-16T09:15:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:37 smithi194 ceph-mon[36729]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-16T09:15:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:37 smithi194 ceph-mon[36729]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:15:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:37 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1857454653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:37 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/4124757243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:37 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/562275246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:38.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:37 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/568234433' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:38.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.482+0000 7f67844e9640 1 -- 172.21.15.112:0/3014110900 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f677c02d1b0 con 0x7f677c1006e0 2024-03-16T09:15:38.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.482+0000 7f67737fe640 1 -- 172.21.15.112:0/3014110900 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f677c02d1b0 con 0x7f677c1006e0 2024-03-16T09:15:38.485 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-03-16T09:15:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.484+0000 7f67844e9640 1 -- 172.21.15.112:0/3014110900 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f67680a2800 msgr2=0x7f67680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.484+0000 7f67844e9640 1 --2- 172.21.15.112:0/3014110900 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f67680a2800 0x7f67680a4cc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f677c10add0 tx=0x7f676c005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.484+0000 7f67844e9640 1 -- 172.21.15.112:0/3014110900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f677c1006e0 msgr2=0x7f677c10b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.484+0000 7f67844e9640 1 --2- 172.21.15.112:0/3014110900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f677c1006e0 0x7f677c10b060 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f677800e9c0 tx=0x7f677800ee80 comp rx=0 tx=0).stop 2024-03-16T09:15:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.484+0000 7f67844e9640 1 -- 172.21.15.112:0/3014110900 shutdown_connections 2024-03-16T09:15:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.484+0000 7f67844e9640 1 --2- 172.21.15.112:0/3014110900 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f67680a2800 0x7f67680a4cc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.484+0000 7f67844e9640 1 --2- 172.21.15.112:0/3014110900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f677c102290 0x7f677c109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.484+0000 7f67844e9640 1 --2- 172.21.15.112:0/3014110900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f677c1018d0 0x7f677c1096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.484+0000 7f67844e9640 1 --2- 172.21.15.112:0/3014110900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f677c1006e0 0x7f677c10b060 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.484+0000 7f67844e9640 1 -- 172.21.15.112:0/3014110900 >> 172.21.15.112:0/3014110900 conn(0x7f677c0fbc70 msgr2=0x7f677c104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.484+0000 7f67844e9640 1 -- 172.21.15.112:0/3014110900 shutdown_connections 2024-03-16T09:15:38.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.485+0000 7f67844e9640 1 -- 172.21.15.112:0/3014110900 wait complete. 2024-03-16T09:15:38.497 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T09:15:38.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.600+0000 7f3eb52d3640 1 -- 172.21.15.112:0/1220267343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3eb01033a0 msgr2=0x7f3eb0109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:38.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.600+0000 7f3eb52d3640 1 --2- 172.21.15.112:0/1220267343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3eb01033a0 0x7f3eb0109c30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009be0 tx=0x7f3ea802bef0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.600+0000 7f3eb52d3640 1 -- 172.21.15.112:0/1220267343 shutdown_connections 2024-03-16T09:15:38.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.600+0000 7f3eb52d3640 1 --2- 172.21.15.112:0/1220267343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3eb01033a0 0x7f3eb0109c30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.600+0000 7f3eb52d3640 1 --2- 172.21.15.112:0/1220267343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb01029e0 0x7f3eb0102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.600+0000 7f3eb52d3640 1 --2- 172.21.15.112:0/1220267343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3eb01017f0 0x7f3eb0101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.600+0000 7f3eb52d3640 1 -- 172.21.15.112:0/1220267343 >> 172.21.15.112:0/1220267343 conn(0x7f3eb00fcd80 msgr2=0x7f3eb00ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:38.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.600+0000 7f3eb52d3640 1 -- 172.21.15.112:0/1220267343 shutdown_connections 2024-03-16T09:15:38.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.600+0000 7f3eb52d3640 1 -- 172.21.15.112:0/1220267343 wait complete. 2024-03-16T09:15:38.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.601+0000 7f3eb52d3640 1 Processor -- start 2024-03-16T09:15:38.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.601+0000 7f3eb52d3640 1 -- start start 2024-03-16T09:15:38.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.601+0000 7f3eb52d3640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3eb01017f0 0x7f3eb0075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:38.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.601+0000 7f3eb52d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb01029e0 0x7f3eb0073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:38.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.601+0000 7f3eb52d3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3eb01033a0 0x7f3eb0074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:38.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.601+0000 7f3eb52d3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb010d590 con 0x7f3eb01029e0 2024-03-16T09:15:38.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.601+0000 7f3eb52d3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb010d410 con 0x7f3eb01033a0 2024-03-16T09:15:38.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.601+0000 7f3eb52d3640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb010d710 con 0x7f3eb01017f0 2024-03-16T09:15:38.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.601+0000 7f3ea7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb01029e0 0x7f3eb0073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:38.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.601+0000 7f3ea7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb01029e0 0x7f3eb0073bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57208/0 (socket says 172.21.15.112:57208) 2024-03-16T09:15:38.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.601+0000 7f3ea7fff640 1 -- 172.21.15.112:0/3301782505 learned_addr learned my addr 172.21.15.112:0/3301782505 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:38.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.601+0000 7f3eaeffd640 1 --2- 172.21.15.112:0/3301782505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3eb01017f0 0x7f3eb0075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:38.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.601+0000 7f3eaf7fe640 1 --2- 172.21.15.112:0/3301782505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3eb01033a0 0x7f3eb0074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:38.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.601+0000 7f3ea7fff640 1 -- 172.21.15.112:0/3301782505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3eb01017f0 msgr2=0x7f3eb0075590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:38.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.601+0000 7f3ea7fff640 1 --2- 172.21.15.112:0/3301782505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3eb01017f0 0x7f3eb0075590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.601+0000 7f3ea7fff640 1 -- 172.21.15.112:0/3301782505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3eb01033a0 msgr2=0x7f3eb0074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:38.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.601+0000 7f3ea7fff640 1 --2- 172.21.15.112:0/3301782505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3eb01033a0 0x7f3eb0074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.601+0000 7f3ea7fff640 1 -- 172.21.15.112:0/3301782505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb00749a0 con 0x7f3eb01029e0 2024-03-16T09:15:38.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.601+0000 7f3ea7fff640 1 --2- 172.21.15.112:0/3301782505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb01029e0 0x7f3eb0073bd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800e9c0 tx=0x7f3e9800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:38.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.602+0000 7f3eacff9640 1 -- 172.21.15.112:0/3301782505 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800ccc0 con 0x7f3eb01029e0 2024-03-16T09:15:38.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.602+0000 7f3eacff9640 1 -- 172.21.15.112:0/3301782505 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3e98005520 con 0x7f3eb01029e0 2024-03-16T09:15:38.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.602+0000 7f3eb52d3640 1 -- 172.21.15.112:0/3301782505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb01a1c60 con 0x7f3eb01029e0 2024-03-16T09:15:38.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.602+0000 7f3eacff9640 1 -- 172.21.15.112:0/3301782505 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98010690 con 0x7f3eb01029e0 2024-03-16T09:15:38.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.602+0000 7f3eb52d3640 1 -- 172.21.15.112:0/3301782505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb01a21b0 con 0x7f3eb01029e0 2024-03-16T09:15:38.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.603+0000 7f3eb52d3640 1 -- 172.21.15.112:0/3301782505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb0108ab0 con 0x7f3eb01029e0 2024-03-16T09:15:38.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.604+0000 7f3eacff9640 1 -- 172.21.15.112:0/3301782505 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f3e980046c0 con 0x7f3eb01029e0 2024-03-16T09:15:38.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.604+0000 7f3eacff9640 1 --2- 172.21.15.112:0/3301782505 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3e9c0a2800 0x7f3e9c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:38.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.604+0000 7f3eaeffd640 1 --2- 172.21.15.112:0/3301782505 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3e9c0a2800 0x7f3e9c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:38.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.605+0000 7f3eacff9640 1 -- 172.21.15.112:0/3301782505 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f3e980ced00 con 0x7f3eb01029e0 2024-03-16T09:15:38.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.605+0000 7f3eaeffd640 1 --2- 172.21.15.112:0/3301782505 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3e9c0a2800 0x7f3e9c0a4cc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00095f0 tx=0x7f3ea0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:38.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.609+0000 7f3eacff9640 1 -- 172.21.15.112:0/3301782505 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3e9809bdb0 con 0x7f3eb01029e0 2024-03-16T09:15:38.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.781+0000 7f3eb52d3640 1 -- 172.21.15.112:0/3301782505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3eb0062480 con 0x7f3eb01029e0 2024-03-16T09:15:38.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.782+0000 7f3eacff9640 1 -- 172.21.15.112:0/3301782505 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f3e980bb030 con 0x7f3eb01029e0 2024-03-16T09:15:38.784 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-03-16T09:15:38.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.784+0000 7f3eb52d3640 1 -- 172.21.15.112:0/3301782505 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3e9c0a2800 msgr2=0x7f3e9c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:38.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.784+0000 7f3eb52d3640 1 --2- 172.21.15.112:0/3301782505 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3e9c0a2800 0x7f3e9c0a4cc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00095f0 tx=0x7f3ea0005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:38.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.784+0000 7f3eb52d3640 1 -- 172.21.15.112:0/3301782505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb01029e0 msgr2=0x7f3eb0073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:38.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.784+0000 7f3eb52d3640 1 --2- 172.21.15.112:0/3301782505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb01029e0 0x7f3eb0073bd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800e9c0 tx=0x7f3e9800ee80 comp rx=0 tx=0).stop 2024-03-16T09:15:38.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.784+0000 7f3eb52d3640 1 -- 172.21.15.112:0/3301782505 shutdown_connections 2024-03-16T09:15:38.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.784+0000 7f3eb52d3640 1 --2- 172.21.15.112:0/3301782505 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3e9c0a2800 0x7f3e9c0a4cc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.784+0000 7f3eb52d3640 1 --2- 172.21.15.112:0/3301782505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3eb01033a0 0x7f3eb0074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.784+0000 7f3eb52d3640 1 --2- 172.21.15.112:0/3301782505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb01029e0 0x7f3eb0073bd0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.784+0000 7f3eb52d3640 1 --2- 172.21.15.112:0/3301782505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3eb01017f0 0x7f3eb0075590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.784+0000 7f3eb52d3640 1 -- 172.21.15.112:0/3301782505 >> 172.21.15.112:0/3301782505 conn(0x7f3eb00fcd80 msgr2=0x7f3eb0105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:38.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.784+0000 7f3eb52d3640 1 -- 172.21.15.112:0/3301782505 shutdown_connections 2024-03-16T09:15:38.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.784+0000 7f3eb52d3640 1 -- 172.21.15.112:0/3301782505 wait complete. 2024-03-16T09:15:38.797 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-16T09:15:38.797 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:15:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.896+0000 7f308abf4640 1 -- 172.21.15.112:0/3457320939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3084107540 msgr2=0x7f3084109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.896+0000 7f308abf4640 1 --2- 172.21.15.112:0/3457320939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3084107540 0x7f3084109930 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f307c009ca0 tx=0x7f307c02c0f0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f308abf4640 1 -- 172.21.15.112:0/3457320939 shutdown_connections 2024-03-16T09:15:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f308abf4640 1 --2- 172.21.15.112:0/3457320939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3084107540 0x7f3084109930 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f308abf4640 1 --2- 172.21.15.112:0/3457320939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3084104c10 0x7f3084107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f308abf4640 1 --2- 172.21.15.112:0/3457320939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30841022e0 0x7f30841046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f308abf4640 1 -- 172.21.15.112:0/3457320939 >> 172.21.15.112:0/3457320939 conn(0x7f30840fbc70 msgr2=0x7f30840fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f308abf4640 1 -- 172.21.15.112:0/3457320939 shutdown_connections 2024-03-16T09:15:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f308abf4640 1 -- 172.21.15.112:0/3457320939 wait complete. 2024-03-16T09:15:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f308abf4640 1 Processor -- start 2024-03-16T09:15:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f308abf4640 1 -- start start 2024-03-16T09:15:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f308abf4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30841022e0 0x7f3084197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f308abf4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3084104c10 0x7f30841980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f308abf4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3084107540 0x7f308419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f308abf4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f308410d5b0 con 0x7f30841022e0 2024-03-16T09:15:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f308abf4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f308410d430 con 0x7f3084107540 2024-03-16T09:15:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f308abf4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f308410d730 con 0x7f3084104c10 2024-03-16T09:15:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f3088990640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30841022e0 0x7f3084197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f3088990640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30841022e0 0x7f3084197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57218/0 (socket says 172.21.15.112:57218) 2024-03-16T09:15:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f3088990640 1 -- 172.21.15.112:0/2979006223 learned_addr learned my addr 172.21.15.112:0/2979006223 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f3083fff640 1 --2- 172.21.15.112:0/2979006223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3084104c10 0x7f30841980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.897+0000 7f3089191640 1 --2- 172.21.15.112:0/2979006223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3084107540 0x7f308419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.898+0000 7f3088990640 1 -- 172.21.15.112:0/2979006223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3084104c10 msgr2=0x7f30841980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.898+0000 7f3088990640 1 --2- 172.21.15.112:0/2979006223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3084104c10 0x7f30841980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.898+0000 7f3088990640 1 -- 172.21.15.112:0/2979006223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3084107540 msgr2=0x7f308419f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.898+0000 7f3088990640 1 --2- 172.21.15.112:0/2979006223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3084107540 0x7f308419f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.898+0000 7f3088990640 1 -- 172.21.15.112:0/2979006223 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308419f7b0 con 0x7f30841022e0 2024-03-16T09:15:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.898+0000 7f3088990640 1 --2- 172.21.15.112:0/2979006223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30841022e0 0x7f3084197b90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f307400ea60 tx=0x7f307400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.898+0000 7f3081ffb640 1 -- 172.21.15.112:0/2979006223 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307400cde0 con 0x7f30841022e0 2024-03-16T09:15:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.898+0000 7f3081ffb640 1 -- 172.21.15.112:0/2979006223 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f30740035f0 con 0x7f30841022e0 2024-03-16T09:15:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.898+0000 7f308abf4640 1 -- 172.21.15.112:0/2979006223 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f308419faa0 con 0x7f30841022e0 2024-03-16T09:15:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.898+0000 7f3081ffb640 1 -- 172.21.15.112:0/2979006223 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3074010720 con 0x7f30841022e0 2024-03-16T09:15:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.898+0000 7f308abf4640 1 -- 172.21.15.112:0/2979006223 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f308419fff0 con 0x7f30841022e0 2024-03-16T09:15:38.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.899+0000 7f308abf4640 1 -- 172.21.15.112:0/2979006223 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30840fd760 con 0x7f30841022e0 2024-03-16T09:15:38.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.900+0000 7f3081ffb640 1 -- 172.21.15.112:0/2979006223 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137291+0+0 (secure 0 0 0) 0x7f30740037a0 con 0x7f30841022e0 2024-03-16T09:15:38.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.900+0000 7f3081ffb640 1 --2- 172.21.15.112:0/2979006223 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f306c0a2800 0x7f306c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:38.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.901+0000 7f3083fff640 1 --2- 172.21.15.112:0/2979006223 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f306c0a2800 0x7f306c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:38.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.901+0000 7f3081ffb640 1 -- 172.21.15.112:0/2979006223 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f30740cee30 con 0x7f30841022e0 2024-03-16T09:15:38.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.901+0000 7f3083fff640 1 --2- 172.21.15.112:0/2979006223 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f306c0a2800 0x7f306c0a4cc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f30841990a0 tx=0x7f3078005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:38.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.905+0000 7f3081ffb640 1 -- 172.21.15.112:0/2979006223 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f307409bee0 con 0x7f30841022e0 2024-03-16T09:15:38.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:38.986+0000 7f3081ffb640 1 -- 172.21.15.112:0/2979006223 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7f3074097e50 con 0x7f30841022e0 2024-03-16T09:15:39.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:38 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1433566425' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:39.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:38 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3014110900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:39.092 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:38 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3301782505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:39.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.127+0000 7f308abf4640 1 -- 172.21.15.112:0/2979006223 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30840623e0 con 0x7f30841022e0 2024-03-16T09:15:39.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.128+0000 7f3081ffb640 1 -- 172.21.15.112:0/2979006223 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11881 (secure 0 0 0) 0x7f3074064ea0 con 0x7f30841022e0 2024-03-16T09:15:39.130 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:39.130 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3939748854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3939748854},{"type":"v1","addr":"172.21.15.112:6839","nonce":3939748854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3118523258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3118523258},{"type":"v1","addr":"172.21.15.112:6841","nonce":3118523258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T09:13:36.450216+0000","modified":"2024-03-16T09:15:34.982470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/1682076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":1682076},{"type":"v1","addr":"172.21.15.194:6837","nonce":1682076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.194:6839/2528943309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2528943309},{"type":"v1","addr":"172.21.15.194:6839","nonce":2528943309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/272895965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":272895965},{"type":"v1","addr":"172.21.15.159:6839","nonce":272895965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24452,24466]},"id":1}]} 2024-03-16T09:15:39.131 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 22 2024-03-16T09:15:39.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.130+0000 7f308abf4640 1 -- 172.21.15.112:0/2979006223 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f306c0a2800 msgr2=0x7f306c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:39.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.130+0000 7f308abf4640 1 --2- 172.21.15.112:0/2979006223 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f306c0a2800 0x7f306c0a4cc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f30841990a0 tx=0x7f3078005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:39.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.130+0000 7f308abf4640 1 -- 172.21.15.112:0/2979006223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30841022e0 msgr2=0x7f3084197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:39.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.130+0000 7f308abf4640 1 --2- 172.21.15.112:0/2979006223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30841022e0 0x7f3084197b90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f307400ea60 tx=0x7f307400ef20 comp rx=0 tx=0).stop 2024-03-16T09:15:39.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.130+0000 7f308abf4640 1 -- 172.21.15.112:0/2979006223 shutdown_connections 2024-03-16T09:15:39.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.130+0000 7f308abf4640 1 --2- 172.21.15.112:0/2979006223 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f306c0a2800 0x7f306c0a4cc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.130+0000 7f308abf4640 1 --2- 172.21.15.112:0/2979006223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3084107540 0x7f308419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.130+0000 7f308abf4640 1 --2- 172.21.15.112:0/2979006223 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3084104c10 0x7f30841980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.130+0000 7f308abf4640 1 --2- 172.21.15.112:0/2979006223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30841022e0 0x7f3084197b90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.130+0000 7f308abf4640 1 -- 172.21.15.112:0/2979006223 >> 172.21.15.112:0/2979006223 conn(0x7f30840fbc70 msgr2=0x7f3084102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:39.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.130+0000 7f308abf4640 1 -- 172.21.15.112:0/2979006223 shutdown_connections 2024-03-16T09:15:39.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.130+0000 7f308abf4640 1 -- 172.21.15.112:0/2979006223 wait complete. 2024-03-16T09:15:39.144 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-16T09:15:39.144 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-16T09:15:39.144 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T09:15:39.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:38 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1433566425' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:39.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:38 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3014110900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:39.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:38 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3301782505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:39.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.242+0000 7efc045b9640 1 -- 172.21.15.112:0/3109877641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbfc1074e0 msgr2=0x7efbfc109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:39.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.242+0000 7efc045b9640 1 --2- 172.21.15.112:0/3109877641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbfc1074e0 0x7efbfc109930 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009ca0 tx=0x7efbf402c0f0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc045b9640 1 -- 172.21.15.112:0/3109877641 shutdown_connections 2024-03-16T09:15:39.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc045b9640 1 --2- 172.21.15.112:0/3109877641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbfc1074e0 0x7efbfc109930 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc045b9640 1 --2- 172.21.15.112:0/3109877641 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efbfc104bb0 0x7efbfc106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc045b9640 1 --2- 172.21.15.112:0/3109877641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efbfc102280 0x7efbfc104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc045b9640 1 -- 172.21.15.112:0/3109877641 >> 172.21.15.112:0/3109877641 conn(0x7efbfc0fbc70 msgr2=0x7efbfc0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:39.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc045b9640 1 -- 172.21.15.112:0/3109877641 shutdown_connections 2024-03-16T09:15:39.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc045b9640 1 -- 172.21.15.112:0/3109877641 wait complete. 2024-03-16T09:15:39.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc045b9640 1 Processor -- start 2024-03-16T09:15:39.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc045b9640 1 -- start start 2024-03-16T09:15:39.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc045b9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efbfc102280 0x7efbfc197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:39.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc045b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbfc104bb0 0x7efbfc1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:39.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc045b9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efbfc1074e0 0x7efbfc19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:39.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc045b9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc10d5d0 con 0x7efbfc104bb0 2024-03-16T09:15:39.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc045b9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc10d450 con 0x7efbfc1074e0 2024-03-16T09:15:39.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc045b9640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc10d750 con 0x7efbfc102280 2024-03-16T09:15:39.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc01b54640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbfc104bb0 0x7efbfc1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:39.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc01b54640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbfc104bb0 0x7efbfc1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57232/0 (socket says 172.21.15.112:57232) 2024-03-16T09:15:39.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc01b54640 1 -- 172.21.15.112:0/4015992634 learned_addr learned my addr 172.21.15.112:0/4015992634 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:39.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc02355640 1 --2- 172.21.15.112:0/4015992634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efbfc102280 0x7efbfc197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:39.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.243+0000 7efc02b56640 1 --2- 172.21.15.112:0/4015992634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efbfc1074e0 0x7efbfc19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:39.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.244+0000 7efc01b54640 1 -- 172.21.15.112:0/4015992634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efbfc102280 msgr2=0x7efbfc197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:39.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.244+0000 7efc01b54640 1 --2- 172.21.15.112:0/4015992634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efbfc102280 0x7efbfc197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.244+0000 7efc01b54640 1 -- 172.21.15.112:0/4015992634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efbfc1074e0 msgr2=0x7efbfc19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:39.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.244+0000 7efc01b54640 1 --2- 172.21.15.112:0/4015992634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efbfc1074e0 0x7efbfc19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.244+0000 7efc01b54640 1 -- 172.21.15.112:0/4015992634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbfc19f7b0 con 0x7efbfc104bb0 2024-03-16T09:15:39.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.244+0000 7efc01b54640 1 --2- 172.21.15.112:0/4015992634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbfc104bb0 0x7efbfc1980d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efbf000e9f0 tx=0x7efbf000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:39.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.244+0000 7efbfb7fe640 1 -- 172.21.15.112:0/4015992634 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf000ccc0 con 0x7efbfc104bb0 2024-03-16T09:15:39.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.244+0000 7efc045b9640 1 -- 172.21.15.112:0/4015992634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbfc19faa0 con 0x7efbfc104bb0 2024-03-16T09:15:39.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.244+0000 7efbfb7fe640 1 -- 172.21.15.112:0/4015992634 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efbf0005520 con 0x7efbfc104bb0 2024-03-16T09:15:39.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.244+0000 7efbfb7fe640 1 -- 172.21.15.112:0/4015992634 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf0010690 con 0x7efbfc104bb0 2024-03-16T09:15:39.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.244+0000 7efc045b9640 1 -- 172.21.15.112:0/4015992634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbfc19fff0 con 0x7efbfc104bb0 2024-03-16T09:15:39.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.245+0000 7efc045b9640 1 -- 172.21.15.112:0/4015992634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbfc0fd760 con 0x7efbfc104bb0 2024-03-16T09:15:39.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.246+0000 7efbfb7fe640 1 -- 172.21.15.112:0/4015992634 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7efbf0004f70 con 0x7efbfc104bb0 2024-03-16T09:15:39.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.246+0000 7efbfb7fe640 1 --2- 172.21.15.112:0/4015992634 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efbe40a2890 0x7efbe40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:39.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.247+0000 7efc02355640 1 --2- 172.21.15.112:0/4015992634 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efbe40a2890 0x7efbe40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:39.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.247+0000 7efbfb7fe640 1 -- 172.21.15.112:0/4015992634 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7efbf00cef80 con 0x7efbfc104bb0 2024-03-16T09:15:39.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.247+0000 7efc02355640 1 --2- 172.21.15.112:0/4015992634 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efbe40a2890 0x7efbe40a4d50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efbec009bd0 tx=0x7efbec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:39.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.251+0000 7efbfb7fe640 1 -- 172.21.15.112:0/4015992634 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efbf009bed0 con 0x7efbfc104bb0 2024-03-16T09:15:39.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:38 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1433566425' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:39.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:38 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3014110900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:39.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:38 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3301782505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:39.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.422+0000 7efc045b9640 1 -- 172.21.15.112:0/4015992634 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efbfc061c20 con 0x7efbe40a2890 2024-03-16T09:15:39.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.430+0000 7efbfb7fe640 1 -- 172.21.15.112:0/4015992634 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259060 (secure 0 0 0) 0x7efbfc061c20 con 0x7efbe40a2890 2024-03-16T09:15:39.435 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:39.437 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-03-16T09:15:39.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.433+0000 7efc045b9640 1 -- 172.21.15.112:0/4015992634 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efbe40a2890 msgr2=0x7efbe40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:39.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.433+0000 7efc045b9640 1 --2- 172.21.15.112:0/4015992634 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efbe40a2890 0x7efbe40a4d50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efbec009bd0 tx=0x7efbec005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:39.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.433+0000 7efc045b9640 1 -- 172.21.15.112:0/4015992634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbfc104bb0 msgr2=0x7efbfc1980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:39.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.433+0000 7efc045b9640 1 --2- 172.21.15.112:0/4015992634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbfc104bb0 0x7efbfc1980d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efbf000e9f0 tx=0x7efbf000eeb0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.434+0000 7efc045b9640 1 -- 172.21.15.112:0/4015992634 shutdown_connections 2024-03-16T09:15:39.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.434+0000 7efc045b9640 1 --2- 172.21.15.112:0/4015992634 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efbe40a2890 0x7efbe40a4d50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.434+0000 7efc045b9640 1 --2- 172.21.15.112:0/4015992634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efbfc1074e0 0x7efbfc19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.434+0000 7efc045b9640 1 --2- 172.21.15.112:0/4015992634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efbfc104bb0 0x7efbfc1980d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.434+0000 7efc045b9640 1 --2- 172.21.15.112:0/4015992634 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efbfc102280 0x7efbfc197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.434+0000 7efc045b9640 1 -- 172.21.15.112:0/4015992634 >> 172.21.15.112:0/4015992634 conn(0x7efbfc0fbc70 msgr2=0x7efbfc102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:39.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.434+0000 7efc045b9640 1 -- 172.21.15.112:0/4015992634 shutdown_connections 2024-03-16T09:15:39.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.434+0000 7efc045b9640 1 -- 172.21.15.112:0/4015992634 wait complete. 2024-03-16T09:15:39.449 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-03-16T09:15:38.906882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654625,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":854,"num_read_kb":540,"num_write":348,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181900,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":781,"ondisk_log_size":781,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352404,"kb_used_data":11572,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589420,"statfs":{"total":1151940427776,"available":1151579566080,"internally_reserved":0,"allocated":11849728,"data_stored":7447041,"data_compressed":112689,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":699,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":37,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002575"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.923673+0000","last_change":"2024-03-16T09:13:37.119969+0000","last_active":"2024-03-16T09:15:00.923673+0000","last_peered":"2024-03-16T09:15:00.923673+0000","last_clean":"2024-03-16T09:15:00.923673+0000","last_became_active":"2024-03-16T09:13:37.119177+0000","last_became_peered":"2024-03-16T09:13:37.119177+0000","last_unstale":"2024-03-16T09:15:00.923673+0000","last_undegraded":"2024-03-16T09:15:00.923673+0000","last_fullsized":"2024-03-16T09:15:00.923673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:25:28.038898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.919826+0000","last_change":"2024-03-16T09:13:37.127564+0000","last_active":"2024-03-16T09:15:00.919826+0000","last_peered":"2024-03-16T09:15:00.919826+0000","last_clean":"2024-03-16T09:15:00.919826+0000","last_became_active":"2024-03-16T09:13:37.127214+0000","last_became_peered":"2024-03-16T09:13:37.127214+0000","last_unstale":"2024-03-16T09:15:00.919826+0000","last_undegraded":"2024-03-16T09:15:00.919826+0000","last_fullsized":"2024-03-16T09:15:00.919826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:46:14.655707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.933238+0000","last_change":"2024-03-16T09:13:37.126288+0000","last_active":"2024-03-16T09:15:00.933238+0000","last_peered":"2024-03-16T09:15:00.933238+0000","last_clean":"2024-03-16T09:15:00.933238+0000","last_became_active":"2024-03-16T09:13:37.125383+0000","last_became_peered":"2024-03-16T09:13:37.125383+0000","last_unstale":"2024-03-16T09:15:00.933238+0000","last_undegraded":"2024-03-16T09:15:00.933238+0000","last_fullsized":"2024-03-16T09:15:00.933238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:47:09.901508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.919434+0000","last_change":"2024-03-16T09:13:37.125503+0000","last_active":"2024-03-16T09:15:00.919434+0000","last_peered":"2024-03-16T09:15:00.919434+0000","last_clean":"2024-03-16T09:15:00.919434+0000","last_became_active":"2024-03-16T09:13:37.125144+0000","last_became_peered":"2024-03-16T09:13:37.125144+0000","last_unstale":"2024-03-16T09:15:00.919434+0000","last_undegraded":"2024-03-16T09:15:00.919434+0000","last_fullsized":"2024-03-16T09:15:00.919434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:30:26.529080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.900877+0000","last_change":"2024-03-16T09:13:37.114237+0000","last_active":"2024-03-16T09:15:00.900877+0000","last_peered":"2024-03-16T09:15:00.900877+0000","last_clean":"2024-03-16T09:15:00.900877+0000","last_became_active":"2024-03-16T09:13:37.113595+0000","last_became_peered":"2024-03-16T09:13:37.113595+0000","last_unstale":"2024-03-16T09:15:00.900877+0000","last_undegraded":"2024-03-16T09:15:00.900877+0000","last_fullsized":"2024-03-16T09:15:00.900877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:06:17.735301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.914380+0000","last_change":"2024-03-16T09:13:37.118843+0000","last_active":"2024-03-16T09:15:00.914380+0000","last_peered":"2024-03-16T09:15:00.914380+0000","last_clean":"2024-03-16T09:15:00.914380+0000","last_became_active":"2024-03-16T09:13:37.117086+0000","last_became_peered":"2024-03-16T09:13:37.117086+0000","last_unstale":"2024-03-16T09:15:00.914380+0000","last_undegraded":"2024-03-16T09:15:00.914380+0000","last_fullsized":"2024-03-16T09:15:00.914380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:23:12.929931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.916488+0000","last_change":"2024-03-16T09:13:37.133427+0000","last_active":"2024-03-16T09:15:00.916488+0000","last_peered":"2024-03-16T09:15:00.916488+0000","last_clean":"2024-03-16T09:15:00.916488+0000","last_became_active":"2024-03-16T09:13:37.133032+0000","last_became_peered":"2024-03-16T09:13:37.133032+0000","last_unstale":"2024-03-16T09:15:00.916488+0000","last_undegraded":"2024-03-16T09:15:00.916488+0000","last_fullsized":"2024-03-16T09:15:00.916488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:49:28.179883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.920627+0000","last_change":"2024-03-16T09:13:37.132132+0000","last_active":"2024-03-16T09:15:00.920627+0000","last_peered":"2024-03-16T09:15:00.920627+0000","last_clean":"2024-03-16T09:15:00.920627+0000","last_became_active":"2024-03-16T09:13:37.131801+0000","last_became_peered":"2024-03-16T09:13:37.131801+0000","last_unstale":"2024-03-16T09:15:00.920627+0000","last_undegraded":"2024-03-16T09:15:00.920627+0000","last_fullsized":"2024-03-16T09:15:00.920627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:45:03.891725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.930016+0000","last_change":"2024-03-16T09:13:37.116378+0000","last_active":"2024-03-16T09:15:00.930016+0000","last_peered":"2024-03-16T09:15:00.930016+0000","last_clean":"2024-03-16T09:15:00.930016+0000","last_became_active":"2024-03-16T09:13:37.116027+0000","last_became_peered":"2024-03-16T09:13:37.116027+0000","last_unstale":"2024-03-16T09:15:00.930016+0000","last_undegraded":"2024-03-16T09:15:00.930016+0000","last_fullsized":"2024-03-16T09:15:00.930016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:14:38.681613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.919236+0000","last_change":"2024-03-16T09:13:37.121417+0000","last_active":"2024-03-16T09:15:00.919236+0000","last_peered":"2024-03-16T09:15:00.919236+0000","last_clean":"2024-03-16T09:15:00.919236+0000","last_became_active":"2024-03-16T09:13:37.121112+0000","last_became_peered":"2024-03-16T09:13:37.121112+0000","last_unstale":"2024-03-16T09:15:00.919236+0000","last_undegraded":"2024-03-16T09:15:00.919236+0000","last_fullsized":"2024-03-16T09:15:00.919236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:38:07.823604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.935183+0000","last_change":"2024-03-16T09:13:37.121368+0000","last_active":"2024-03-16T09:15:00.935183+0000","last_peered":"2024-03-16T09:15:00.935183+0000","last_clean":"2024-03-16T09:15:00.935183+0000","last_became_active":"2024-03-16T09:13:37.121060+0000","last_became_peered":"2024-03-16T09:13:37.121060+0000","last_unstale":"2024-03-16T09:15:00.935183+0000","last_undegraded":"2024-03-16T09:15:00.935183+0000","last_fullsized":"2024-03-16T09:15:00.935183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:48:49.168285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.903523+0000","last_change":"2024-03-16T09:13:37.127004+0000","last_active":"2024-03-16T09:15:00.903523+0000","last_peered":"2024-03-16T09:15:00.903523+0000","last_clean":"2024-03-16T09:15:00.903523+0000","last_became_active":"2024-03-16T09:13:37.126579+0000","last_became_peered":"2024-03-16T09:13:37.126579+0000","last_unstale":"2024-03-16T09:15:00.903523+0000","last_undegraded":"2024-03-16T09:15:00.903523+0000","last_fullsized":"2024-03-16T09:15:00.903523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:54:18.382514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.918879+0000","last_change":"2024-03-16T09:13:37.104534+0000","last_active":"2024-03-16T09:15:00.918879+0000","last_peered":"2024-03-16T09:15:00.918879+0000","last_clean":"2024-03-16T09:15:00.918879+0000","last_became_active":"2024-03-16T09:13:37.104202+0000","last_became_peered":"2024-03-16T09:13:37.104202+0000","last_unstale":"2024-03-16T09:15:00.918879+0000","last_undegraded":"2024-03-16T09:15:00.918879+0000","last_fullsized":"2024-03-16T09:15:00.918879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:40:55.448238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.920211+0000","last_change":"2024-03-16T09:13:37.117166+0000","last_active":"2024-03-16T09:15:00.920211+0000","last_peered":"2024-03-16T09:15:00.920211+0000","last_clean":"2024-03-16T09:15:00.920211+0000","last_became_active":"2024-03-16T09:13:37.116747+0000","last_became_peered":"2024-03-16T09:13:37.116747+0000","last_unstale":"2024-03-16T09:15:00.920211+0000","last_undegraded":"2024-03-16T09:15:00.920211+0000","last_fullsized":"2024-03-16T09:15:00.920211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:50:58.705683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.921643+0000","last_change":"2024-03-16T09:14:32.000383+0000","last_active":"2024-03-16T09:15:00.921643+0000","last_peered":"2024-03-16T09:15:00.921643+0000","last_clean":"2024-03-16T09:15:00.921643+0000","last_became_active":"2024-03-16T09:14:32.000027+0000","last_became_peered":"2024-03-16T09:14:32.000027+0000","last_unstale":"2024-03-16T09:15:00.921643+0000","last_undegraded":"2024-03-16T09:15:00.921643+0000","last_fullsized":"2024-03-16T09:15:00.921643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:04:03.478849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.934752+0000","last_change":"2024-03-16T09:13:37.120583+0000","last_active":"2024-03-16T09:15:00.934752+0000","last_peered":"2024-03-16T09:15:00.934752+0000","last_clean":"2024-03-16T09:15:00.934752+0000","last_became_active":"2024-03-16T09:13:37.120217+0000","last_became_peered":"2024-03-16T09:13:37.120217+0000","last_unstale":"2024-03-16T09:15:00.934752+0000","last_undegraded":"2024-03-16T09:15:00.934752+0000","last_fullsized":"2024-03-16T09:15:00.934752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:00:17.653736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.928060+0000","last_change":"2024-03-16T09:13:37.121179+0000","last_active":"2024-03-16T09:15:00.928060+0000","last_peered":"2024-03-16T09:15:00.928060+0000","last_clean":"2024-03-16T09:15:00.928060+0000","last_became_active":"2024-03-16T09:13:37.120070+0000","last_became_peered":"2024-03-16T09:13:37.120070+0000","last_unstale":"2024-03-16T09:15:00.928060+0000","last_undegraded":"2024-03-16T09:15:00.928060+0000","last_fullsized":"2024-03-16T09:15:00.928060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:39:24.937068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.935539+0000","last_change":"2024-03-16T09:13:37.110074+0000","last_active":"2024-03-16T09:15:00.935539+0000","last_peered":"2024-03-16T09:15:00.935539+0000","last_clean":"2024-03-16T09:15:00.935539+0000","last_became_active":"2024-03-16T09:13:37.109767+0000","last_became_peered":"2024-03-16T09:13:37.109767+0000","last_unstale":"2024-03-16T09:15:00.935539+0000","last_undegraded":"2024-03-16T09:15:00.935539+0000","last_fullsized":"2024-03-16T09:15:00.935539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:20:37.408262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.911978+0000","last_change":"2024-03-16T09:13:37.130791+0000","last_active":"2024-03-16T09:15:00.911978+0000","last_peered":"2024-03-16T09:15:00.911978+0000","last_clean":"2024-03-16T09:15:00.911978+0000","last_became_active":"2024-03-16T09:13:37.130428+0000","last_became_peered":"2024-03-16T09:13:37.130428+0000","last_unstale":"2024-03-16T09:15:00.911978+0000","last_undegraded":"2024-03-16T09:15:00.911978+0000","last_fullsized":"2024-03-16T09:15:00.911978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:32:33.185589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.006081+0000","last_change":"2024-03-16T09:14:32.006081+0000","last_active":"2024-03-16T09:14:32.006081+0000","last_peered":"2024-03-16T09:14:32.006081+0000","last_clean":"2024-03-16T09:14:32.006081+0000","last_became_active":"2024-03-16T09:14:32.000165+0000","last_became_peered":"2024-03-16T09:14:32.000165+0000","last_unstale":"2024-03-16T09:14:32.006081+0000","last_undegraded":"2024-03-16T09:14:32.006081+0000","last_fullsized":"2024-03-16T09:14:32.006081+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:12:54.685988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.918321+0000","last_change":"2024-03-16T09:13:37.124946+0000","last_active":"2024-03-16T09:15:00.918321+0000","last_peered":"2024-03-16T09:15:00.918321+0000","last_clean":"2024-03-16T09:15:00.918321+0000","last_became_active":"2024-03-16T09:13:37.124624+0000","last_became_peered":"2024-03-16T09:13:37.124624+0000","last_unstale":"2024-03-16T09:15:00.918321+0000","last_undegraded":"2024-03-16T09:15:00.918321+0000","last_fullsized":"2024-03-16T09:15:00.918321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:18:15.188823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517217+0000","last_change":"2024-03-16T09:13:36.116312+0000","last_active":"2024-03-16T09:14:32.517217+0000","last_peered":"2024-03-16T09:14:32.517217+0000","last_clean":"2024-03-16T09:14:32.517217+0000","last_became_active":"2024-03-16T09:13:36.115999+0000","last_became_peered":"2024-03-16T09:13:36.115999+0000","last_unstale":"2024-03-16T09:14:32.517217+0000","last_undegraded":"2024-03-16T09:14:32.517217+0000","last_fullsized":"2024-03-16T09:14:32.517217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:04:46.795291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.933147+0000","last_change":"2024-03-16T09:13:37.126024+0000","last_active":"2024-03-16T09:15:00.933147+0000","last_peered":"2024-03-16T09:15:00.933147+0000","last_clean":"2024-03-16T09:15:00.933147+0000","last_became_active":"2024-03-16T09:13:37.125722+0000","last_became_peered":"2024-03-16T09:13:37.125722+0000","last_unstale":"2024-03-16T09:15:00.933147+0000","last_undegraded":"2024-03-16T09:15:00.933147+0000","last_fullsized":"2024-03-16T09:15:00.933147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:27:24.923022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.005879+0000","last_change":"2024-03-16T09:14:32.005295+0000","last_active":"2024-03-16T09:14:32.005879+0000","last_peered":"2024-03-16T09:14:32.005879+0000","last_clean":"2024-03-16T09:14:32.005879+0000","last_became_active":"2024-03-16T09:14:31.999981+0000","last_became_peered":"2024-03-16T09:14:31.999981+0000","last_unstale":"2024-03-16T09:14:32.005879+0000","last_undegraded":"2024-03-16T09:14:32.005879+0000","last_fullsized":"2024-03-16T09:14:32.005879+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:13:32.661689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.932390+0000","last_change":"2024-03-16T09:13:37.127593+0000","last_active":"2024-03-16T09:15:00.932390+0000","last_peered":"2024-03-16T09:15:00.932390+0000","last_clean":"2024-03-16T09:15:00.932390+0000","last_became_active":"2024-03-16T09:13:37.127255+0000","last_became_peered":"2024-03-16T09:13:37.127255+0000","last_unstale":"2024-03-16T09:15:00.932390+0000","last_undegraded":"2024-03-16T09:15:00.932390+0000","last_fullsized":"2024-03-16T09:15:00.932390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:07:51.263266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.995143+0000","last_change":"2024-03-16T09:13:36.117583+0000","last_active":"2024-03-16T09:14:31.995143+0000","last_peered":"2024-03-16T09:14:31.995143+0000","last_clean":"2024-03-16T09:14:31.995143+0000","last_became_active":"2024-03-16T09:13:36.117270+0000","last_became_peered":"2024-03-16T09:13:36.117270+0000","last_unstale":"2024-03-16T09:14:31.995143+0000","last_undegraded":"2024-03-16T09:14:31.995143+0000","last_fullsized":"2024-03-16T09:14:31.995143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:10:53.541872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.924161+0000","last_change":"2024-03-16T09:13:37.119019+0000","last_active":"2024-03-16T09:15:00.924161+0000","last_peered":"2024-03-16T09:15:00.924161+0000","last_clean":"2024-03-16T09:15:00.924161+0000","last_became_active":"2024-03-16T09:13:37.118148+0000","last_became_peered":"2024-03-16T09:13:37.118148+0000","last_unstale":"2024-03-16T09:15:00.924161+0000","last_undegraded":"2024-03-16T09:15:00.924161+0000","last_fullsized":"2024-03-16T09:15:00.924161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:17:43.042779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":143,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:35.446937+0000","last_change":"2024-03-16T09:13:36.114415+0000","last_active":"2024-03-16T09:15:35.446937+0000","last_peered":"2024-03-16T09:15:35.446937+0000","last_clean":"2024-03-16T09:15:35.446937+0000","last_became_active":"2024-03-16T09:13:36.114056+0000","last_became_peered":"2024-03-16T09:13:36.114056+0000","last_unstale":"2024-03-16T09:15:35.446937+0000","last_undegraded":"2024-03-16T09:15:35.446937+0000","last_fullsized":"2024-03-16T09:15:35.446937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:50:28.322627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.904292+0000","last_change":"2024-03-16T09:13:37.124997+0000","last_active":"2024-03-16T09:15:00.904292+0000","last_peered":"2024-03-16T09:15:00.904292+0000","last_clean":"2024-03-16T09:15:00.904292+0000","last_became_active":"2024-03-16T09:13:37.124664+0000","last_became_peered":"2024-03-16T09:13:37.124664+0000","last_unstale":"2024-03-16T09:15:00.904292+0000","last_undegraded":"2024-03-16T09:15:00.904292+0000","last_fullsized":"2024-03-16T09:15:00.904292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:24:30.104576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.230143+0000","last_change":"2024-03-16T09:13:36.121980+0000","last_active":"2024-03-16T09:14:32.230143+0000","last_peered":"2024-03-16T09:14:32.230143+0000","last_clean":"2024-03-16T09:14:32.230143+0000","last_became_active":"2024-03-16T09:13:36.121703+0000","last_became_peered":"2024-03-16T09:13:36.121703+0000","last_unstale":"2024-03-16T09:14:32.230143+0000","last_undegraded":"2024-03-16T09:14:32.230143+0000","last_fullsized":"2024-03-16T09:14:32.230143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:40:26.601688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.920701+0000","last_change":"2024-03-16T09:13:37.123443+0000","last_active":"2024-03-16T09:15:00.920701+0000","last_peered":"2024-03-16T09:15:00.920701+0000","last_clean":"2024-03-16T09:15:00.920701+0000","last_became_active":"2024-03-16T09:13:37.121987+0000","last_became_peered":"2024-03-16T09:13:37.121987+0000","last_unstale":"2024-03-16T09:15:00.920701+0000","last_undegraded":"2024-03-16T09:15:00.920701+0000","last_fullsized":"2024-03-16T09:15:00.920701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:37:08.478552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385000+0000","last_change":"2024-03-16T09:13:36.106391+0000","last_active":"2024-03-16T09:14:32.385000+0000","last_peered":"2024-03-16T09:14:32.385000+0000","last_clean":"2024-03-16T09:14:32.385000+0000","last_became_active":"2024-03-16T09:13:36.106046+0000","last_became_peered":"2024-03-16T09:13:36.106046+0000","last_unstale":"2024-03-16T09:14:32.385000+0000","last_undegraded":"2024-03-16T09:14:32.385000+0000","last_fullsized":"2024-03-16T09:14:32.385000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:03:23.255519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.911156+0000","last_change":"2024-03-16T09:13:37.114337+0000","last_active":"2024-03-16T09:15:00.911156+0000","last_peered":"2024-03-16T09:15:00.911156+0000","last_clean":"2024-03-16T09:15:00.911156+0000","last_became_active":"2024-03-16T09:13:37.113991+0000","last_became_peered":"2024-03-16T09:13:37.113991+0000","last_unstale":"2024-03-16T09:15:00.911156+0000","last_undegraded":"2024-03-16T09:15:00.911156+0000","last_fullsized":"2024-03-16T09:15:00.911156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:23:54.141479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248113+0000","last_change":"2024-03-16T09:13:36.113297+0000","last_active":"2024-03-16T09:14:32.248113+0000","last_peered":"2024-03-16T09:14:32.248113+0000","last_clean":"2024-03-16T09:14:32.248113+0000","last_became_active":"2024-03-16T09:13:36.112950+0000","last_became_peered":"2024-03-16T09:13:36.112950+0000","last_unstale":"2024-03-16T09:14:32.248113+0000","last_undegraded":"2024-03-16T09:14:32.248113+0000","last_fullsized":"2024-03-16T09:14:32.248113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:40:17.682450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.922064+0000","last_change":"2024-03-16T09:13:37.112239+0000","last_active":"2024-03-16T09:15:00.922064+0000","last_peered":"2024-03-16T09:15:00.922064+0000","last_clean":"2024-03-16T09:15:00.922064+0000","last_became_active":"2024-03-16T09:13:37.111885+0000","last_became_peered":"2024-03-16T09:13:37.111885+0000","last_unstale":"2024-03-16T09:15:00.922064+0000","last_undegraded":"2024-03-16T09:15:00.922064+0000","last_fullsized":"2024-03-16T09:15:00.922064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:49.565846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":138,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:34.445555+0000","last_change":"2024-03-16T09:13:36.111341+0000","last_active":"2024-03-16T09:15:34.445555+0000","last_peered":"2024-03-16T09:15:34.445555+0000","last_clean":"2024-03-16T09:15:34.445555+0000","last_became_active":"2024-03-16T09:13:36.110558+0000","last_became_peered":"2024-03-16T09:13:36.110558+0000","last_unstale":"2024-03-16T09:15:34.445555+0000","last_undegraded":"2024-03-16T09:15:34.445555+0000","last_fullsized":"2024-03-16T09:15:34.445555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:56:44.252002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.933077+0000","last_change":"2024-03-16T09:13:37.126808+0000","last_active":"2024-03-16T09:15:00.933077+0000","last_peered":"2024-03-16T09:15:00.933077+0000","last_clean":"2024-03-16T09:15:00.933077+0000","last_became_active":"2024-03-16T09:13:37.126442+0000","last_became_peered":"2024-03-16T09:13:37.126442+0000","last_unstale":"2024-03-16T09:15:00.933077+0000","last_undegraded":"2024-03-16T09:15:00.933077+0000","last_fullsized":"2024-03-16T09:15:00.933077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:04:39.216835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.004330+0000","last_change":"2024-03-16T09:14:32.004330+0000","last_active":"2024-03-16T09:14:32.004330+0000","last_peered":"2024-03-16T09:14:32.004330+0000","last_clean":"2024-03-16T09:14:32.004330+0000","last_became_active":"2024-03-16T09:14:32.001741+0000","last_became_peered":"2024-03-16T09:14:32.001741+0000","last_unstale":"2024-03-16T09:14:32.004330+0000","last_undegraded":"2024-03-16T09:14:32.004330+0000","last_fullsized":"2024-03-16T09:14:32.004330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:08:19.002873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.921799+0000","last_change":"2024-03-16T09:13:37.125833+0000","last_active":"2024-03-16T09:15:00.921799+0000","last_peered":"2024-03-16T09:15:00.921799+0000","last_clean":"2024-03-16T09:15:00.921799+0000","last_became_active":"2024-03-16T09:13:37.125508+0000","last_became_peered":"2024-03-16T09:13:37.125508+0000","last_unstale":"2024-03-16T09:15:00.921799+0000","last_undegraded":"2024-03-16T09:15:00.921799+0000","last_fullsized":"2024-03-16T09:15:00.921799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:14:47.749715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248156+0000","last_change":"2024-03-16T09:13:36.112200+0000","last_active":"2024-03-16T09:14:32.248156+0000","last_peered":"2024-03-16T09:14:32.248156+0000","last_clean":"2024-03-16T09:14:32.248156+0000","last_became_active":"2024-03-16T09:13:36.111830+0000","last_became_peered":"2024-03-16T09:13:36.111830+0000","last_unstale":"2024-03-16T09:14:32.248156+0000","last_undegraded":"2024-03-16T09:14:32.248156+0000","last_fullsized":"2024-03-16T09:14:32.248156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:07:57.147960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.934446+0000","last_change":"2024-03-16T09:13:37.118434+0000","last_active":"2024-03-16T09:15:00.934446+0000","last_peered":"2024-03-16T09:15:00.934446+0000","last_clean":"2024-03-16T09:15:00.934446+0000","last_became_active":"2024-03-16T09:13:37.118069+0000","last_became_peered":"2024-03-16T09:13:37.118069+0000","last_unstale":"2024-03-16T09:15:00.934446+0000","last_undegraded":"2024-03-16T09:15:00.934446+0000","last_fullsized":"2024-03-16T09:15:00.934446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:02:56.559384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'32","reported_seq":292,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:35.447044+0000","last_change":"2024-03-16T09:13:36.114591+0000","last_active":"2024-03-16T09:15:35.447044+0000","last_peered":"2024-03-16T09:15:35.447044+0000","last_clean":"2024-03-16T09:15:35.447044+0000","last_became_active":"2024-03-16T09:13:36.114252+0000","last_became_peered":"2024-03-16T09:13:36.114252+0000","last_unstale":"2024-03-16T09:15:35.447044+0000","last_undegraded":"2024-03-16T09:15:35.447044+0000","last_fullsized":"2024-03-16T09:15:35.447044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:37:52.637394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60022,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":241,"num_read_kb":179,"num_write":33,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.913494+0000","last_change":"2024-03-16T09:13:37.133102+0000","last_active":"2024-03-16T09:15:00.913494+0000","last_peered":"2024-03-16T09:15:00.913494+0000","last_clean":"2024-03-16T09:15:00.913494+0000","last_became_active":"2024-03-16T09:13:37.132728+0000","last_became_peered":"2024-03-16T09:13:37.132728+0000","last_unstale":"2024-03-16T09:15:00.913494+0000","last_undegraded":"2024-03-16T09:15:00.913494+0000","last_fullsized":"2024-03-16T09:15:00.913494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:54:27.511997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998770+0000","last_change":"2024-03-16T09:13:36.114195+0000","last_active":"2024-03-16T09:14:31.998770+0000","last_peered":"2024-03-16T09:14:31.998770+0000","last_clean":"2024-03-16T09:14:31.998770+0000","last_became_active":"2024-03-16T09:13:36.113829+0000","last_became_peered":"2024-03-16T09:13:36.113829+0000","last_unstale":"2024-03-16T09:14:31.998770+0000","last_undegraded":"2024-03-16T09:14:31.998770+0000","last_fullsized":"2024-03-16T09:14:31.998770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:28:40.509877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.912363+0000","last_change":"2024-03-16T09:13:37.132974+0000","last_active":"2024-03-16T09:15:00.912363+0000","last_peered":"2024-03-16T09:15:00.912363+0000","last_clean":"2024-03-16T09:15:00.912363+0000","last_became_active":"2024-03-16T09:13:37.132474+0000","last_became_peered":"2024-03-16T09:13:37.132474+0000","last_unstale":"2024-03-16T09:15:00.912363+0000","last_undegraded":"2024-03-16T09:15:00.912363+0000","last_fullsized":"2024-03-16T09:15:00.912363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:58:04.480441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.247992+0000","last_change":"2024-03-16T09:13:36.115242+0000","last_active":"2024-03-16T09:14:32.247992+0000","last_peered":"2024-03-16T09:14:32.247992+0000","last_clean":"2024-03-16T09:14:32.247992+0000","last_became_active":"2024-03-16T09:13:36.113566+0000","last_became_peered":"2024-03-16T09:13:36.113566+0000","last_unstale":"2024-03-16T09:14:32.247992+0000","last_undegraded":"2024-03-16T09:14:32.247992+0000","last_fullsized":"2024-03-16T09:14:32.247992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:47:50.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.927334+0000","last_change":"2024-03-16T09:14:32.386530+0000","last_active":"2024-03-16T09:15:00.927334+0000","last_peered":"2024-03-16T09:15:00.927334+0000","last_clean":"2024-03-16T09:15:00.927334+0000","last_became_active":"2024-03-16T09:14:32.386088+0000","last_became_peered":"2024-03-16T09:14:32.386088+0000","last_unstale":"2024-03-16T09:15:00.927334+0000","last_undegraded":"2024-03-16T09:15:00.927334+0000","last_fullsized":"2024-03-16T09:15:00.927334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:44:57.427154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":144,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:36.457841+0000","last_change":"2024-03-16T09:13:36.109585+0000","last_active":"2024-03-16T09:15:36.457841+0000","last_peered":"2024-03-16T09:15:36.457841+0000","last_clean":"2024-03-16T09:15:36.457841+0000","last_became_active":"2024-03-16T09:13:36.109267+0000","last_became_peered":"2024-03-16T09:13:36.109267+0000","last_unstale":"2024-03-16T09:15:36.457841+0000","last_undegraded":"2024-03-16T09:15:36.457841+0000","last_fullsized":"2024-03-16T09:15:36.457841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:26:55.700791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.926195+0000","last_change":"2024-03-16T09:13:37.136465+0000","last_active":"2024-03-16T09:15:00.926195+0000","last_peered":"2024-03-16T09:15:00.926195+0000","last_clean":"2024-03-16T09:15:00.926195+0000","last_became_active":"2024-03-16T09:13:37.135933+0000","last_became_peered":"2024-03-16T09:13:37.135933+0000","last_unstale":"2024-03-16T09:15:00.926195+0000","last_undegraded":"2024-03-16T09:15:00.926195+0000","last_fullsized":"2024-03-16T09:15:00.926195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:08:28.490533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994732+0000","last_change":"2024-03-16T09:13:36.119428+0000","last_active":"2024-03-16T09:14:31.994732+0000","last_peered":"2024-03-16T09:14:31.994732+0000","last_clean":"2024-03-16T09:14:31.994732+0000","last_became_active":"2024-03-16T09:13:36.119083+0000","last_became_peered":"2024-03-16T09:13:36.119083+0000","last_unstale":"2024-03-16T09:14:31.994732+0000","last_undegraded":"2024-03-16T09:14:31.994732+0000","last_fullsized":"2024-03-16T09:14:31.994732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:58:56.065151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.920460+0000","last_change":"2024-03-16T09:13:37.118082+0000","last_active":"2024-03-16T09:15:00.920460+0000","last_peered":"2024-03-16T09:15:00.920460+0000","last_clean":"2024-03-16T09:15:00.920460+0000","last_became_active":"2024-03-16T09:13:37.117671+0000","last_became_peered":"2024-03-16T09:13:37.117671+0000","last_unstale":"2024-03-16T09:15:00.920460+0000","last_undegraded":"2024-03-16T09:15:00.920460+0000","last_fullsized":"2024-03-16T09:15:00.920460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:14:05.352191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.995286+0000","last_change":"2024-03-16T09:13:36.121077+0000","last_active":"2024-03-16T09:14:31.995286+0000","last_peered":"2024-03-16T09:14:31.995286+0000","last_clean":"2024-03-16T09:14:31.995286+0000","last_became_active":"2024-03-16T09:13:36.120723+0000","last_became_peered":"2024-03-16T09:13:36.120723+0000","last_unstale":"2024-03-16T09:14:31.995286+0000","last_undegraded":"2024-03-16T09:14:31.995286+0000","last_fullsized":"2024-03-16T09:14:31.995286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:58:14.746215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.922037+0000","last_change":"2024-03-16T09:13:37.116725+0000","last_active":"2024-03-16T09:15:00.922037+0000","last_peered":"2024-03-16T09:15:00.922037+0000","last_clean":"2024-03-16T09:15:00.922037+0000","last_became_active":"2024-03-16T09:13:37.116406+0000","last_became_peered":"2024-03-16T09:13:37.116406+0000","last_unstale":"2024-03-16T09:15:00.922037+0000","last_undegraded":"2024-03-16T09:15:00.922037+0000","last_fullsized":"2024-03-16T09:15:00.922037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:56:59.136649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996706+0000","last_change":"2024-03-16T09:13:36.110568+0000","last_active":"2024-03-16T09:14:31.996706+0000","last_peered":"2024-03-16T09:14:31.996706+0000","last_clean":"2024-03-16T09:14:31.996706+0000","last_became_active":"2024-03-16T09:13:36.110246+0000","last_became_peered":"2024-03-16T09:13:36.110246+0000","last_unstale":"2024-03-16T09:14:31.996706+0000","last_undegraded":"2024-03-16T09:14:31.996706+0000","last_fullsized":"2024-03-16T09:14:31.996706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:36:50.011941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.921527+0000","last_change":"2024-03-16T09:13:37.122296+0000","last_active":"2024-03-16T09:15:00.921527+0000","last_peered":"2024-03-16T09:15:00.921527+0000","last_clean":"2024-03-16T09:15:00.921527+0000","last_became_active":"2024-03-16T09:13:37.121968+0000","last_became_peered":"2024-03-16T09:13:37.121968+0000","last_unstale":"2024-03-16T09:15:00.921527+0000","last_undegraded":"2024-03-16T09:15:00.921527+0000","last_fullsized":"2024-03-16T09:15:00.921527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:10:43.449397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997895+0000","last_change":"2024-03-16T09:13:36.112262+0000","last_active":"2024-03-16T09:14:31.997895+0000","last_peered":"2024-03-16T09:14:31.997895+0000","last_clean":"2024-03-16T09:14:31.997895+0000","last_became_active":"2024-03-16T09:13:36.111916+0000","last_became_peered":"2024-03-16T09:13:36.111916+0000","last_unstale":"2024-03-16T09:14:31.997895+0000","last_undegraded":"2024-03-16T09:14:31.997895+0000","last_fullsized":"2024-03-16T09:14:31.997895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:27:41.163508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.906629+0000","last_change":"2024-03-16T09:13:37.119814+0000","last_active":"2024-03-16T09:15:00.906629+0000","last_peered":"2024-03-16T09:15:00.906629+0000","last_clean":"2024-03-16T09:15:00.906629+0000","last_became_active":"2024-03-16T09:13:37.119454+0000","last_became_peered":"2024-03-16T09:13:37.119454+0000","last_unstale":"2024-03-16T09:15:00.906629+0000","last_undegraded":"2024-03-16T09:15:00.906629+0000","last_fullsized":"2024-03-16T09:15:00.906629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:57:25.988044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.904133+0000","last_change":"2024-03-16T09:14:31.999902+0000","last_active":"2024-03-16T09:15:00.904133+0000","last_peered":"2024-03-16T09:15:00.904133+0000","last_clean":"2024-03-16T09:15:00.904133+0000","last_became_active":"2024-03-16T09:14:31.999609+0000","last_became_peered":"2024-03-16T09:14:31.999609+0000","last_unstale":"2024-03-16T09:15:00.904133+0000","last_undegraded":"2024-03-16T09:15:00.904133+0000","last_fullsized":"2024-03-16T09:15:00.904133+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:58:47.495746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":142,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:36.456168+0000","last_change":"2024-03-16T09:13:36.111300+0000","last_active":"2024-03-16T09:15:36.456168+0000","last_peered":"2024-03-16T09:15:36.456168+0000","last_clean":"2024-03-16T09:15:36.456168+0000","last_became_active":"2024-03-16T09:13:36.110966+0000","last_became_peered":"2024-03-16T09:13:36.110966+0000","last_unstale":"2024-03-16T09:15:36.456168+0000","last_undegraded":"2024-03-16T09:15:36.456168+0000","last_fullsized":"2024-03-16T09:15:36.456168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:18:33.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.931071+0000","last_change":"2024-03-16T09:13:37.125791+0000","last_active":"2024-03-16T09:15:00.931071+0000","last_peered":"2024-03-16T09:15:00.931071+0000","last_clean":"2024-03-16T09:15:00.931071+0000","last_became_active":"2024-03-16T09:13:37.125448+0000","last_became_peered":"2024-03-16T09:13:37.125448+0000","last_unstale":"2024-03-16T09:15:00.931071+0000","last_undegraded":"2024-03-16T09:15:00.931071+0000","last_fullsized":"2024-03-16T09:15:00.931071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:51:30.767750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996257+0000","last_change":"2024-03-16T09:13:36.107462+0000","last_active":"2024-03-16T09:14:31.996257+0000","last_peered":"2024-03-16T09:14:31.996257+0000","last_clean":"2024-03-16T09:14:31.996257+0000","last_became_active":"2024-03-16T09:13:36.107115+0000","last_became_peered":"2024-03-16T09:13:36.107115+0000","last_unstale":"2024-03-16T09:14:31.996257+0000","last_undegraded":"2024-03-16T09:14:31.996257+0000","last_fullsized":"2024-03-16T09:14:31.996257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:37.796719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997730+0000","last_change":"2024-03-16T09:12:24.331159+0000","last_active":"2024-03-16T09:14:31.997730+0000","last_peered":"2024-03-16T09:14:31.997730+0000","last_clean":"2024-03-16T09:14:31.997730+0000","last_became_active":"2024-03-16T09:12:24.330554+0000","last_became_peered":"2024-03-16T09:12:24.330554+0000","last_unstale":"2024-03-16T09:14:31.997730+0000","last_undegraded":"2024-03-16T09:14:31.997730+0000","last_fullsized":"2024-03-16T09:14:31.997730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:08:57.697012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:08:57.697012+0000","last_clean_scrub_stamp":"2024-03-16T09:08:57.697012+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:43:02.986865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.909651+0000","last_change":"2024-03-16T09:13:37.114299+0000","last_active":"2024-03-16T09:15:00.909651+0000","last_peered":"2024-03-16T09:15:00.909651+0000","last_clean":"2024-03-16T09:15:00.909651+0000","last_became_active":"2024-03-16T09:13:37.113981+0000","last_became_peered":"2024-03-16T09:13:37.113981+0000","last_unstale":"2024-03-16T09:15:00.909651+0000","last_undegraded":"2024-03-16T09:15:00.909651+0000","last_fullsized":"2024-03-16T09:15:00.909651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:17:15.882130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997873+0000","last_change":"2024-03-16T09:13:36.113962+0000","last_active":"2024-03-16T09:14:31.997873+0000","last_peered":"2024-03-16T09:14:31.997873+0000","last_clean":"2024-03-16T09:14:31.997873+0000","last_became_active":"2024-03-16T09:13:36.113616+0000","last_became_peered":"2024-03-16T09:13:36.113616+0000","last_unstale":"2024-03-16T09:14:31.997873+0000","last_undegraded":"2024-03-16T09:14:31.997873+0000","last_fullsized":"2024-03-16T09:14:31.997873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:15:15.004360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.995031+0000","last_change":"2024-03-16T09:13:36.121946+0000","last_active":"2024-03-16T09:14:31.995031+0000","last_peered":"2024-03-16T09:14:31.995031+0000","last_clean":"2024-03-16T09:14:31.995031+0000","last_became_active":"2024-03-16T09:13:36.121209+0000","last_became_peered":"2024-03-16T09:13:36.121209+0000","last_unstale":"2024-03-16T09:14:31.995031+0000","last_undegraded":"2024-03-16T09:14:31.995031+0000","last_fullsized":"2024-03-16T09:14:31.995031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:09:51.139468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.908991+0000","last_change":"2024-03-16T09:13:37.128931+0000","last_active":"2024-03-16T09:15:00.908991+0000","last_peered":"2024-03-16T09:15:00.908991+0000","last_clean":"2024-03-16T09:15:00.908991+0000","last_became_active":"2024-03-16T09:13:37.128515+0000","last_became_peered":"2024-03-16T09:13:37.128515+0000","last_unstale":"2024-03-16T09:15:00.908991+0000","last_undegraded":"2024-03-16T09:15:00.908991+0000","last_fullsized":"2024-03-16T09:15:00.908991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:58:46.429144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:01.126760+0000","last_change":"2024-03-16T09:13:36.118831+0000","last_active":"2024-03-16T09:15:01.126760+0000","last_peered":"2024-03-16T09:15:01.126760+0000","last_clean":"2024-03-16T09:15:01.126760+0000","last_became_active":"2024-03-16T09:13:36.118489+0000","last_became_peered":"2024-03-16T09:13:36.118489+0000","last_unstale":"2024-03-16T09:15:01.126760+0000","last_undegraded":"2024-03-16T09:15:01.126760+0000","last_fullsized":"2024-03-16T09:15:01.126760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:55:34.089451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.923760+0000","last_change":"2024-03-16T09:13:37.120581+0000","last_active":"2024-03-16T09:15:00.923760+0000","last_peered":"2024-03-16T09:15:00.923760+0000","last_clean":"2024-03-16T09:15:00.923760+0000","last_became_active":"2024-03-16T09:13:37.120266+0000","last_became_peered":"2024-03-16T09:13:37.120266+0000","last_unstale":"2024-03-16T09:15:00.923760+0000","last_undegraded":"2024-03-16T09:15:00.923760+0000","last_fullsized":"2024-03-16T09:15:00.923760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:16:10.784229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996414+0000","last_change":"2024-03-16T09:13:36.107125+0000","last_active":"2024-03-16T09:14:31.996414+0000","last_peered":"2024-03-16T09:14:31.996414+0000","last_clean":"2024-03-16T09:14:31.996414+0000","last_became_active":"2024-03-16T09:13:36.106795+0000","last_became_peered":"2024-03-16T09:13:36.106795+0000","last_unstale":"2024-03-16T09:14:31.996414+0000","last_undegraded":"2024-03-16T09:14:31.996414+0000","last_fullsized":"2024-03-16T09:14:31.996414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:29:10.130565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.922341+0000","last_change":"2024-03-16T09:13:37.123013+0000","last_active":"2024-03-16T09:15:00.922341+0000","last_peered":"2024-03-16T09:15:00.922341+0000","last_clean":"2024-03-16T09:15:00.922341+0000","last_became_active":"2024-03-16T09:13:37.122642+0000","last_became_peered":"2024-03-16T09:13:37.122642+0000","last_unstale":"2024-03-16T09:15:00.922341+0000","last_undegraded":"2024-03-16T09:15:00.922341+0000","last_fullsized":"2024-03-16T09:15:00.922341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:35:05.297337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.000335+0000","last_change":"2024-03-16T09:13:36.110868+0000","last_active":"2024-03-16T09:14:32.000335+0000","last_peered":"2024-03-16T09:14:32.000335+0000","last_clean":"2024-03-16T09:14:32.000335+0000","last_became_active":"2024-03-16T09:13:36.110536+0000","last_became_peered":"2024-03-16T09:13:36.110536+0000","last_unstale":"2024-03-16T09:14:32.000335+0000","last_undegraded":"2024-03-16T09:14:32.000335+0000","last_fullsized":"2024-03-16T09:14:32.000335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:07:06.829458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.882568+0000","last_change":"2024-03-16T09:13:37.116579+0000","last_active":"2024-03-16T09:15:00.882568+0000","last_peered":"2024-03-16T09:15:00.882568+0000","last_clean":"2024-03-16T09:15:00.882568+0000","last_became_active":"2024-03-16T09:13:37.116226+0000","last_became_peered":"2024-03-16T09:13:37.116226+0000","last_unstale":"2024-03-16T09:15:00.882568+0000","last_undegraded":"2024-03-16T09:15:00.882568+0000","last_fullsized":"2024-03-16T09:15:00.882568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:08:39.015162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517053+0000","last_change":"2024-03-16T09:13:36.107881+0000","last_active":"2024-03-16T09:14:32.517053+0000","last_peered":"2024-03-16T09:14:32.517053+0000","last_clean":"2024-03-16T09:14:32.517053+0000","last_became_active":"2024-03-16T09:13:36.107538+0000","last_became_peered":"2024-03-16T09:13:36.107538+0000","last_unstale":"2024-03-16T09:14:32.517053+0000","last_undegraded":"2024-03-16T09:14:32.517053+0000","last_fullsized":"2024-03-16T09:14:32.517053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:49:41.932089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.923202+0000","last_change":"2024-03-16T09:13:37.118580+0000","last_active":"2024-03-16T09:15:00.923202+0000","last_peered":"2024-03-16T09:15:00.923202+0000","last_clean":"2024-03-16T09:15:00.923202+0000","last_became_active":"2024-03-16T09:13:37.118166+0000","last_became_peered":"2024-03-16T09:13:37.118166+0000","last_unstale":"2024-03-16T09:15:00.923202+0000","last_undegraded":"2024-03-16T09:15:00.923202+0000","last_fullsized":"2024-03-16T09:15:00.923202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:10:13.957439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998331+0000","last_change":"2024-03-16T09:13:36.122307+0000","last_active":"2024-03-16T09:14:31.998331+0000","last_peered":"2024-03-16T09:14:31.998331+0000","last_clean":"2024-03-16T09:14:31.998331+0000","last_became_active":"2024-03-16T09:13:36.121961+0000","last_became_peered":"2024-03-16T09:13:36.121961+0000","last_unstale":"2024-03-16T09:14:31.998331+0000","last_undegraded":"2024-03-16T09:14:31.998331+0000","last_fullsized":"2024-03-16T09:14:31.998331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:50:03.482300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.921973+0000","last_change":"2024-03-16T09:13:37.131918+0000","last_active":"2024-03-16T09:15:00.921973+0000","last_peered":"2024-03-16T09:15:00.921973+0000","last_clean":"2024-03-16T09:15:00.921973+0000","last_became_active":"2024-03-16T09:13:37.131560+0000","last_became_peered":"2024-03-16T09:13:37.131560+0000","last_unstale":"2024-03-16T09:15:00.921973+0000","last_undegraded":"2024-03-16T09:15:00.921973+0000","last_fullsized":"2024-03-16T09:15:00.921973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:57.315125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994873+0000","last_change":"2024-03-16T09:13:36.120192+0000","last_active":"2024-03-16T09:14:31.994873+0000","last_peered":"2024-03-16T09:14:31.994873+0000","last_clean":"2024-03-16T09:14:31.994873+0000","last_became_active":"2024-03-16T09:13:36.119063+0000","last_became_peered":"2024-03-16T09:13:36.119063+0000","last_unstale":"2024-03-16T09:14:31.994873+0000","last_undegraded":"2024-03-16T09:14:31.994873+0000","last_fullsized":"2024-03-16T09:14:31.994873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:18:56.616624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.935639+0000","last_change":"2024-03-16T09:13:37.129597+0000","last_active":"2024-03-16T09:15:00.935639+0000","last_peered":"2024-03-16T09:15:00.935639+0000","last_clean":"2024-03-16T09:15:00.935639+0000","last_became_active":"2024-03-16T09:13:37.129248+0000","last_became_peered":"2024-03-16T09:13:37.129248+0000","last_unstale":"2024-03-16T09:15:00.935639+0000","last_undegraded":"2024-03-16T09:15:00.935639+0000","last_fullsized":"2024-03-16T09:15:00.935639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:41:11.118192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.010274+0000","last_change":"2024-03-16T09:14:32.009577+0000","last_active":"2024-03-16T09:14:32.010274+0000","last_peered":"2024-03-16T09:14:32.010274+0000","last_clean":"2024-03-16T09:14:32.010274+0000","last_became_active":"2024-03-16T09:14:32.000521+0000","last_became_peered":"2024-03-16T09:14:32.000521+0000","last_unstale":"2024-03-16T09:14:32.010274+0000","last_undegraded":"2024-03-16T09:14:32.010274+0000","last_fullsized":"2024-03-16T09:14:32.010274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:29:07.221252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.935373+0000","last_change":"2024-03-16T09:13:37.120317+0000","last_active":"2024-03-16T09:15:00.935373+0000","last_peered":"2024-03-16T09:15:00.935373+0000","last_clean":"2024-03-16T09:15:00.935373+0000","last_became_active":"2024-03-16T09:13:37.119974+0000","last_became_peered":"2024-03-16T09:13:37.119974+0000","last_unstale":"2024-03-16T09:15:00.935373+0000","last_undegraded":"2024-03-16T09:15:00.935373+0000","last_fullsized":"2024-03-16T09:15:00.935373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:25:34.774377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248531+0000","last_change":"2024-03-16T09:13:36.112395+0000","last_active":"2024-03-16T09:14:32.248531+0000","last_peered":"2024-03-16T09:14:32.248531+0000","last_clean":"2024-03-16T09:14:32.248531+0000","last_became_active":"2024-03-16T09:13:36.112036+0000","last_became_peered":"2024-03-16T09:13:36.112036+0000","last_unstale":"2024-03-16T09:14:32.248531+0000","last_undegraded":"2024-03-16T09:14:32.248531+0000","last_fullsized":"2024-03-16T09:14:32.248531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:41:02.895820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.923449+0000","last_change":"2024-03-16T09:13:37.133845+0000","last_active":"2024-03-16T09:15:00.923449+0000","last_peered":"2024-03-16T09:15:00.923449+0000","last_clean":"2024-03-16T09:15:00.923449+0000","last_became_active":"2024-03-16T09:13:37.132833+0000","last_became_peered":"2024-03-16T09:13:37.132833+0000","last_unstale":"2024-03-16T09:15:00.923449+0000","last_undegraded":"2024-03-16T09:15:00.923449+0000","last_fullsized":"2024-03-16T09:15:00.923449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:12:18.049210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385012+0000","last_change":"2024-03-16T09:13:36.114830+0000","last_active":"2024-03-16T09:14:32.385012+0000","last_peered":"2024-03-16T09:14:32.385012+0000","last_clean":"2024-03-16T09:14:32.385012+0000","last_became_active":"2024-03-16T09:13:36.113898+0000","last_became_peered":"2024-03-16T09:13:36.113898+0000","last_unstale":"2024-03-16T09:14:32.385012+0000","last_undegraded":"2024-03-16T09:14:32.385012+0000","last_fullsized":"2024-03-16T09:14:32.385012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:27:38.910819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.925474+0000","last_change":"2024-03-16T09:13:37.115991+0000","last_active":"2024-03-16T09:15:00.925474+0000","last_peered":"2024-03-16T09:15:00.925474+0000","last_clean":"2024-03-16T09:15:00.925474+0000","last_became_active":"2024-03-16T09:13:37.115671+0000","last_became_peered":"2024-03-16T09:13:37.115671+0000","last_unstale":"2024-03-16T09:15:00.925474+0000","last_undegraded":"2024-03-16T09:15:00.925474+0000","last_fullsized":"2024-03-16T09:15:00.925474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:05:59.703675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.919368+0000","last_change":"2024-03-16T09:13:37.115122+0000","last_active":"2024-03-16T09:15:00.919368+0000","last_peered":"2024-03-16T09:15:00.919368+0000","last_clean":"2024-03-16T09:15:00.919368+0000","last_became_active":"2024-03-16T09:13:37.114779+0000","last_became_peered":"2024-03-16T09:13:37.114779+0000","last_unstale":"2024-03-16T09:15:00.919368+0000","last_undegraded":"2024-03-16T09:15:00.919368+0000","last_fullsized":"2024-03-16T09:15:00.919368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:50:02.051104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.920923+0000","last_change":"2024-03-16T09:13:37.116542+0000","last_active":"2024-03-16T09:15:00.920923+0000","last_peered":"2024-03-16T09:15:00.920923+0000","last_clean":"2024-03-16T09:15:00.920923+0000","last_became_active":"2024-03-16T09:13:37.116224+0000","last_became_peered":"2024-03-16T09:13:37.116224+0000","last_unstale":"2024-03-16T09:15:00.920923+0000","last_undegraded":"2024-03-16T09:15:00.920923+0000","last_fullsized":"2024-03-16T09:15:00.920923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:32:40.106979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.923128+0000","last_change":"2024-03-16T09:13:37.128750+0000","last_active":"2024-03-16T09:15:00.923128+0000","last_peered":"2024-03-16T09:15:00.923128+0000","last_clean":"2024-03-16T09:15:00.923128+0000","last_became_active":"2024-03-16T09:13:37.128411+0000","last_became_peered":"2024-03-16T09:13:37.128411+0000","last_unstale":"2024-03-16T09:15:00.923128+0000","last_undegraded":"2024-03-16T09:15:00.923128+0000","last_fullsized":"2024-03-16T09:15:00.923128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:01:02.119578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.919187+0000","last_change":"2024-03-16T09:13:37.124024+0000","last_active":"2024-03-16T09:15:00.919187+0000","last_peered":"2024-03-16T09:15:00.919187+0000","last_clean":"2024-03-16T09:15:00.919187+0000","last_became_active":"2024-03-16T09:13:37.123721+0000","last_became_peered":"2024-03-16T09:13:37.123721+0000","last_unstale":"2024-03-16T09:15:00.919187+0000","last_undegraded":"2024-03-16T09:15:00.919187+0000","last_fullsized":"2024-03-16T09:15:00.919187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:29:28.079163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.910102+0000","last_change":"2024-03-16T09:13:37.116603+0000","last_active":"2024-03-16T09:15:00.910102+0000","last_peered":"2024-03-16T09:15:00.910102+0000","last_clean":"2024-03-16T09:15:00.910102+0000","last_became_active":"2024-03-16T09:13:37.116279+0000","last_became_peered":"2024-03-16T09:13:37.116279+0000","last_unstale":"2024-03-16T09:15:00.910102+0000","last_undegraded":"2024-03-16T09:15:00.910102+0000","last_fullsized":"2024-03-16T09:15:00.910102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:25:13.001817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.918731+0000","last_change":"2024-03-16T09:13:37.116863+0000","last_active":"2024-03-16T09:15:00.918731+0000","last_peered":"2024-03-16T09:15:00.918731+0000","last_clean":"2024-03-16T09:15:00.918731+0000","last_became_active":"2024-03-16T09:13:37.116543+0000","last_became_peered":"2024-03-16T09:13:37.116543+0000","last_unstale":"2024-03-16T09:15:00.918731+0000","last_undegraded":"2024-03-16T09:15:00.918731+0000","last_fullsized":"2024-03-16T09:15:00.918731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:06:35.501566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.915096+0000","last_change":"2024-03-16T09:13:37.120325+0000","last_active":"2024-03-16T09:15:00.915096+0000","last_peered":"2024-03-16T09:15:00.915096+0000","last_clean":"2024-03-16T09:15:00.915096+0000","last_became_active":"2024-03-16T09:13:37.119985+0000","last_became_peered":"2024-03-16T09:13:37.119985+0000","last_unstale":"2024-03-16T09:15:00.915096+0000","last_undegraded":"2024-03-16T09:15:00.915096+0000","last_fullsized":"2024-03-16T09:15:00.915096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:35.419225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.908253+0000","last_change":"2024-03-16T09:13:37.119604+0000","last_active":"2024-03-16T09:15:00.908253+0000","last_peered":"2024-03-16T09:15:00.908253+0000","last_clean":"2024-03-16T09:15:00.908253+0000","last_became_active":"2024-03-16T09:13:37.119019+0000","last_became_peered":"2024-03-16T09:13:37.119019+0000","last_unstale":"2024-03-16T09:15:00.908253+0000","last_undegraded":"2024-03-16T09:15:00.908253+0000","last_fullsized":"2024-03-16T09:15:00.908253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:42:57.902370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.915549+0000","last_change":"2024-03-16T09:13:37.123845+0000","last_active":"2024-03-16T09:15:00.915549+0000","last_peered":"2024-03-16T09:15:00.915549+0000","last_clean":"2024-03-16T09:15:00.915549+0000","last_became_active":"2024-03-16T09:13:37.121753+0000","last_became_peered":"2024-03-16T09:13:37.121753+0000","last_unstale":"2024-03-16T09:15:00.915549+0000","last_undegraded":"2024-03-16T09:15:00.915549+0000","last_fullsized":"2024-03-16T09:15:00.915549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:23:01.289378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.910994+0000","last_change":"2024-03-16T09:13:37.118182+0000","last_active":"2024-03-16T09:15:00.910994+0000","last_peered":"2024-03-16T09:15:00.910994+0000","last_clean":"2024-03-16T09:15:00.910994+0000","last_became_active":"2024-03-16T09:13:37.117820+0000","last_became_peered":"2024-03-16T09:13:37.117820+0000","last_unstale":"2024-03-16T09:15:00.910994+0000","last_undegraded":"2024-03-16T09:15:00.910994+0000","last_fullsized":"2024-03-16T09:15:00.910994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:27:31.393266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.935406+0000","last_change":"2024-03-16T09:13:37.121348+0000","last_active":"2024-03-16T09:15:00.935406+0000","last_peered":"2024-03-16T09:15:00.935406+0000","last_clean":"2024-03-16T09:15:00.935406+0000","last_became_active":"2024-03-16T09:13:37.120826+0000","last_became_peered":"2024-03-16T09:13:37.120826+0000","last_unstale":"2024-03-16T09:15:00.935406+0000","last_undegraded":"2024-03-16T09:15:00.935406+0000","last_fullsized":"2024-03-16T09:15:00.935406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:36:33.456230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.919550+0000","last_change":"2024-03-16T09:13:37.131889+0000","last_active":"2024-03-16T09:15:00.919550+0000","last_peered":"2024-03-16T09:15:00.919550+0000","last_clean":"2024-03-16T09:15:00.919550+0000","last_became_active":"2024-03-16T09:13:37.131564+0000","last_became_peered":"2024-03-16T09:13:37.131564+0000","last_unstale":"2024-03-16T09:15:00.919550+0000","last_undegraded":"2024-03-16T09:15:00.919550+0000","last_fullsized":"2024-03-16T09:15:00.919550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:43:34.255378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.913691+0000","last_change":"2024-03-16T09:14:31.998395+0000","last_active":"2024-03-16T09:15:00.913691+0000","last_peered":"2024-03-16T09:15:00.913691+0000","last_clean":"2024-03-16T09:15:00.913691+0000","last_became_active":"2024-03-16T09:14:31.998063+0000","last_became_peered":"2024-03-16T09:14:31.998063+0000","last_unstale":"2024-03-16T09:15:00.913691+0000","last_undegraded":"2024-03-16T09:15:00.913691+0000","last_fullsized":"2024-03-16T09:15:00.913691+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:50:57.125765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280795,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64023,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":728,"num_read_kb":431,"num_write":165,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1164,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":207855,"data_compressed":5547,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383718,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":456779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645354,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":1107700,"data_compressed":37563,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070516,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332154,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":1047310,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":455822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":523921,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":457093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":524689,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":456133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":457232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":1047576,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":68306,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69074,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":61600,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T09:15:39.453 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T09:15:39.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.589+0000 7fa784659640 1 -- 172.21.15.112:0/2532646927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa77c1022b0 msgr2=0x7fa77c108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:39.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.589+0000 7fa784659640 1 --2- 172.21.15.112:0/2532646927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa77c1022b0 0x7fa77c108b40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa774009900 tx=0x7fa77402bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:39.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.589+0000 7fa784659640 1 -- 172.21.15.112:0/2532646927 shutdown_connections 2024-03-16T09:15:39.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.589+0000 7fa784659640 1 --2- 172.21.15.112:0/2532646927 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa77c1022b0 0x7fa77c108b40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.589+0000 7fa784659640 1 --2- 172.21.15.112:0/2532646927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa77c1018f0 0x7fa77c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.589+0000 7fa784659640 1 --2- 172.21.15.112:0/2532646927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa77c100700 0x7fa77c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.589+0000 7fa784659640 1 -- 172.21.15.112:0/2532646927 >> 172.21.15.112:0/2532646927 conn(0x7fa77c0fbc70 msgr2=0x7fa77c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:39.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.589+0000 7fa784659640 1 -- 172.21.15.112:0/2532646927 shutdown_connections 2024-03-16T09:15:39.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.589+0000 7fa784659640 1 -- 172.21.15.112:0/2532646927 wait complete. 2024-03-16T09:15:39.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.589+0000 7fa784659640 1 Processor -- start 2024-03-16T09:15:39.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.589+0000 7fa784659640 1 -- start start 2024-03-16T09:15:39.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.589+0000 7fa784659640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa77c100700 0x7fa77c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:39.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.589+0000 7fa784659640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa77c1018f0 0x7fa77c19c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:39.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.589+0000 7fa784659640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa77c1022b0 0x7fa77c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:39.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.589+0000 7fa784659640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c10c460 con 0x7fa77c1022b0 2024-03-16T09:15:39.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.589+0000 7fa784659640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c10c2e0 con 0x7fa77c100700 2024-03-16T09:15:39.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.589+0000 7fa784659640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa77c10c5e0 con 0x7fa77c1018f0 2024-03-16T09:15:39.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.590+0000 7fa782bf6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa77c1022b0 0x7fa77c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:39.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.590+0000 7fa782bf6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa77c1022b0 0x7fa77c19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57250/0 (socket says 172.21.15.112:57250) 2024-03-16T09:15:39.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.590+0000 7fa7823f5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa77c100700 0x7fa77c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:39.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.590+0000 7fa782bf6640 1 -- 172.21.15.112:0/1607313916 learned_addr learned my addr 172.21.15.112:0/1607313916 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:39.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.590+0000 7fa781bf4640 1 --2- 172.21.15.112:0/1607313916 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa77c1018f0 0x7fa77c19c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:39.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.590+0000 7fa782bf6640 1 -- 172.21.15.112:0/1607313916 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa77c1018f0 msgr2=0x7fa77c19c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:39.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.590+0000 7fa782bf6640 1 --2- 172.21.15.112:0/1607313916 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa77c1018f0 0x7fa77c19c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.590+0000 7fa782bf6640 1 -- 172.21.15.112:0/1607313916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa77c100700 msgr2=0x7fa77c0694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:39.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.590+0000 7fa782bf6640 1 --2- 172.21.15.112:0/1607313916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa77c100700 0x7fa77c0694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.590+0000 7fa782bf6640 1 -- 172.21.15.112:0/1607313916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77c19f7f0 con 0x7fa77c1022b0 2024-03-16T09:15:39.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.590+0000 7fa782bf6640 1 --2- 172.21.15.112:0/1607313916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa77c1022b0 0x7fa77c19eee0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa774005ab0 tx=0x7fa774004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:39.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.590+0000 7fa77b7fe640 1 -- 172.21.15.112:0/1607313916 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa774043020 con 0x7fa77c1022b0 2024-03-16T09:15:39.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.590+0000 7fa77b7fe640 1 -- 172.21.15.112:0/1607313916 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa7740042d0 con 0x7fa77c1022b0 2024-03-16T09:15:39.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.590+0000 7fa784659640 1 -- 172.21.15.112:0/1607313916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77c19fa20 con 0x7fa77c1022b0 2024-03-16T09:15:39.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.590+0000 7fa77b7fe640 1 -- 172.21.15.112:0/1607313916 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa774003c50 con 0x7fa77c1022b0 2024-03-16T09:15:39.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.590+0000 7fa784659640 1 -- 172.21.15.112:0/1607313916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa77c19ffc0 con 0x7fa77c1022b0 2024-03-16T09:15:39.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.591+0000 7fa784659640 1 -- 172.21.15.112:0/1607313916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa77c1079c0 con 0x7fa77c1022b0 2024-03-16T09:15:39.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.592+0000 7fa77b7fe640 1 -- 172.21.15.112:0/1607313916 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7fa774046020 con 0x7fa77c1022b0 2024-03-16T09:15:39.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.593+0000 7fa77b7fe640 1 --2- 172.21.15.112:0/1607313916 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa7640a2910 0x7fa7640a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:39.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.593+0000 7fa7823f5640 1 --2- 172.21.15.112:0/1607313916 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa7640a2910 0x7fa7640a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:39.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.593+0000 7fa77b7fe640 1 -- 172.21.15.112:0/1607313916 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa7740f0c00 con 0x7fa77c1022b0 2024-03-16T09:15:39.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.593+0000 7fa7823f5640 1 --2- 172.21.15.112:0/1607313916 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa7640a2910 0x7fa7640a4dd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa77c101750 tx=0x7fa76c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:39.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.597+0000 7fa77b7fe640 1 -- 172.21.15.112:0/1607313916 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa77404d020 con 0x7fa77c1022b0 2024-03-16T09:15:39.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.770+0000 7fa784659640 1 -- 172.21.15.112:0/1607313916 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa77c061c20 con 0x7fa7640a2910 2024-03-16T09:15:39.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.777+0000 7fa77b7fe640 1 -- 172.21.15.112:0/1607313916 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259060 (secure 0 0 0) 0x7fa77c061c20 con 0x7fa7640a2910 2024-03-16T09:15:39.783 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:39.784 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-03-16T09:15:39.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.780+0000 7fa784659640 1 -- 172.21.15.112:0/1607313916 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa7640a2910 msgr2=0x7fa7640a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:39.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.780+0000 7fa784659640 1 --2- 172.21.15.112:0/1607313916 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa7640a2910 0x7fa7640a4dd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa77c101750 tx=0x7fa76c009500 comp rx=0 tx=0).stop 2024-03-16T09:15:39.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.780+0000 7fa784659640 1 -- 172.21.15.112:0/1607313916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa77c1022b0 msgr2=0x7fa77c19eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:39.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.780+0000 7fa784659640 1 --2- 172.21.15.112:0/1607313916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa77c1022b0 0x7fa77c19eee0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa774005ab0 tx=0x7fa774004ad0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.780+0000 7fa784659640 1 -- 172.21.15.112:0/1607313916 shutdown_connections 2024-03-16T09:15:39.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.780+0000 7fa784659640 1 --2- 172.21.15.112:0/1607313916 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa7640a2910 0x7fa7640a4dd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.780+0000 7fa784659640 1 --2- 172.21.15.112:0/1607313916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa77c1022b0 0x7fa77c19eee0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.780+0000 7fa784659640 1 --2- 172.21.15.112:0/1607313916 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa77c1018f0 0x7fa77c19c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.780+0000 7fa784659640 1 --2- 172.21.15.112:0/1607313916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa77c100700 0x7fa77c0694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.780+0000 7fa784659640 1 -- 172.21.15.112:0/1607313916 >> 172.21.15.112:0/1607313916 conn(0x7fa77c0fbc70 msgr2=0x7fa77c104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:39.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.780+0000 7fa784659640 1 -- 172.21.15.112:0/1607313916 shutdown_connections 2024-03-16T09:15:39.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.780+0000 7fa784659640 1 -- 172.21.15.112:0/1607313916 wait complete. 2024-03-16T09:15:39.793 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-03-16T09:15:38.906882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654625,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":854,"num_read_kb":540,"num_write":348,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181900,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":781,"ondisk_log_size":781,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352404,"kb_used_data":11572,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589420,"statfs":{"total":1151940427776,"available":1151579566080,"internally_reserved":0,"allocated":11849728,"data_stored":7447041,"data_compressed":112689,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":699,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":37,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002575"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.923673+0000","last_change":"2024-03-16T09:13:37.119969+0000","last_active":"2024-03-16T09:15:00.923673+0000","last_peered":"2024-03-16T09:15:00.923673+0000","last_clean":"2024-03-16T09:15:00.923673+0000","last_became_active":"2024-03-16T09:13:37.119177+0000","last_became_peered":"2024-03-16T09:13:37.119177+0000","last_unstale":"2024-03-16T09:15:00.923673+0000","last_undegraded":"2024-03-16T09:15:00.923673+0000","last_fullsized":"2024-03-16T09:15:00.923673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:25:28.038898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.919826+0000","last_change":"2024-03-16T09:13:37.127564+0000","last_active":"2024-03-16T09:15:00.919826+0000","last_peered":"2024-03-16T09:15:00.919826+0000","last_clean":"2024-03-16T09:15:00.919826+0000","last_became_active":"2024-03-16T09:13:37.127214+0000","last_became_peered":"2024-03-16T09:13:37.127214+0000","last_unstale":"2024-03-16T09:15:00.919826+0000","last_undegraded":"2024-03-16T09:15:00.919826+0000","last_fullsized":"2024-03-16T09:15:00.919826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:46:14.655707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.933238+0000","last_change":"2024-03-16T09:13:37.126288+0000","last_active":"2024-03-16T09:15:00.933238+0000","last_peered":"2024-03-16T09:15:00.933238+0000","last_clean":"2024-03-16T09:15:00.933238+0000","last_became_active":"2024-03-16T09:13:37.125383+0000","last_became_peered":"2024-03-16T09:13:37.125383+0000","last_unstale":"2024-03-16T09:15:00.933238+0000","last_undegraded":"2024-03-16T09:15:00.933238+0000","last_fullsized":"2024-03-16T09:15:00.933238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:47:09.901508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.919434+0000","last_change":"2024-03-16T09:13:37.125503+0000","last_active":"2024-03-16T09:15:00.919434+0000","last_peered":"2024-03-16T09:15:00.919434+0000","last_clean":"2024-03-16T09:15:00.919434+0000","last_became_active":"2024-03-16T09:13:37.125144+0000","last_became_peered":"2024-03-16T09:13:37.125144+0000","last_unstale":"2024-03-16T09:15:00.919434+0000","last_undegraded":"2024-03-16T09:15:00.919434+0000","last_fullsized":"2024-03-16T09:15:00.919434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:30:26.529080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.900877+0000","last_change":"2024-03-16T09:13:37.114237+0000","last_active":"2024-03-16T09:15:00.900877+0000","last_peered":"2024-03-16T09:15:00.900877+0000","last_clean":"2024-03-16T09:15:00.900877+0000","last_became_active":"2024-03-16T09:13:37.113595+0000","last_became_peered":"2024-03-16T09:13:37.113595+0000","last_unstale":"2024-03-16T09:15:00.900877+0000","last_undegraded":"2024-03-16T09:15:00.900877+0000","last_fullsized":"2024-03-16T09:15:00.900877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:06:17.735301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.914380+0000","last_change":"2024-03-16T09:13:37.118843+0000","last_active":"2024-03-16T09:15:00.914380+0000","last_peered":"2024-03-16T09:15:00.914380+0000","last_clean":"2024-03-16T09:15:00.914380+0000","last_became_active":"2024-03-16T09:13:37.117086+0000","last_became_peered":"2024-03-16T09:13:37.117086+0000","last_unstale":"2024-03-16T09:15:00.914380+0000","last_undegraded":"2024-03-16T09:15:00.914380+0000","last_fullsized":"2024-03-16T09:15:00.914380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:23:12.929931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.916488+0000","last_change":"2024-03-16T09:13:37.133427+0000","last_active":"2024-03-16T09:15:00.916488+0000","last_peered":"2024-03-16T09:15:00.916488+0000","last_clean":"2024-03-16T09:15:00.916488+0000","last_became_active":"2024-03-16T09:13:37.133032+0000","last_became_peered":"2024-03-16T09:13:37.133032+0000","last_unstale":"2024-03-16T09:15:00.916488+0000","last_undegraded":"2024-03-16T09:15:00.916488+0000","last_fullsized":"2024-03-16T09:15:00.916488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:49:28.179883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.920627+0000","last_change":"2024-03-16T09:13:37.132132+0000","last_active":"2024-03-16T09:15:00.920627+0000","last_peered":"2024-03-16T09:15:00.920627+0000","last_clean":"2024-03-16T09:15:00.920627+0000","last_became_active":"2024-03-16T09:13:37.131801+0000","last_became_peered":"2024-03-16T09:13:37.131801+0000","last_unstale":"2024-03-16T09:15:00.920627+0000","last_undegraded":"2024-03-16T09:15:00.920627+0000","last_fullsized":"2024-03-16T09:15:00.920627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:45:03.891725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.930016+0000","last_change":"2024-03-16T09:13:37.116378+0000","last_active":"2024-03-16T09:15:00.930016+0000","last_peered":"2024-03-16T09:15:00.930016+0000","last_clean":"2024-03-16T09:15:00.930016+0000","last_became_active":"2024-03-16T09:13:37.116027+0000","last_became_peered":"2024-03-16T09:13:37.116027+0000","last_unstale":"2024-03-16T09:15:00.930016+0000","last_undegraded":"2024-03-16T09:15:00.930016+0000","last_fullsized":"2024-03-16T09:15:00.930016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:14:38.681613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.919236+0000","last_change":"2024-03-16T09:13:37.121417+0000","last_active":"2024-03-16T09:15:00.919236+0000","last_peered":"2024-03-16T09:15:00.919236+0000","last_clean":"2024-03-16T09:15:00.919236+0000","last_became_active":"2024-03-16T09:13:37.121112+0000","last_became_peered":"2024-03-16T09:13:37.121112+0000","last_unstale":"2024-03-16T09:15:00.919236+0000","last_undegraded":"2024-03-16T09:15:00.919236+0000","last_fullsized":"2024-03-16T09:15:00.919236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:38:07.823604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.935183+0000","last_change":"2024-03-16T09:13:37.121368+0000","last_active":"2024-03-16T09:15:00.935183+0000","last_peered":"2024-03-16T09:15:00.935183+0000","last_clean":"2024-03-16T09:15:00.935183+0000","last_became_active":"2024-03-16T09:13:37.121060+0000","last_became_peered":"2024-03-16T09:13:37.121060+0000","last_unstale":"2024-03-16T09:15:00.935183+0000","last_undegraded":"2024-03-16T09:15:00.935183+0000","last_fullsized":"2024-03-16T09:15:00.935183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:48:49.168285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.903523+0000","last_change":"2024-03-16T09:13:37.127004+0000","last_active":"2024-03-16T09:15:00.903523+0000","last_peered":"2024-03-16T09:15:00.903523+0000","last_clean":"2024-03-16T09:15:00.903523+0000","last_became_active":"2024-03-16T09:13:37.126579+0000","last_became_peered":"2024-03-16T09:13:37.126579+0000","last_unstale":"2024-03-16T09:15:00.903523+0000","last_undegraded":"2024-03-16T09:15:00.903523+0000","last_fullsized":"2024-03-16T09:15:00.903523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:54:18.382514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.918879+0000","last_change":"2024-03-16T09:13:37.104534+0000","last_active":"2024-03-16T09:15:00.918879+0000","last_peered":"2024-03-16T09:15:00.918879+0000","last_clean":"2024-03-16T09:15:00.918879+0000","last_became_active":"2024-03-16T09:13:37.104202+0000","last_became_peered":"2024-03-16T09:13:37.104202+0000","last_unstale":"2024-03-16T09:15:00.918879+0000","last_undegraded":"2024-03-16T09:15:00.918879+0000","last_fullsized":"2024-03-16T09:15:00.918879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:40:55.448238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.920211+0000","last_change":"2024-03-16T09:13:37.117166+0000","last_active":"2024-03-16T09:15:00.920211+0000","last_peered":"2024-03-16T09:15:00.920211+0000","last_clean":"2024-03-16T09:15:00.920211+0000","last_became_active":"2024-03-16T09:13:37.116747+0000","last_became_peered":"2024-03-16T09:13:37.116747+0000","last_unstale":"2024-03-16T09:15:00.920211+0000","last_undegraded":"2024-03-16T09:15:00.920211+0000","last_fullsized":"2024-03-16T09:15:00.920211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:50:58.705683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.921643+0000","last_change":"2024-03-16T09:14:32.000383+0000","last_active":"2024-03-16T09:15:00.921643+0000","last_peered":"2024-03-16T09:15:00.921643+0000","last_clean":"2024-03-16T09:15:00.921643+0000","last_became_active":"2024-03-16T09:14:32.000027+0000","last_became_peered":"2024-03-16T09:14:32.000027+0000","last_unstale":"2024-03-16T09:15:00.921643+0000","last_undegraded":"2024-03-16T09:15:00.921643+0000","last_fullsized":"2024-03-16T09:15:00.921643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:04:03.478849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.934752+0000","last_change":"2024-03-16T09:13:37.120583+0000","last_active":"2024-03-16T09:15:00.934752+0000","last_peered":"2024-03-16T09:15:00.934752+0000","last_clean":"2024-03-16T09:15:00.934752+0000","last_became_active":"2024-03-16T09:13:37.120217+0000","last_became_peered":"2024-03-16T09:13:37.120217+0000","last_unstale":"2024-03-16T09:15:00.934752+0000","last_undegraded":"2024-03-16T09:15:00.934752+0000","last_fullsized":"2024-03-16T09:15:00.934752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:00:17.653736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.928060+0000","last_change":"2024-03-16T09:13:37.121179+0000","last_active":"2024-03-16T09:15:00.928060+0000","last_peered":"2024-03-16T09:15:00.928060+0000","last_clean":"2024-03-16T09:15:00.928060+0000","last_became_active":"2024-03-16T09:13:37.120070+0000","last_became_peered":"2024-03-16T09:13:37.120070+0000","last_unstale":"2024-03-16T09:15:00.928060+0000","last_undegraded":"2024-03-16T09:15:00.928060+0000","last_fullsized":"2024-03-16T09:15:00.928060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:39:24.937068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.935539+0000","last_change":"2024-03-16T09:13:37.110074+0000","last_active":"2024-03-16T09:15:00.935539+0000","last_peered":"2024-03-16T09:15:00.935539+0000","last_clean":"2024-03-16T09:15:00.935539+0000","last_became_active":"2024-03-16T09:13:37.109767+0000","last_became_peered":"2024-03-16T09:13:37.109767+0000","last_unstale":"2024-03-16T09:15:00.935539+0000","last_undegraded":"2024-03-16T09:15:00.935539+0000","last_fullsized":"2024-03-16T09:15:00.935539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:20:37.408262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.911978+0000","last_change":"2024-03-16T09:13:37.130791+0000","last_active":"2024-03-16T09:15:00.911978+0000","last_peered":"2024-03-16T09:15:00.911978+0000","last_clean":"2024-03-16T09:15:00.911978+0000","last_became_active":"2024-03-16T09:13:37.130428+0000","last_became_peered":"2024-03-16T09:13:37.130428+0000","last_unstale":"2024-03-16T09:15:00.911978+0000","last_undegraded":"2024-03-16T09:15:00.911978+0000","last_fullsized":"2024-03-16T09:15:00.911978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:32:33.185589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.006081+0000","last_change":"2024-03-16T09:14:32.006081+0000","last_active":"2024-03-16T09:14:32.006081+0000","last_peered":"2024-03-16T09:14:32.006081+0000","last_clean":"2024-03-16T09:14:32.006081+0000","last_became_active":"2024-03-16T09:14:32.000165+0000","last_became_peered":"2024-03-16T09:14:32.000165+0000","last_unstale":"2024-03-16T09:14:32.006081+0000","last_undegraded":"2024-03-16T09:14:32.006081+0000","last_fullsized":"2024-03-16T09:14:32.006081+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:12:54.685988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.918321+0000","last_change":"2024-03-16T09:13:37.124946+0000","last_active":"2024-03-16T09:15:00.918321+0000","last_peered":"2024-03-16T09:15:00.918321+0000","last_clean":"2024-03-16T09:15:00.918321+0000","last_became_active":"2024-03-16T09:13:37.124624+0000","last_became_peered":"2024-03-16T09:13:37.124624+0000","last_unstale":"2024-03-16T09:15:00.918321+0000","last_undegraded":"2024-03-16T09:15:00.918321+0000","last_fullsized":"2024-03-16T09:15:00.918321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:18:15.188823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517217+0000","last_change":"2024-03-16T09:13:36.116312+0000","last_active":"2024-03-16T09:14:32.517217+0000","last_peered":"2024-03-16T09:14:32.517217+0000","last_clean":"2024-03-16T09:14:32.517217+0000","last_became_active":"2024-03-16T09:13:36.115999+0000","last_became_peered":"2024-03-16T09:13:36.115999+0000","last_unstale":"2024-03-16T09:14:32.517217+0000","last_undegraded":"2024-03-16T09:14:32.517217+0000","last_fullsized":"2024-03-16T09:14:32.517217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:04:46.795291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.933147+0000","last_change":"2024-03-16T09:13:37.126024+0000","last_active":"2024-03-16T09:15:00.933147+0000","last_peered":"2024-03-16T09:15:00.933147+0000","last_clean":"2024-03-16T09:15:00.933147+0000","last_became_active":"2024-03-16T09:13:37.125722+0000","last_became_peered":"2024-03-16T09:13:37.125722+0000","last_unstale":"2024-03-16T09:15:00.933147+0000","last_undegraded":"2024-03-16T09:15:00.933147+0000","last_fullsized":"2024-03-16T09:15:00.933147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:27:24.923022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.005879+0000","last_change":"2024-03-16T09:14:32.005295+0000","last_active":"2024-03-16T09:14:32.005879+0000","last_peered":"2024-03-16T09:14:32.005879+0000","last_clean":"2024-03-16T09:14:32.005879+0000","last_became_active":"2024-03-16T09:14:31.999981+0000","last_became_peered":"2024-03-16T09:14:31.999981+0000","last_unstale":"2024-03-16T09:14:32.005879+0000","last_undegraded":"2024-03-16T09:14:32.005879+0000","last_fullsized":"2024-03-16T09:14:32.005879+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:13:32.661689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.932390+0000","last_change":"2024-03-16T09:13:37.127593+0000","last_active":"2024-03-16T09:15:00.932390+0000","last_peered":"2024-03-16T09:15:00.932390+0000","last_clean":"2024-03-16T09:15:00.932390+0000","last_became_active":"2024-03-16T09:13:37.127255+0000","last_became_peered":"2024-03-16T09:13:37.127255+0000","last_unstale":"2024-03-16T09:15:00.932390+0000","last_undegraded":"2024-03-16T09:15:00.932390+0000","last_fullsized":"2024-03-16T09:15:00.932390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:07:51.263266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.995143+0000","last_change":"2024-03-16T09:13:36.117583+0000","last_active":"2024-03-16T09:14:31.995143+0000","last_peered":"2024-03-16T09:14:31.995143+0000","last_clean":"2024-03-16T09:14:31.995143+0000","last_became_active":"2024-03-16T09:13:36.117270+0000","last_became_peered":"2024-03-16T09:13:36.117270+0000","last_unstale":"2024-03-16T09:14:31.995143+0000","last_undegraded":"2024-03-16T09:14:31.995143+0000","last_fullsized":"2024-03-16T09:14:31.995143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:10:53.541872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.924161+0000","last_change":"2024-03-16T09:13:37.119019+0000","last_active":"2024-03-16T09:15:00.924161+0000","last_peered":"2024-03-16T09:15:00.924161+0000","last_clean":"2024-03-16T09:15:00.924161+0000","last_became_active":"2024-03-16T09:13:37.118148+0000","last_became_peered":"2024-03-16T09:13:37.118148+0000","last_unstale":"2024-03-16T09:15:00.924161+0000","last_undegraded":"2024-03-16T09:15:00.924161+0000","last_fullsized":"2024-03-16T09:15:00.924161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:17:43.042779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":143,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:35.446937+0000","last_change":"2024-03-16T09:13:36.114415+0000","last_active":"2024-03-16T09:15:35.446937+0000","last_peered":"2024-03-16T09:15:35.446937+0000","last_clean":"2024-03-16T09:15:35.446937+0000","last_became_active":"2024-03-16T09:13:36.114056+0000","last_became_peered":"2024-03-16T09:13:36.114056+0000","last_unstale":"2024-03-16T09:15:35.446937+0000","last_undegraded":"2024-03-16T09:15:35.446937+0000","last_fullsized":"2024-03-16T09:15:35.446937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:50:28.322627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.904292+0000","last_change":"2024-03-16T09:13:37.124997+0000","last_active":"2024-03-16T09:15:00.904292+0000","last_peered":"2024-03-16T09:15:00.904292+0000","last_clean":"2024-03-16T09:15:00.904292+0000","last_became_active":"2024-03-16T09:13:37.124664+0000","last_became_peered":"2024-03-16T09:13:37.124664+0000","last_unstale":"2024-03-16T09:15:00.904292+0000","last_undegraded":"2024-03-16T09:15:00.904292+0000","last_fullsized":"2024-03-16T09:15:00.904292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:24:30.104576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.230143+0000","last_change":"2024-03-16T09:13:36.121980+0000","last_active":"2024-03-16T09:14:32.230143+0000","last_peered":"2024-03-16T09:14:32.230143+0000","last_clean":"2024-03-16T09:14:32.230143+0000","last_became_active":"2024-03-16T09:13:36.121703+0000","last_became_peered":"2024-03-16T09:13:36.121703+0000","last_unstale":"2024-03-16T09:14:32.230143+0000","last_undegraded":"2024-03-16T09:14:32.230143+0000","last_fullsized":"2024-03-16T09:14:32.230143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:40:26.601688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.920701+0000","last_change":"2024-03-16T09:13:37.123443+0000","last_active":"2024-03-16T09:15:00.920701+0000","last_peered":"2024-03-16T09:15:00.920701+0000","last_clean":"2024-03-16T09:15:00.920701+0000","last_became_active":"2024-03-16T09:13:37.121987+0000","last_became_peered":"2024-03-16T09:13:37.121987+0000","last_unstale":"2024-03-16T09:15:00.920701+0000","last_undegraded":"2024-03-16T09:15:00.920701+0000","last_fullsized":"2024-03-16T09:15:00.920701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:37:08.478552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385000+0000","last_change":"2024-03-16T09:13:36.106391+0000","last_active":"2024-03-16T09:14:32.385000+0000","last_peered":"2024-03-16T09:14:32.385000+0000","last_clean":"2024-03-16T09:14:32.385000+0000","last_became_active":"2024-03-16T09:13:36.106046+0000","last_became_peered":"2024-03-16T09:13:36.106046+0000","last_unstale":"2024-03-16T09:14:32.385000+0000","last_undegraded":"2024-03-16T09:14:32.385000+0000","last_fullsized":"2024-03-16T09:14:32.385000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:03:23.255519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.911156+0000","last_change":"2024-03-16T09:13:37.114337+0000","last_active":"2024-03-16T09:15:00.911156+0000","last_peered":"2024-03-16T09:15:00.911156+0000","last_clean":"2024-03-16T09:15:00.911156+0000","last_became_active":"2024-03-16T09:13:37.113991+0000","last_became_peered":"2024-03-16T09:13:37.113991+0000","last_unstale":"2024-03-16T09:15:00.911156+0000","last_undegraded":"2024-03-16T09:15:00.911156+0000","last_fullsized":"2024-03-16T09:15:00.911156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:23:54.141479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248113+0000","last_change":"2024-03-16T09:13:36.113297+0000","last_active":"2024-03-16T09:14:32.248113+0000","last_peered":"2024-03-16T09:14:32.248113+0000","last_clean":"2024-03-16T09:14:32.248113+0000","last_became_active":"2024-03-16T09:13:36.112950+0000","last_became_peered":"2024-03-16T09:13:36.112950+0000","last_unstale":"2024-03-16T09:14:32.248113+0000","last_undegraded":"2024-03-16T09:14:32.248113+0000","last_fullsized":"2024-03-16T09:14:32.248113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:40:17.682450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.922064+0000","last_change":"2024-03-16T09:13:37.112239+0000","last_active":"2024-03-16T09:15:00.922064+0000","last_peered":"2024-03-16T09:15:00.922064+0000","last_clean":"2024-03-16T09:15:00.922064+0000","last_became_active":"2024-03-16T09:13:37.111885+0000","last_became_peered":"2024-03-16T09:13:37.111885+0000","last_unstale":"2024-03-16T09:15:00.922064+0000","last_undegraded":"2024-03-16T09:15:00.922064+0000","last_fullsized":"2024-03-16T09:15:00.922064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:49.565846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":138,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:34.445555+0000","last_change":"2024-03-16T09:13:36.111341+0000","last_active":"2024-03-16T09:15:34.445555+0000","last_peered":"2024-03-16T09:15:34.445555+0000","last_clean":"2024-03-16T09:15:34.445555+0000","last_became_active":"2024-03-16T09:13:36.110558+0000","last_became_peered":"2024-03-16T09:13:36.110558+0000","last_unstale":"2024-03-16T09:15:34.445555+0000","last_undegraded":"2024-03-16T09:15:34.445555+0000","last_fullsized":"2024-03-16T09:15:34.445555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:56:44.252002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.933077+0000","last_change":"2024-03-16T09:13:37.126808+0000","last_active":"2024-03-16T09:15:00.933077+0000","last_peered":"2024-03-16T09:15:00.933077+0000","last_clean":"2024-03-16T09:15:00.933077+0000","last_became_active":"2024-03-16T09:13:37.126442+0000","last_became_peered":"2024-03-16T09:13:37.126442+0000","last_unstale":"2024-03-16T09:15:00.933077+0000","last_undegraded":"2024-03-16T09:15:00.933077+0000","last_fullsized":"2024-03-16T09:15:00.933077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:04:39.216835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.004330+0000","last_change":"2024-03-16T09:14:32.004330+0000","last_active":"2024-03-16T09:14:32.004330+0000","last_peered":"2024-03-16T09:14:32.004330+0000","last_clean":"2024-03-16T09:14:32.004330+0000","last_became_active":"2024-03-16T09:14:32.001741+0000","last_became_peered":"2024-03-16T09:14:32.001741+0000","last_unstale":"2024-03-16T09:14:32.004330+0000","last_undegraded":"2024-03-16T09:14:32.004330+0000","last_fullsized":"2024-03-16T09:14:32.004330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:08:19.002873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.921799+0000","last_change":"2024-03-16T09:13:37.125833+0000","last_active":"2024-03-16T09:15:00.921799+0000","last_peered":"2024-03-16T09:15:00.921799+0000","last_clean":"2024-03-16T09:15:00.921799+0000","last_became_active":"2024-03-16T09:13:37.125508+0000","last_became_peered":"2024-03-16T09:13:37.125508+0000","last_unstale":"2024-03-16T09:15:00.921799+0000","last_undegraded":"2024-03-16T09:15:00.921799+0000","last_fullsized":"2024-03-16T09:15:00.921799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:14:47.749715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248156+0000","last_change":"2024-03-16T09:13:36.112200+0000","last_active":"2024-03-16T09:14:32.248156+0000","last_peered":"2024-03-16T09:14:32.248156+0000","last_clean":"2024-03-16T09:14:32.248156+0000","last_became_active":"2024-03-16T09:13:36.111830+0000","last_became_peered":"2024-03-16T09:13:36.111830+0000","last_unstale":"2024-03-16T09:14:32.248156+0000","last_undegraded":"2024-03-16T09:14:32.248156+0000","last_fullsized":"2024-03-16T09:14:32.248156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:07:57.147960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.934446+0000","last_change":"2024-03-16T09:13:37.118434+0000","last_active":"2024-03-16T09:15:00.934446+0000","last_peered":"2024-03-16T09:15:00.934446+0000","last_clean":"2024-03-16T09:15:00.934446+0000","last_became_active":"2024-03-16T09:13:37.118069+0000","last_became_peered":"2024-03-16T09:13:37.118069+0000","last_unstale":"2024-03-16T09:15:00.934446+0000","last_undegraded":"2024-03-16T09:15:00.934446+0000","last_fullsized":"2024-03-16T09:15:00.934446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:02:56.559384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'32","reported_seq":292,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:35.447044+0000","last_change":"2024-03-16T09:13:36.114591+0000","last_active":"2024-03-16T09:15:35.447044+0000","last_peered":"2024-03-16T09:15:35.447044+0000","last_clean":"2024-03-16T09:15:35.447044+0000","last_became_active":"2024-03-16T09:13:36.114252+0000","last_became_peered":"2024-03-16T09:13:36.114252+0000","last_unstale":"2024-03-16T09:15:35.447044+0000","last_undegraded":"2024-03-16T09:15:35.447044+0000","last_fullsized":"2024-03-16T09:15:35.447044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:37:52.637394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60022,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":241,"num_read_kb":179,"num_write":33,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.913494+0000","last_change":"2024-03-16T09:13:37.133102+0000","last_active":"2024-03-16T09:15:00.913494+0000","last_peered":"2024-03-16T09:15:00.913494+0000","last_clean":"2024-03-16T09:15:00.913494+0000","last_became_active":"2024-03-16T09:13:37.132728+0000","last_became_peered":"2024-03-16T09:13:37.132728+0000","last_unstale":"2024-03-16T09:15:00.913494+0000","last_undegraded":"2024-03-16T09:15:00.913494+0000","last_fullsized":"2024-03-16T09:15:00.913494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:54:27.511997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998770+0000","last_change":"2024-03-16T09:13:36.114195+0000","last_active":"2024-03-16T09:14:31.998770+0000","last_peered":"2024-03-16T09:14:31.998770+0000","last_clean":"2024-03-16T09:14:31.998770+0000","last_became_active":"2024-03-16T09:13:36.113829+0000","last_became_peered":"2024-03-16T09:13:36.113829+0000","last_unstale":"2024-03-16T09:14:31.998770+0000","last_undegraded":"2024-03-16T09:14:31.998770+0000","last_fullsized":"2024-03-16T09:14:31.998770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:28:40.509877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.912363+0000","last_change":"2024-03-16T09:13:37.132974+0000","last_active":"2024-03-16T09:15:00.912363+0000","last_peered":"2024-03-16T09:15:00.912363+0000","last_clean":"2024-03-16T09:15:00.912363+0000","last_became_active":"2024-03-16T09:13:37.132474+0000","last_became_peered":"2024-03-16T09:13:37.132474+0000","last_unstale":"2024-03-16T09:15:00.912363+0000","last_undegraded":"2024-03-16T09:15:00.912363+0000","last_fullsized":"2024-03-16T09:15:00.912363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:58:04.480441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.247992+0000","last_change":"2024-03-16T09:13:36.115242+0000","last_active":"2024-03-16T09:14:32.247992+0000","last_peered":"2024-03-16T09:14:32.247992+0000","last_clean":"2024-03-16T09:14:32.247992+0000","last_became_active":"2024-03-16T09:13:36.113566+0000","last_became_peered":"2024-03-16T09:13:36.113566+0000","last_unstale":"2024-03-16T09:14:32.247992+0000","last_undegraded":"2024-03-16T09:14:32.247992+0000","last_fullsized":"2024-03-16T09:14:32.247992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:47:50.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.927334+0000","last_change":"2024-03-16T09:14:32.386530+0000","last_active":"2024-03-16T09:15:00.927334+0000","last_peered":"2024-03-16T09:15:00.927334+0000","last_clean":"2024-03-16T09:15:00.927334+0000","last_became_active":"2024-03-16T09:14:32.386088+0000","last_became_peered":"2024-03-16T09:14:32.386088+0000","last_unstale":"2024-03-16T09:15:00.927334+0000","last_undegraded":"2024-03-16T09:15:00.927334+0000","last_fullsized":"2024-03-16T09:15:00.927334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:44:57.427154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":144,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:36.457841+0000","last_change":"2024-03-16T09:13:36.109585+0000","last_active":"2024-03-16T09:15:36.457841+0000","last_peered":"2024-03-16T09:15:36.457841+0000","last_clean":"2024-03-16T09:15:36.457841+0000","last_became_active":"2024-03-16T09:13:36.109267+0000","last_became_peered":"2024-03-16T09:13:36.109267+0000","last_unstale":"2024-03-16T09:15:36.457841+0000","last_undegraded":"2024-03-16T09:15:36.457841+0000","last_fullsized":"2024-03-16T09:15:36.457841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:26:55.700791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.926195+0000","last_change":"2024-03-16T09:13:37.136465+0000","last_active":"2024-03-16T09:15:00.926195+0000","last_peered":"2024-03-16T09:15:00.926195+0000","last_clean":"2024-03-16T09:15:00.926195+0000","last_became_active":"2024-03-16T09:13:37.135933+0000","last_became_peered":"2024-03-16T09:13:37.135933+0000","last_unstale":"2024-03-16T09:15:00.926195+0000","last_undegraded":"2024-03-16T09:15:00.926195+0000","last_fullsized":"2024-03-16T09:15:00.926195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:08:28.490533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994732+0000","last_change":"2024-03-16T09:13:36.119428+0000","last_active":"2024-03-16T09:14:31.994732+0000","last_peered":"2024-03-16T09:14:31.994732+0000","last_clean":"2024-03-16T09:14:31.994732+0000","last_became_active":"2024-03-16T09:13:36.119083+0000","last_became_peered":"2024-03-16T09:13:36.119083+0000","last_unstale":"2024-03-16T09:14:31.994732+0000","last_undegraded":"2024-03-16T09:14:31.994732+0000","last_fullsized":"2024-03-16T09:14:31.994732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:58:56.065151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.920460+0000","last_change":"2024-03-16T09:13:37.118082+0000","last_active":"2024-03-16T09:15:00.920460+0000","last_peered":"2024-03-16T09:15:00.920460+0000","last_clean":"2024-03-16T09:15:00.920460+0000","last_became_active":"2024-03-16T09:13:37.117671+0000","last_became_peered":"2024-03-16T09:13:37.117671+0000","last_unstale":"2024-03-16T09:15:00.920460+0000","last_undegraded":"2024-03-16T09:15:00.920460+0000","last_fullsized":"2024-03-16T09:15:00.920460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:14:05.352191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.995286+0000","last_change":"2024-03-16T09:13:36.121077+0000","last_active":"2024-03-16T09:14:31.995286+0000","last_peered":"2024-03-16T09:14:31.995286+0000","last_clean":"2024-03-16T09:14:31.995286+0000","last_became_active":"2024-03-16T09:13:36.120723+0000","last_became_peered":"2024-03-16T09:13:36.120723+0000","last_unstale":"2024-03-16T09:14:31.995286+0000","last_undegraded":"2024-03-16T09:14:31.995286+0000","last_fullsized":"2024-03-16T09:14:31.995286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:58:14.746215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.922037+0000","last_change":"2024-03-16T09:13:37.116725+0000","last_active":"2024-03-16T09:15:00.922037+0000","last_peered":"2024-03-16T09:15:00.922037+0000","last_clean":"2024-03-16T09:15:00.922037+0000","last_became_active":"2024-03-16T09:13:37.116406+0000","last_became_peered":"2024-03-16T09:13:37.116406+0000","last_unstale":"2024-03-16T09:15:00.922037+0000","last_undegraded":"2024-03-16T09:15:00.922037+0000","last_fullsized":"2024-03-16T09:15:00.922037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:56:59.136649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996706+0000","last_change":"2024-03-16T09:13:36.110568+0000","last_active":"2024-03-16T09:14:31.996706+0000","last_peered":"2024-03-16T09:14:31.996706+0000","last_clean":"2024-03-16T09:14:31.996706+0000","last_became_active":"2024-03-16T09:13:36.110246+0000","last_became_peered":"2024-03-16T09:13:36.110246+0000","last_unstale":"2024-03-16T09:14:31.996706+0000","last_undegraded":"2024-03-16T09:14:31.996706+0000","last_fullsized":"2024-03-16T09:14:31.996706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:36:50.011941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.921527+0000","last_change":"2024-03-16T09:13:37.122296+0000","last_active":"2024-03-16T09:15:00.921527+0000","last_peered":"2024-03-16T09:15:00.921527+0000","last_clean":"2024-03-16T09:15:00.921527+0000","last_became_active":"2024-03-16T09:13:37.121968+0000","last_became_peered":"2024-03-16T09:13:37.121968+0000","last_unstale":"2024-03-16T09:15:00.921527+0000","last_undegraded":"2024-03-16T09:15:00.921527+0000","last_fullsized":"2024-03-16T09:15:00.921527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:10:43.449397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997895+0000","last_change":"2024-03-16T09:13:36.112262+0000","last_active":"2024-03-16T09:14:31.997895+0000","last_peered":"2024-03-16T09:14:31.997895+0000","last_clean":"2024-03-16T09:14:31.997895+0000","last_became_active":"2024-03-16T09:13:36.111916+0000","last_became_peered":"2024-03-16T09:13:36.111916+0000","last_unstale":"2024-03-16T09:14:31.997895+0000","last_undegraded":"2024-03-16T09:14:31.997895+0000","last_fullsized":"2024-03-16T09:14:31.997895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:27:41.163508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.906629+0000","last_change":"2024-03-16T09:13:37.119814+0000","last_active":"2024-03-16T09:15:00.906629+0000","last_peered":"2024-03-16T09:15:00.906629+0000","last_clean":"2024-03-16T09:15:00.906629+0000","last_became_active":"2024-03-16T09:13:37.119454+0000","last_became_peered":"2024-03-16T09:13:37.119454+0000","last_unstale":"2024-03-16T09:15:00.906629+0000","last_undegraded":"2024-03-16T09:15:00.906629+0000","last_fullsized":"2024-03-16T09:15:00.906629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:57:25.988044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.904133+0000","last_change":"2024-03-16T09:14:31.999902+0000","last_active":"2024-03-16T09:15:00.904133+0000","last_peered":"2024-03-16T09:15:00.904133+0000","last_clean":"2024-03-16T09:15:00.904133+0000","last_became_active":"2024-03-16T09:14:31.999609+0000","last_became_peered":"2024-03-16T09:14:31.999609+0000","last_unstale":"2024-03-16T09:15:00.904133+0000","last_undegraded":"2024-03-16T09:15:00.904133+0000","last_fullsized":"2024-03-16T09:15:00.904133+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:58:47.495746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":142,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:36.456168+0000","last_change":"2024-03-16T09:13:36.111300+0000","last_active":"2024-03-16T09:15:36.456168+0000","last_peered":"2024-03-16T09:15:36.456168+0000","last_clean":"2024-03-16T09:15:36.456168+0000","last_became_active":"2024-03-16T09:13:36.110966+0000","last_became_peered":"2024-03-16T09:13:36.110966+0000","last_unstale":"2024-03-16T09:15:36.456168+0000","last_undegraded":"2024-03-16T09:15:36.456168+0000","last_fullsized":"2024-03-16T09:15:36.456168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:18:33.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.931071+0000","last_change":"2024-03-16T09:13:37.125791+0000","last_active":"2024-03-16T09:15:00.931071+0000","last_peered":"2024-03-16T09:15:00.931071+0000","last_clean":"2024-03-16T09:15:00.931071+0000","last_became_active":"2024-03-16T09:13:37.125448+0000","last_became_peered":"2024-03-16T09:13:37.125448+0000","last_unstale":"2024-03-16T09:15:00.931071+0000","last_undegraded":"2024-03-16T09:15:00.931071+0000","last_fullsized":"2024-03-16T09:15:00.931071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:51:30.767750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996257+0000","last_change":"2024-03-16T09:13:36.107462+0000","last_active":"2024-03-16T09:14:31.996257+0000","last_peered":"2024-03-16T09:14:31.996257+0000","last_clean":"2024-03-16T09:14:31.996257+0000","last_became_active":"2024-03-16T09:13:36.107115+0000","last_became_peered":"2024-03-16T09:13:36.107115+0000","last_unstale":"2024-03-16T09:14:31.996257+0000","last_undegraded":"2024-03-16T09:14:31.996257+0000","last_fullsized":"2024-03-16T09:14:31.996257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:37.796719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997730+0000","last_change":"2024-03-16T09:12:24.331159+0000","last_active":"2024-03-16T09:14:31.997730+0000","last_peered":"2024-03-16T09:14:31.997730+0000","last_clean":"2024-03-16T09:14:31.997730+0000","last_became_active":"2024-03-16T09:12:24.330554+0000","last_became_peered":"2024-03-16T09:12:24.330554+0000","last_unstale":"2024-03-16T09:14:31.997730+0000","last_undegraded":"2024-03-16T09:14:31.997730+0000","last_fullsized":"2024-03-16T09:14:31.997730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:08:57.697012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:08:57.697012+0000","last_clean_scrub_stamp":"2024-03-16T09:08:57.697012+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:43:02.986865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.909651+0000","last_change":"2024-03-16T09:13:37.114299+0000","last_active":"2024-03-16T09:15:00.909651+0000","last_peered":"2024-03-16T09:15:00.909651+0000","last_clean":"2024-03-16T09:15:00.909651+0000","last_became_active":"2024-03-16T09:13:37.113981+0000","last_became_peered":"2024-03-16T09:13:37.113981+0000","last_unstale":"2024-03-16T09:15:00.909651+0000","last_undegraded":"2024-03-16T09:15:00.909651+0000","last_fullsized":"2024-03-16T09:15:00.909651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:17:15.882130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997873+0000","last_change":"2024-03-16T09:13:36.113962+0000","last_active":"2024-03-16T09:14:31.997873+0000","last_peered":"2024-03-16T09:14:31.997873+0000","last_clean":"2024-03-16T09:14:31.997873+0000","last_became_active":"2024-03-16T09:13:36.113616+0000","last_became_peered":"2024-03-16T09:13:36.113616+0000","last_unstale":"2024-03-16T09:14:31.997873+0000","last_undegraded":"2024-03-16T09:14:31.997873+0000","last_fullsized":"2024-03-16T09:14:31.997873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:15:15.004360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.995031+0000","last_change":"2024-03-16T09:13:36.121946+0000","last_active":"2024-03-16T09:14:31.995031+0000","last_peered":"2024-03-16T09:14:31.995031+0000","last_clean":"2024-03-16T09:14:31.995031+0000","last_became_active":"2024-03-16T09:13:36.121209+0000","last_became_peered":"2024-03-16T09:13:36.121209+0000","last_unstale":"2024-03-16T09:14:31.995031+0000","last_undegraded":"2024-03-16T09:14:31.995031+0000","last_fullsized":"2024-03-16T09:14:31.995031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:09:51.139468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.908991+0000","last_change":"2024-03-16T09:13:37.128931+0000","last_active":"2024-03-16T09:15:00.908991+0000","last_peered":"2024-03-16T09:15:00.908991+0000","last_clean":"2024-03-16T09:15:00.908991+0000","last_became_active":"2024-03-16T09:13:37.128515+0000","last_became_peered":"2024-03-16T09:13:37.128515+0000","last_unstale":"2024-03-16T09:15:00.908991+0000","last_undegraded":"2024-03-16T09:15:00.908991+0000","last_fullsized":"2024-03-16T09:15:00.908991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:58:46.429144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:01.126760+0000","last_change":"2024-03-16T09:13:36.118831+0000","last_active":"2024-03-16T09:15:01.126760+0000","last_peered":"2024-03-16T09:15:01.126760+0000","last_clean":"2024-03-16T09:15:01.126760+0000","last_became_active":"2024-03-16T09:13:36.118489+0000","last_became_peered":"2024-03-16T09:13:36.118489+0000","last_unstale":"2024-03-16T09:15:01.126760+0000","last_undegraded":"2024-03-16T09:15:01.126760+0000","last_fullsized":"2024-03-16T09:15:01.126760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:55:34.089451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.923760+0000","last_change":"2024-03-16T09:13:37.120581+0000","last_active":"2024-03-16T09:15:00.923760+0000","last_peered":"2024-03-16T09:15:00.923760+0000","last_clean":"2024-03-16T09:15:00.923760+0000","last_became_active":"2024-03-16T09:13:37.120266+0000","last_became_peered":"2024-03-16T09:13:37.120266+0000","last_unstale":"2024-03-16T09:15:00.923760+0000","last_undegraded":"2024-03-16T09:15:00.923760+0000","last_fullsized":"2024-03-16T09:15:00.923760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:16:10.784229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996414+0000","last_change":"2024-03-16T09:13:36.107125+0000","last_active":"2024-03-16T09:14:31.996414+0000","last_peered":"2024-03-16T09:14:31.996414+0000","last_clean":"2024-03-16T09:14:31.996414+0000","last_became_active":"2024-03-16T09:13:36.106795+0000","last_became_peered":"2024-03-16T09:13:36.106795+0000","last_unstale":"2024-03-16T09:14:31.996414+0000","last_undegraded":"2024-03-16T09:14:31.996414+0000","last_fullsized":"2024-03-16T09:14:31.996414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:29:10.130565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.922341+0000","last_change":"2024-03-16T09:13:37.123013+0000","last_active":"2024-03-16T09:15:00.922341+0000","last_peered":"2024-03-16T09:15:00.922341+0000","last_clean":"2024-03-16T09:15:00.922341+0000","last_became_active":"2024-03-16T09:13:37.122642+0000","last_became_peered":"2024-03-16T09:13:37.122642+0000","last_unstale":"2024-03-16T09:15:00.922341+0000","last_undegraded":"2024-03-16T09:15:00.922341+0000","last_fullsized":"2024-03-16T09:15:00.922341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:35:05.297337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.000335+0000","last_change":"2024-03-16T09:13:36.110868+0000","last_active":"2024-03-16T09:14:32.000335+0000","last_peered":"2024-03-16T09:14:32.000335+0000","last_clean":"2024-03-16T09:14:32.000335+0000","last_became_active":"2024-03-16T09:13:36.110536+0000","last_became_peered":"2024-03-16T09:13:36.110536+0000","last_unstale":"2024-03-16T09:14:32.000335+0000","last_undegraded":"2024-03-16T09:14:32.000335+0000","last_fullsized":"2024-03-16T09:14:32.000335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:07:06.829458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.882568+0000","last_change":"2024-03-16T09:13:37.116579+0000","last_active":"2024-03-16T09:15:00.882568+0000","last_peered":"2024-03-16T09:15:00.882568+0000","last_clean":"2024-03-16T09:15:00.882568+0000","last_became_active":"2024-03-16T09:13:37.116226+0000","last_became_peered":"2024-03-16T09:13:37.116226+0000","last_unstale":"2024-03-16T09:15:00.882568+0000","last_undegraded":"2024-03-16T09:15:00.882568+0000","last_fullsized":"2024-03-16T09:15:00.882568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:08:39.015162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517053+0000","last_change":"2024-03-16T09:13:36.107881+0000","last_active":"2024-03-16T09:14:32.517053+0000","last_peered":"2024-03-16T09:14:32.517053+0000","last_clean":"2024-03-16T09:14:32.517053+0000","last_became_active":"2024-03-16T09:13:36.107538+0000","last_became_peered":"2024-03-16T09:13:36.107538+0000","last_unstale":"2024-03-16T09:14:32.517053+0000","last_undegraded":"2024-03-16T09:14:32.517053+0000","last_fullsized":"2024-03-16T09:14:32.517053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:49:41.932089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.923202+0000","last_change":"2024-03-16T09:13:37.118580+0000","last_active":"2024-03-16T09:15:00.923202+0000","last_peered":"2024-03-16T09:15:00.923202+0000","last_clean":"2024-03-16T09:15:00.923202+0000","last_became_active":"2024-03-16T09:13:37.118166+0000","last_became_peered":"2024-03-16T09:13:37.118166+0000","last_unstale":"2024-03-16T09:15:00.923202+0000","last_undegraded":"2024-03-16T09:15:00.923202+0000","last_fullsized":"2024-03-16T09:15:00.923202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:10:13.957439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998331+0000","last_change":"2024-03-16T09:13:36.122307+0000","last_active":"2024-03-16T09:14:31.998331+0000","last_peered":"2024-03-16T09:14:31.998331+0000","last_clean":"2024-03-16T09:14:31.998331+0000","last_became_active":"2024-03-16T09:13:36.121961+0000","last_became_peered":"2024-03-16T09:13:36.121961+0000","last_unstale":"2024-03-16T09:14:31.998331+0000","last_undegraded":"2024-03-16T09:14:31.998331+0000","last_fullsized":"2024-03-16T09:14:31.998331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:50:03.482300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.921973+0000","last_change":"2024-03-16T09:13:37.131918+0000","last_active":"2024-03-16T09:15:00.921973+0000","last_peered":"2024-03-16T09:15:00.921973+0000","last_clean":"2024-03-16T09:15:00.921973+0000","last_became_active":"2024-03-16T09:13:37.131560+0000","last_became_peered":"2024-03-16T09:13:37.131560+0000","last_unstale":"2024-03-16T09:15:00.921973+0000","last_undegraded":"2024-03-16T09:15:00.921973+0000","last_fullsized":"2024-03-16T09:15:00.921973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:57.315125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994873+0000","last_change":"2024-03-16T09:13:36.120192+0000","last_active":"2024-03-16T09:14:31.994873+0000","last_peered":"2024-03-16T09:14:31.994873+0000","last_clean":"2024-03-16T09:14:31.994873+0000","last_became_active":"2024-03-16T09:13:36.119063+0000","last_became_peered":"2024-03-16T09:13:36.119063+0000","last_unstale":"2024-03-16T09:14:31.994873+0000","last_undegraded":"2024-03-16T09:14:31.994873+0000","last_fullsized":"2024-03-16T09:14:31.994873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:18:56.616624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.935639+0000","last_change":"2024-03-16T09:13:37.129597+0000","last_active":"2024-03-16T09:15:00.935639+0000","last_peered":"2024-03-16T09:15:00.935639+0000","last_clean":"2024-03-16T09:15:00.935639+0000","last_became_active":"2024-03-16T09:13:37.129248+0000","last_became_peered":"2024-03-16T09:13:37.129248+0000","last_unstale":"2024-03-16T09:15:00.935639+0000","last_undegraded":"2024-03-16T09:15:00.935639+0000","last_fullsized":"2024-03-16T09:15:00.935639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:41:11.118192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.010274+0000","last_change":"2024-03-16T09:14:32.009577+0000","last_active":"2024-03-16T09:14:32.010274+0000","last_peered":"2024-03-16T09:14:32.010274+0000","last_clean":"2024-03-16T09:14:32.010274+0000","last_became_active":"2024-03-16T09:14:32.000521+0000","last_became_peered":"2024-03-16T09:14:32.000521+0000","last_unstale":"2024-03-16T09:14:32.010274+0000","last_undegraded":"2024-03-16T09:14:32.010274+0000","last_fullsized":"2024-03-16T09:14:32.010274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:29:07.221252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.935373+0000","last_change":"2024-03-16T09:13:37.120317+0000","last_active":"2024-03-16T09:15:00.935373+0000","last_peered":"2024-03-16T09:15:00.935373+0000","last_clean":"2024-03-16T09:15:00.935373+0000","last_became_active":"2024-03-16T09:13:37.119974+0000","last_became_peered":"2024-03-16T09:13:37.119974+0000","last_unstale":"2024-03-16T09:15:00.935373+0000","last_undegraded":"2024-03-16T09:15:00.935373+0000","last_fullsized":"2024-03-16T09:15:00.935373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:25:34.774377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248531+0000","last_change":"2024-03-16T09:13:36.112395+0000","last_active":"2024-03-16T09:14:32.248531+0000","last_peered":"2024-03-16T09:14:32.248531+0000","last_clean":"2024-03-16T09:14:32.248531+0000","last_became_active":"2024-03-16T09:13:36.112036+0000","last_became_peered":"2024-03-16T09:13:36.112036+0000","last_unstale":"2024-03-16T09:14:32.248531+0000","last_undegraded":"2024-03-16T09:14:32.248531+0000","last_fullsized":"2024-03-16T09:14:32.248531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:41:02.895820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.923449+0000","last_change":"2024-03-16T09:13:37.133845+0000","last_active":"2024-03-16T09:15:00.923449+0000","last_peered":"2024-03-16T09:15:00.923449+0000","last_clean":"2024-03-16T09:15:00.923449+0000","last_became_active":"2024-03-16T09:13:37.132833+0000","last_became_peered":"2024-03-16T09:13:37.132833+0000","last_unstale":"2024-03-16T09:15:00.923449+0000","last_undegraded":"2024-03-16T09:15:00.923449+0000","last_fullsized":"2024-03-16T09:15:00.923449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:12:18.049210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385012+0000","last_change":"2024-03-16T09:13:36.114830+0000","last_active":"2024-03-16T09:14:32.385012+0000","last_peered":"2024-03-16T09:14:32.385012+0000","last_clean":"2024-03-16T09:14:32.385012+0000","last_became_active":"2024-03-16T09:13:36.113898+0000","last_became_peered":"2024-03-16T09:13:36.113898+0000","last_unstale":"2024-03-16T09:14:32.385012+0000","last_undegraded":"2024-03-16T09:14:32.385012+0000","last_fullsized":"2024-03-16T09:14:32.385012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:27:38.910819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.925474+0000","last_change":"2024-03-16T09:13:37.115991+0000","last_active":"2024-03-16T09:15:00.925474+0000","last_peered":"2024-03-16T09:15:00.925474+0000","last_clean":"2024-03-16T09:15:00.925474+0000","last_became_active":"2024-03-16T09:13:37.115671+0000","last_became_peered":"2024-03-16T09:13:37.115671+0000","last_unstale":"2024-03-16T09:15:00.925474+0000","last_undegraded":"2024-03-16T09:15:00.925474+0000","last_fullsized":"2024-03-16T09:15:00.925474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:05:59.703675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.919368+0000","last_change":"2024-03-16T09:13:37.115122+0000","last_active":"2024-03-16T09:15:00.919368+0000","last_peered":"2024-03-16T09:15:00.919368+0000","last_clean":"2024-03-16T09:15:00.919368+0000","last_became_active":"2024-03-16T09:13:37.114779+0000","last_became_peered":"2024-03-16T09:13:37.114779+0000","last_unstale":"2024-03-16T09:15:00.919368+0000","last_undegraded":"2024-03-16T09:15:00.919368+0000","last_fullsized":"2024-03-16T09:15:00.919368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:50:02.051104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.920923+0000","last_change":"2024-03-16T09:13:37.116542+0000","last_active":"2024-03-16T09:15:00.920923+0000","last_peered":"2024-03-16T09:15:00.920923+0000","last_clean":"2024-03-16T09:15:00.920923+0000","last_became_active":"2024-03-16T09:13:37.116224+0000","last_became_peered":"2024-03-16T09:13:37.116224+0000","last_unstale":"2024-03-16T09:15:00.920923+0000","last_undegraded":"2024-03-16T09:15:00.920923+0000","last_fullsized":"2024-03-16T09:15:00.920923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:32:40.106979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.923128+0000","last_change":"2024-03-16T09:13:37.128750+0000","last_active":"2024-03-16T09:15:00.923128+0000","last_peered":"2024-03-16T09:15:00.923128+0000","last_clean":"2024-03-16T09:15:00.923128+0000","last_became_active":"2024-03-16T09:13:37.128411+0000","last_became_peered":"2024-03-16T09:13:37.128411+0000","last_unstale":"2024-03-16T09:15:00.923128+0000","last_undegraded":"2024-03-16T09:15:00.923128+0000","last_fullsized":"2024-03-16T09:15:00.923128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:01:02.119578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.919187+0000","last_change":"2024-03-16T09:13:37.124024+0000","last_active":"2024-03-16T09:15:00.919187+0000","last_peered":"2024-03-16T09:15:00.919187+0000","last_clean":"2024-03-16T09:15:00.919187+0000","last_became_active":"2024-03-16T09:13:37.123721+0000","last_became_peered":"2024-03-16T09:13:37.123721+0000","last_unstale":"2024-03-16T09:15:00.919187+0000","last_undegraded":"2024-03-16T09:15:00.919187+0000","last_fullsized":"2024-03-16T09:15:00.919187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:29:28.079163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.910102+0000","last_change":"2024-03-16T09:13:37.116603+0000","last_active":"2024-03-16T09:15:00.910102+0000","last_peered":"2024-03-16T09:15:00.910102+0000","last_clean":"2024-03-16T09:15:00.910102+0000","last_became_active":"2024-03-16T09:13:37.116279+0000","last_became_peered":"2024-03-16T09:13:37.116279+0000","last_unstale":"2024-03-16T09:15:00.910102+0000","last_undegraded":"2024-03-16T09:15:00.910102+0000","last_fullsized":"2024-03-16T09:15:00.910102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:25:13.001817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.918731+0000","last_change":"2024-03-16T09:13:37.116863+0000","last_active":"2024-03-16T09:15:00.918731+0000","last_peered":"2024-03-16T09:15:00.918731+0000","last_clean":"2024-03-16T09:15:00.918731+0000","last_became_active":"2024-03-16T09:13:37.116543+0000","last_became_peered":"2024-03-16T09:13:37.116543+0000","last_unstale":"2024-03-16T09:15:00.918731+0000","last_undegraded":"2024-03-16T09:15:00.918731+0000","last_fullsized":"2024-03-16T09:15:00.918731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:06:35.501566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.915096+0000","last_change":"2024-03-16T09:13:37.120325+0000","last_active":"2024-03-16T09:15:00.915096+0000","last_peered":"2024-03-16T09:15:00.915096+0000","last_clean":"2024-03-16T09:15:00.915096+0000","last_became_active":"2024-03-16T09:13:37.119985+0000","last_became_peered":"2024-03-16T09:13:37.119985+0000","last_unstale":"2024-03-16T09:15:00.915096+0000","last_undegraded":"2024-03-16T09:15:00.915096+0000","last_fullsized":"2024-03-16T09:15:00.915096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:35.419225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.908253+0000","last_change":"2024-03-16T09:13:37.119604+0000","last_active":"2024-03-16T09:15:00.908253+0000","last_peered":"2024-03-16T09:15:00.908253+0000","last_clean":"2024-03-16T09:15:00.908253+0000","last_became_active":"2024-03-16T09:13:37.119019+0000","last_became_peered":"2024-03-16T09:13:37.119019+0000","last_unstale":"2024-03-16T09:15:00.908253+0000","last_undegraded":"2024-03-16T09:15:00.908253+0000","last_fullsized":"2024-03-16T09:15:00.908253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:42:57.902370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.915549+0000","last_change":"2024-03-16T09:13:37.123845+0000","last_active":"2024-03-16T09:15:00.915549+0000","last_peered":"2024-03-16T09:15:00.915549+0000","last_clean":"2024-03-16T09:15:00.915549+0000","last_became_active":"2024-03-16T09:13:37.121753+0000","last_became_peered":"2024-03-16T09:13:37.121753+0000","last_unstale":"2024-03-16T09:15:00.915549+0000","last_undegraded":"2024-03-16T09:15:00.915549+0000","last_fullsized":"2024-03-16T09:15:00.915549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:23:01.289378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.910994+0000","last_change":"2024-03-16T09:13:37.118182+0000","last_active":"2024-03-16T09:15:00.910994+0000","last_peered":"2024-03-16T09:15:00.910994+0000","last_clean":"2024-03-16T09:15:00.910994+0000","last_became_active":"2024-03-16T09:13:37.117820+0000","last_became_peered":"2024-03-16T09:13:37.117820+0000","last_unstale":"2024-03-16T09:15:00.910994+0000","last_undegraded":"2024-03-16T09:15:00.910994+0000","last_fullsized":"2024-03-16T09:15:00.910994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:27:31.393266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.935406+0000","last_change":"2024-03-16T09:13:37.121348+0000","last_active":"2024-03-16T09:15:00.935406+0000","last_peered":"2024-03-16T09:15:00.935406+0000","last_clean":"2024-03-16T09:15:00.935406+0000","last_became_active":"2024-03-16T09:13:37.120826+0000","last_became_peered":"2024-03-16T09:13:37.120826+0000","last_unstale":"2024-03-16T09:15:00.935406+0000","last_undegraded":"2024-03-16T09:15:00.935406+0000","last_fullsized":"2024-03-16T09:15:00.935406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:36:33.456230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.919550+0000","last_change":"2024-03-16T09:13:37.131889+0000","last_active":"2024-03-16T09:15:00.919550+0000","last_peered":"2024-03-16T09:15:00.919550+0000","last_clean":"2024-03-16T09:15:00.919550+0000","last_became_active":"2024-03-16T09:13:37.131564+0000","last_became_peered":"2024-03-16T09:13:37.131564+0000","last_unstale":"2024-03-16T09:15:00.919550+0000","last_undegraded":"2024-03-16T09:15:00.919550+0000","last_fullsized":"2024-03-16T09:15:00.919550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:43:34.255378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.913691+0000","last_change":"2024-03-16T09:14:31.998395+0000","last_active":"2024-03-16T09:15:00.913691+0000","last_peered":"2024-03-16T09:15:00.913691+0000","last_clean":"2024-03-16T09:15:00.913691+0000","last_became_active":"2024-03-16T09:14:31.998063+0000","last_became_peered":"2024-03-16T09:14:31.998063+0000","last_unstale":"2024-03-16T09:15:00.913691+0000","last_undegraded":"2024-03-16T09:15:00.913691+0000","last_fullsized":"2024-03-16T09:15:00.913691+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:50:57.125765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280795,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64023,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":728,"num_read_kb":431,"num_write":165,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1164,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":207855,"data_compressed":5547,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383718,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":456779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645354,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":1107700,"data_compressed":37563,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070516,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332154,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":1047310,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":455822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":523921,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":457093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":524689,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":456133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":457232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":1047576,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":68306,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69074,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":61600,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T09:15:39.798 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-16T09:15:39.798 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T09:15:39.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.933+0000 7f3d6ca62640 1 -- 172.21.15.112:0/3609140965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d68102e30 msgr2=0x7f3d68103250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:39.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.933+0000 7f3d6ca62640 1 --2- 172.21.15.112:0/3609140965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d68102e30 0x7f3d68103250 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3d58009860 tx=0x7f3d5802bb90 comp rx=0 tx=0).stop 2024-03-16T09:15:39.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.933+0000 7f3d6ca62640 1 -- 172.21.15.112:0/3609140965 shutdown_connections 2024-03-16T09:15:39.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.933+0000 7f3d6ca62640 1 --2- 172.21.15.112:0/3609140965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d68100260 0x7f3d68109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.933+0000 7f3d6ca62640 1 --2- 172.21.15.112:0/3609140965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d680ff8a0 0x7f3d680ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.933+0000 7f3d6ca62640 1 --2- 172.21.15.112:0/3609140965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d68102e30 0x7f3d68103250 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.933+0000 7f3d6ca62640 1 -- 172.21.15.112:0/3609140965 >> 172.21.15.112:0/3609140965 conn(0x7f3d680fb400 msgr2=0x7f3d680fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:39.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.933+0000 7f3d6ca62640 1 -- 172.21.15.112:0/3609140965 shutdown_connections 2024-03-16T09:15:39.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.933+0000 7f3d6ca62640 1 -- 172.21.15.112:0/3609140965 wait complete. 2024-03-16T09:15:39.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.934+0000 7f3d6ca62640 1 Processor -- start 2024-03-16T09:15:39.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.934+0000 7f3d6ca62640 1 -- start start 2024-03-16T09:15:39.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.934+0000 7f3d6ca62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d680ff8a0 0x7f3d68197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:39.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.934+0000 7f3d6ca62640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d68100260 0x7f3d681981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:39.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.934+0000 7f3d6ca62640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d68102e30 0x7f3d6819f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:39.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.934+0000 7f3d6ca62640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6810d5e0 con 0x7f3d680ff8a0 2024-03-16T09:15:39.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.934+0000 7f3d6ca62640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6810d460 con 0x7f3d68100260 2024-03-16T09:15:39.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.934+0000 7f3d6ca62640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6810d760 con 0x7f3d68102e30 2024-03-16T09:15:39.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.934+0000 7f3d6659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d680ff8a0 0x7f3d68197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:39.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.934+0000 7f3d6659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d680ff8a0 0x7f3d68197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57264/0 (socket says 172.21.15.112:57264) 2024-03-16T09:15:39.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.934+0000 7f3d66d9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d68102e30 0x7f3d6819f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:39.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.934+0000 7f3d6659c640 1 -- 172.21.15.112:0/798946183 learned_addr learned my addr 172.21.15.112:0/798946183 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:39.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.934+0000 7f3d65d9b640 1 --2- 172.21.15.112:0/798946183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d68100260 0x7f3d681981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:39.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.934+0000 7f3d6659c640 1 -- 172.21.15.112:0/798946183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d68102e30 msgr2=0x7f3d6819f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:39.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.934+0000 7f3d6659c640 1 --2- 172.21.15.112:0/798946183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d68102e30 0x7f3d6819f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.935+0000 7f3d6659c640 1 -- 172.21.15.112:0/798946183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d68100260 msgr2=0x7f3d681981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:39.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.935+0000 7f3d6659c640 1 --2- 172.21.15.112:0/798946183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d68100260 0x7f3d681981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:39.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.935+0000 7f3d6659c640 1 -- 172.21.15.112:0/798946183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6819f8b0 con 0x7f3d680ff8a0 2024-03-16T09:15:39.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.935+0000 7f3d66d9d640 1 --2- 172.21.15.112:0/798946183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d68102e30 0x7f3d6819f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:39.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.935+0000 7f3d6659c640 1 --2- 172.21.15.112:0/798946183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d680ff8a0 0x7f3d68197c70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3d58034ac0 tx=0x7f3d580323d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:39.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.935+0000 7f3d577fe640 1 -- 172.21.15.112:0/798946183 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58041020 con 0x7f3d680ff8a0 2024-03-16T09:15:39.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.935+0000 7f3d577fe640 1 -- 172.21.15.112:0/798946183 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3d5802ca30 con 0x7f3d680ff8a0 2024-03-16T09:15:39.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.935+0000 7f3d6ca62640 1 -- 172.21.15.112:0/798946183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6819fb40 con 0x7f3d680ff8a0 2024-03-16T09:15:39.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.935+0000 7f3d577fe640 1 -- 172.21.15.112:0/798946183 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58039040 con 0x7f3d680ff8a0 2024-03-16T09:15:39.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.935+0000 7f3d6ca62640 1 -- 172.21.15.112:0/798946183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d681a0030 con 0x7f3d680ff8a0 2024-03-16T09:15:39.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.936+0000 7f3d6ca62640 1 -- 172.21.15.112:0/798946183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d68108c90 con 0x7f3d680ff8a0 2024-03-16T09:15:39.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.937+0000 7f3d577fe640 1 -- 172.21.15.112:0/798946183 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7f3d58044020 con 0x7f3d680ff8a0 2024-03-16T09:15:39.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.938+0000 7f3d577fe640 1 --2- 172.21.15.112:0/798946183 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3d500a2890 0x7f3d500a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:39.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.938+0000 7f3d65d9b640 1 --2- 172.21.15.112:0/798946183 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3d500a2890 0x7f3d500a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:39.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.938+0000 7f3d577fe640 1 -- 172.21.15.112:0/798946183 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f3d580f0330 con 0x7f3d680ff8a0 2024-03-16T09:15:39.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.938+0000 7f3d65d9b640 1 --2- 172.21.15.112:0/798946183 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3d500a2890 0x7f3d500a4d50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3d68199170 tx=0x7f3d5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:39.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:39.942+0000 7f3d577fe640 1 -- 172.21.15.112:0/798946183 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3d5804b020 con 0x7f3d680ff8a0 2024-03-16T09:15:40.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:39 smithi112 ceph-mon[32307]: pgmap v351: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T09:15:40.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:39 smithi112 ceph-mon[32307]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-16T09:15:40.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:39 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2979006223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:40.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.117+0000 7f3d6ca62640 1 -- 172.21.15.112:0/798946183 --> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d68061c20 con 0x7f3d500a2890 2024-03-16T09:15:40.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.124+0000 7f3d577fe640 1 -- 172.21.15.112:0/798946183 <== mgr.14152 v2:172.21.15.112:6800/2628870229 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259060 (secure 0 0 0) 0x7f3d68061c20 con 0x7f3d500a2890 2024-03-16T09:15:40.128 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:40.130 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-03-16T09:15:40.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.127+0000 7f3d6ca62640 1 -- 172.21.15.112:0/798946183 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3d500a2890 msgr2=0x7f3d500a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:40.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.127+0000 7f3d6ca62640 1 --2- 172.21.15.112:0/798946183 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3d500a2890 0x7f3d500a4d50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3d68199170 tx=0x7f3d5c009260 comp rx=0 tx=0).stop 2024-03-16T09:15:40.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.127+0000 7f3d6ca62640 1 -- 172.21.15.112:0/798946183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d680ff8a0 msgr2=0x7f3d68197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:40.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.127+0000 7f3d6ca62640 1 --2- 172.21.15.112:0/798946183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d680ff8a0 0x7f3d68197c70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3d58034ac0 tx=0x7f3d580323d0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.127+0000 7f3d6ca62640 1 -- 172.21.15.112:0/798946183 shutdown_connections 2024-03-16T09:15:40.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.127+0000 7f3d6ca62640 1 --2- 172.21.15.112:0/798946183 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3d500a2890 0x7f3d500a4d50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.127+0000 7f3d6ca62640 1 --2- 172.21.15.112:0/798946183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d68102e30 0x7f3d6819f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.127+0000 7f3d6ca62640 1 --2- 172.21.15.112:0/798946183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d68100260 0x7f3d681981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.127+0000 7f3d6ca62640 1 --2- 172.21.15.112:0/798946183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d680ff8a0 0x7f3d68197c70 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.127+0000 7f3d6ca62640 1 -- 172.21.15.112:0/798946183 >> 172.21.15.112:0/798946183 conn(0x7f3d680fb400 msgr2=0x7f3d68105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:40.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.127+0000 7f3d6ca62640 1 -- 172.21.15.112:0/798946183 shutdown_connections 2024-03-16T09:15:40.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.127+0000 7f3d6ca62640 1 -- 172.21.15.112:0/798946183 wait complete. 2024-03-16T09:15:40.140 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-03-16T09:15:38.906882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654625,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":854,"num_read_kb":540,"num_write":348,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181900,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":781,"ondisk_log_size":781,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352404,"kb_used_data":11572,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589420,"statfs":{"total":1151940427776,"available":1151579566080,"internally_reserved":0,"allocated":11849728,"data_stored":7447041,"data_compressed":112689,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":348894521},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":699,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":37,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002575"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.923673+0000","last_change":"2024-03-16T09:13:37.119969+0000","last_active":"2024-03-16T09:15:00.923673+0000","last_peered":"2024-03-16T09:15:00.923673+0000","last_clean":"2024-03-16T09:15:00.923673+0000","last_became_active":"2024-03-16T09:13:37.119177+0000","last_became_peered":"2024-03-16T09:13:37.119177+0000","last_unstale":"2024-03-16T09:15:00.923673+0000","last_undegraded":"2024-03-16T09:15:00.923673+0000","last_fullsized":"2024-03-16T09:15:00.923673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:25:28.038898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.919826+0000","last_change":"2024-03-16T09:13:37.127564+0000","last_active":"2024-03-16T09:15:00.919826+0000","last_peered":"2024-03-16T09:15:00.919826+0000","last_clean":"2024-03-16T09:15:00.919826+0000","last_became_active":"2024-03-16T09:13:37.127214+0000","last_became_peered":"2024-03-16T09:13:37.127214+0000","last_unstale":"2024-03-16T09:15:00.919826+0000","last_undegraded":"2024-03-16T09:15:00.919826+0000","last_fullsized":"2024-03-16T09:15:00.919826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:46:14.655707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.933238+0000","last_change":"2024-03-16T09:13:37.126288+0000","last_active":"2024-03-16T09:15:00.933238+0000","last_peered":"2024-03-16T09:15:00.933238+0000","last_clean":"2024-03-16T09:15:00.933238+0000","last_became_active":"2024-03-16T09:13:37.125383+0000","last_became_peered":"2024-03-16T09:13:37.125383+0000","last_unstale":"2024-03-16T09:15:00.933238+0000","last_undegraded":"2024-03-16T09:15:00.933238+0000","last_fullsized":"2024-03-16T09:15:00.933238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:47:09.901508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.919434+0000","last_change":"2024-03-16T09:13:37.125503+0000","last_active":"2024-03-16T09:15:00.919434+0000","last_peered":"2024-03-16T09:15:00.919434+0000","last_clean":"2024-03-16T09:15:00.919434+0000","last_became_active":"2024-03-16T09:13:37.125144+0000","last_became_peered":"2024-03-16T09:13:37.125144+0000","last_unstale":"2024-03-16T09:15:00.919434+0000","last_undegraded":"2024-03-16T09:15:00.919434+0000","last_fullsized":"2024-03-16T09:15:00.919434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:30:26.529080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.900877+0000","last_change":"2024-03-16T09:13:37.114237+0000","last_active":"2024-03-16T09:15:00.900877+0000","last_peered":"2024-03-16T09:15:00.900877+0000","last_clean":"2024-03-16T09:15:00.900877+0000","last_became_active":"2024-03-16T09:13:37.113595+0000","last_became_peered":"2024-03-16T09:13:37.113595+0000","last_unstale":"2024-03-16T09:15:00.900877+0000","last_undegraded":"2024-03-16T09:15:00.900877+0000","last_fullsized":"2024-03-16T09:15:00.900877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:06:17.735301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.914380+0000","last_change":"2024-03-16T09:13:37.118843+0000","last_active":"2024-03-16T09:15:00.914380+0000","last_peered":"2024-03-16T09:15:00.914380+0000","last_clean":"2024-03-16T09:15:00.914380+0000","last_became_active":"2024-03-16T09:13:37.117086+0000","last_became_peered":"2024-03-16T09:13:37.117086+0000","last_unstale":"2024-03-16T09:15:00.914380+0000","last_undegraded":"2024-03-16T09:15:00.914380+0000","last_fullsized":"2024-03-16T09:15:00.914380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:23:12.929931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.916488+0000","last_change":"2024-03-16T09:13:37.133427+0000","last_active":"2024-03-16T09:15:00.916488+0000","last_peered":"2024-03-16T09:15:00.916488+0000","last_clean":"2024-03-16T09:15:00.916488+0000","last_became_active":"2024-03-16T09:13:37.133032+0000","last_became_peered":"2024-03-16T09:13:37.133032+0000","last_unstale":"2024-03-16T09:15:00.916488+0000","last_undegraded":"2024-03-16T09:15:00.916488+0000","last_fullsized":"2024-03-16T09:15:00.916488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:49:28.179883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.920627+0000","last_change":"2024-03-16T09:13:37.132132+0000","last_active":"2024-03-16T09:15:00.920627+0000","last_peered":"2024-03-16T09:15:00.920627+0000","last_clean":"2024-03-16T09:15:00.920627+0000","last_became_active":"2024-03-16T09:13:37.131801+0000","last_became_peered":"2024-03-16T09:13:37.131801+0000","last_unstale":"2024-03-16T09:15:00.920627+0000","last_undegraded":"2024-03-16T09:15:00.920627+0000","last_fullsized":"2024-03-16T09:15:00.920627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:45:03.891725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.930016+0000","last_change":"2024-03-16T09:13:37.116378+0000","last_active":"2024-03-16T09:15:00.930016+0000","last_peered":"2024-03-16T09:15:00.930016+0000","last_clean":"2024-03-16T09:15:00.930016+0000","last_became_active":"2024-03-16T09:13:37.116027+0000","last_became_peered":"2024-03-16T09:13:37.116027+0000","last_unstale":"2024-03-16T09:15:00.930016+0000","last_undegraded":"2024-03-16T09:15:00.930016+0000","last_fullsized":"2024-03-16T09:15:00.930016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:14:38.681613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.919236+0000","last_change":"2024-03-16T09:13:37.121417+0000","last_active":"2024-03-16T09:15:00.919236+0000","last_peered":"2024-03-16T09:15:00.919236+0000","last_clean":"2024-03-16T09:15:00.919236+0000","last_became_active":"2024-03-16T09:13:37.121112+0000","last_became_peered":"2024-03-16T09:13:37.121112+0000","last_unstale":"2024-03-16T09:15:00.919236+0000","last_undegraded":"2024-03-16T09:15:00.919236+0000","last_fullsized":"2024-03-16T09:15:00.919236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:38:07.823604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.935183+0000","last_change":"2024-03-16T09:13:37.121368+0000","last_active":"2024-03-16T09:15:00.935183+0000","last_peered":"2024-03-16T09:15:00.935183+0000","last_clean":"2024-03-16T09:15:00.935183+0000","last_became_active":"2024-03-16T09:13:37.121060+0000","last_became_peered":"2024-03-16T09:13:37.121060+0000","last_unstale":"2024-03-16T09:15:00.935183+0000","last_undegraded":"2024-03-16T09:15:00.935183+0000","last_fullsized":"2024-03-16T09:15:00.935183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:48:49.168285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.903523+0000","last_change":"2024-03-16T09:13:37.127004+0000","last_active":"2024-03-16T09:15:00.903523+0000","last_peered":"2024-03-16T09:15:00.903523+0000","last_clean":"2024-03-16T09:15:00.903523+0000","last_became_active":"2024-03-16T09:13:37.126579+0000","last_became_peered":"2024-03-16T09:13:37.126579+0000","last_unstale":"2024-03-16T09:15:00.903523+0000","last_undegraded":"2024-03-16T09:15:00.903523+0000","last_fullsized":"2024-03-16T09:15:00.903523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:54:18.382514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.918879+0000","last_change":"2024-03-16T09:13:37.104534+0000","last_active":"2024-03-16T09:15:00.918879+0000","last_peered":"2024-03-16T09:15:00.918879+0000","last_clean":"2024-03-16T09:15:00.918879+0000","last_became_active":"2024-03-16T09:13:37.104202+0000","last_became_peered":"2024-03-16T09:13:37.104202+0000","last_unstale":"2024-03-16T09:15:00.918879+0000","last_undegraded":"2024-03-16T09:15:00.918879+0000","last_fullsized":"2024-03-16T09:15:00.918879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:40:55.448238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.920211+0000","last_change":"2024-03-16T09:13:37.117166+0000","last_active":"2024-03-16T09:15:00.920211+0000","last_peered":"2024-03-16T09:15:00.920211+0000","last_clean":"2024-03-16T09:15:00.920211+0000","last_became_active":"2024-03-16T09:13:37.116747+0000","last_became_peered":"2024-03-16T09:13:37.116747+0000","last_unstale":"2024-03-16T09:15:00.920211+0000","last_undegraded":"2024-03-16T09:15:00.920211+0000","last_fullsized":"2024-03-16T09:15:00.920211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:50:58.705683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.921643+0000","last_change":"2024-03-16T09:14:32.000383+0000","last_active":"2024-03-16T09:15:00.921643+0000","last_peered":"2024-03-16T09:15:00.921643+0000","last_clean":"2024-03-16T09:15:00.921643+0000","last_became_active":"2024-03-16T09:14:32.000027+0000","last_became_peered":"2024-03-16T09:14:32.000027+0000","last_unstale":"2024-03-16T09:15:00.921643+0000","last_undegraded":"2024-03-16T09:15:00.921643+0000","last_fullsized":"2024-03-16T09:15:00.921643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:04:03.478849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.934752+0000","last_change":"2024-03-16T09:13:37.120583+0000","last_active":"2024-03-16T09:15:00.934752+0000","last_peered":"2024-03-16T09:15:00.934752+0000","last_clean":"2024-03-16T09:15:00.934752+0000","last_became_active":"2024-03-16T09:13:37.120217+0000","last_became_peered":"2024-03-16T09:13:37.120217+0000","last_unstale":"2024-03-16T09:15:00.934752+0000","last_undegraded":"2024-03-16T09:15:00.934752+0000","last_fullsized":"2024-03-16T09:15:00.934752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:00:17.653736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.928060+0000","last_change":"2024-03-16T09:13:37.121179+0000","last_active":"2024-03-16T09:15:00.928060+0000","last_peered":"2024-03-16T09:15:00.928060+0000","last_clean":"2024-03-16T09:15:00.928060+0000","last_became_active":"2024-03-16T09:13:37.120070+0000","last_became_peered":"2024-03-16T09:13:37.120070+0000","last_unstale":"2024-03-16T09:15:00.928060+0000","last_undegraded":"2024-03-16T09:15:00.928060+0000","last_fullsized":"2024-03-16T09:15:00.928060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:39:24.937068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.935539+0000","last_change":"2024-03-16T09:13:37.110074+0000","last_active":"2024-03-16T09:15:00.935539+0000","last_peered":"2024-03-16T09:15:00.935539+0000","last_clean":"2024-03-16T09:15:00.935539+0000","last_became_active":"2024-03-16T09:13:37.109767+0000","last_became_peered":"2024-03-16T09:13:37.109767+0000","last_unstale":"2024-03-16T09:15:00.935539+0000","last_undegraded":"2024-03-16T09:15:00.935539+0000","last_fullsized":"2024-03-16T09:15:00.935539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:20:37.408262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.911978+0000","last_change":"2024-03-16T09:13:37.130791+0000","last_active":"2024-03-16T09:15:00.911978+0000","last_peered":"2024-03-16T09:15:00.911978+0000","last_clean":"2024-03-16T09:15:00.911978+0000","last_became_active":"2024-03-16T09:13:37.130428+0000","last_became_peered":"2024-03-16T09:13:37.130428+0000","last_unstale":"2024-03-16T09:15:00.911978+0000","last_undegraded":"2024-03-16T09:15:00.911978+0000","last_fullsized":"2024-03-16T09:15:00.911978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:32:33.185589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.006081+0000","last_change":"2024-03-16T09:14:32.006081+0000","last_active":"2024-03-16T09:14:32.006081+0000","last_peered":"2024-03-16T09:14:32.006081+0000","last_clean":"2024-03-16T09:14:32.006081+0000","last_became_active":"2024-03-16T09:14:32.000165+0000","last_became_peered":"2024-03-16T09:14:32.000165+0000","last_unstale":"2024-03-16T09:14:32.006081+0000","last_undegraded":"2024-03-16T09:14:32.006081+0000","last_fullsized":"2024-03-16T09:14:32.006081+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:12:54.685988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.918321+0000","last_change":"2024-03-16T09:13:37.124946+0000","last_active":"2024-03-16T09:15:00.918321+0000","last_peered":"2024-03-16T09:15:00.918321+0000","last_clean":"2024-03-16T09:15:00.918321+0000","last_became_active":"2024-03-16T09:13:37.124624+0000","last_became_peered":"2024-03-16T09:13:37.124624+0000","last_unstale":"2024-03-16T09:15:00.918321+0000","last_undegraded":"2024-03-16T09:15:00.918321+0000","last_fullsized":"2024-03-16T09:15:00.918321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:18:15.188823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517217+0000","last_change":"2024-03-16T09:13:36.116312+0000","last_active":"2024-03-16T09:14:32.517217+0000","last_peered":"2024-03-16T09:14:32.517217+0000","last_clean":"2024-03-16T09:14:32.517217+0000","last_became_active":"2024-03-16T09:13:36.115999+0000","last_became_peered":"2024-03-16T09:13:36.115999+0000","last_unstale":"2024-03-16T09:14:32.517217+0000","last_undegraded":"2024-03-16T09:14:32.517217+0000","last_fullsized":"2024-03-16T09:14:32.517217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:04:46.795291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.933147+0000","last_change":"2024-03-16T09:13:37.126024+0000","last_active":"2024-03-16T09:15:00.933147+0000","last_peered":"2024-03-16T09:15:00.933147+0000","last_clean":"2024-03-16T09:15:00.933147+0000","last_became_active":"2024-03-16T09:13:37.125722+0000","last_became_peered":"2024-03-16T09:13:37.125722+0000","last_unstale":"2024-03-16T09:15:00.933147+0000","last_undegraded":"2024-03-16T09:15:00.933147+0000","last_fullsized":"2024-03-16T09:15:00.933147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:27:24.923022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.005879+0000","last_change":"2024-03-16T09:14:32.005295+0000","last_active":"2024-03-16T09:14:32.005879+0000","last_peered":"2024-03-16T09:14:32.005879+0000","last_clean":"2024-03-16T09:14:32.005879+0000","last_became_active":"2024-03-16T09:14:31.999981+0000","last_became_peered":"2024-03-16T09:14:31.999981+0000","last_unstale":"2024-03-16T09:14:32.005879+0000","last_undegraded":"2024-03-16T09:14:32.005879+0000","last_fullsized":"2024-03-16T09:14:32.005879+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:13:32.661689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.932390+0000","last_change":"2024-03-16T09:13:37.127593+0000","last_active":"2024-03-16T09:15:00.932390+0000","last_peered":"2024-03-16T09:15:00.932390+0000","last_clean":"2024-03-16T09:15:00.932390+0000","last_became_active":"2024-03-16T09:13:37.127255+0000","last_became_peered":"2024-03-16T09:13:37.127255+0000","last_unstale":"2024-03-16T09:15:00.932390+0000","last_undegraded":"2024-03-16T09:15:00.932390+0000","last_fullsized":"2024-03-16T09:15:00.932390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:07:51.263266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.995143+0000","last_change":"2024-03-16T09:13:36.117583+0000","last_active":"2024-03-16T09:14:31.995143+0000","last_peered":"2024-03-16T09:14:31.995143+0000","last_clean":"2024-03-16T09:14:31.995143+0000","last_became_active":"2024-03-16T09:13:36.117270+0000","last_became_peered":"2024-03-16T09:13:36.117270+0000","last_unstale":"2024-03-16T09:14:31.995143+0000","last_undegraded":"2024-03-16T09:14:31.995143+0000","last_fullsized":"2024-03-16T09:14:31.995143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:10:53.541872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.924161+0000","last_change":"2024-03-16T09:13:37.119019+0000","last_active":"2024-03-16T09:15:00.924161+0000","last_peered":"2024-03-16T09:15:00.924161+0000","last_clean":"2024-03-16T09:15:00.924161+0000","last_became_active":"2024-03-16T09:13:37.118148+0000","last_became_peered":"2024-03-16T09:13:37.118148+0000","last_unstale":"2024-03-16T09:15:00.924161+0000","last_undegraded":"2024-03-16T09:15:00.924161+0000","last_fullsized":"2024-03-16T09:15:00.924161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:17:43.042779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":143,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:35.446937+0000","last_change":"2024-03-16T09:13:36.114415+0000","last_active":"2024-03-16T09:15:35.446937+0000","last_peered":"2024-03-16T09:15:35.446937+0000","last_clean":"2024-03-16T09:15:35.446937+0000","last_became_active":"2024-03-16T09:13:36.114056+0000","last_became_peered":"2024-03-16T09:13:36.114056+0000","last_unstale":"2024-03-16T09:15:35.446937+0000","last_undegraded":"2024-03-16T09:15:35.446937+0000","last_fullsized":"2024-03-16T09:15:35.446937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:50:28.322627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.904292+0000","last_change":"2024-03-16T09:13:37.124997+0000","last_active":"2024-03-16T09:15:00.904292+0000","last_peered":"2024-03-16T09:15:00.904292+0000","last_clean":"2024-03-16T09:15:00.904292+0000","last_became_active":"2024-03-16T09:13:37.124664+0000","last_became_peered":"2024-03-16T09:13:37.124664+0000","last_unstale":"2024-03-16T09:15:00.904292+0000","last_undegraded":"2024-03-16T09:15:00.904292+0000","last_fullsized":"2024-03-16T09:15:00.904292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:24:30.104576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.230143+0000","last_change":"2024-03-16T09:13:36.121980+0000","last_active":"2024-03-16T09:14:32.230143+0000","last_peered":"2024-03-16T09:14:32.230143+0000","last_clean":"2024-03-16T09:14:32.230143+0000","last_became_active":"2024-03-16T09:13:36.121703+0000","last_became_peered":"2024-03-16T09:13:36.121703+0000","last_unstale":"2024-03-16T09:14:32.230143+0000","last_undegraded":"2024-03-16T09:14:32.230143+0000","last_fullsized":"2024-03-16T09:14:32.230143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:40:26.601688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.920701+0000","last_change":"2024-03-16T09:13:37.123443+0000","last_active":"2024-03-16T09:15:00.920701+0000","last_peered":"2024-03-16T09:15:00.920701+0000","last_clean":"2024-03-16T09:15:00.920701+0000","last_became_active":"2024-03-16T09:13:37.121987+0000","last_became_peered":"2024-03-16T09:13:37.121987+0000","last_unstale":"2024-03-16T09:15:00.920701+0000","last_undegraded":"2024-03-16T09:15:00.920701+0000","last_fullsized":"2024-03-16T09:15:00.920701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:37:08.478552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385000+0000","last_change":"2024-03-16T09:13:36.106391+0000","last_active":"2024-03-16T09:14:32.385000+0000","last_peered":"2024-03-16T09:14:32.385000+0000","last_clean":"2024-03-16T09:14:32.385000+0000","last_became_active":"2024-03-16T09:13:36.106046+0000","last_became_peered":"2024-03-16T09:13:36.106046+0000","last_unstale":"2024-03-16T09:14:32.385000+0000","last_undegraded":"2024-03-16T09:14:32.385000+0000","last_fullsized":"2024-03-16T09:14:32.385000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:03:23.255519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.911156+0000","last_change":"2024-03-16T09:13:37.114337+0000","last_active":"2024-03-16T09:15:00.911156+0000","last_peered":"2024-03-16T09:15:00.911156+0000","last_clean":"2024-03-16T09:15:00.911156+0000","last_became_active":"2024-03-16T09:13:37.113991+0000","last_became_peered":"2024-03-16T09:13:37.113991+0000","last_unstale":"2024-03-16T09:15:00.911156+0000","last_undegraded":"2024-03-16T09:15:00.911156+0000","last_fullsized":"2024-03-16T09:15:00.911156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:23:54.141479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248113+0000","last_change":"2024-03-16T09:13:36.113297+0000","last_active":"2024-03-16T09:14:32.248113+0000","last_peered":"2024-03-16T09:14:32.248113+0000","last_clean":"2024-03-16T09:14:32.248113+0000","last_became_active":"2024-03-16T09:13:36.112950+0000","last_became_peered":"2024-03-16T09:13:36.112950+0000","last_unstale":"2024-03-16T09:14:32.248113+0000","last_undegraded":"2024-03-16T09:14:32.248113+0000","last_fullsized":"2024-03-16T09:14:32.248113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:40:17.682450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.922064+0000","last_change":"2024-03-16T09:13:37.112239+0000","last_active":"2024-03-16T09:15:00.922064+0000","last_peered":"2024-03-16T09:15:00.922064+0000","last_clean":"2024-03-16T09:15:00.922064+0000","last_became_active":"2024-03-16T09:13:37.111885+0000","last_became_peered":"2024-03-16T09:13:37.111885+0000","last_unstale":"2024-03-16T09:15:00.922064+0000","last_undegraded":"2024-03-16T09:15:00.922064+0000","last_fullsized":"2024-03-16T09:15:00.922064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:49.565846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":138,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:34.445555+0000","last_change":"2024-03-16T09:13:36.111341+0000","last_active":"2024-03-16T09:15:34.445555+0000","last_peered":"2024-03-16T09:15:34.445555+0000","last_clean":"2024-03-16T09:15:34.445555+0000","last_became_active":"2024-03-16T09:13:36.110558+0000","last_became_peered":"2024-03-16T09:13:36.110558+0000","last_unstale":"2024-03-16T09:15:34.445555+0000","last_undegraded":"2024-03-16T09:15:34.445555+0000","last_fullsized":"2024-03-16T09:15:34.445555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:56:44.252002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.933077+0000","last_change":"2024-03-16T09:13:37.126808+0000","last_active":"2024-03-16T09:15:00.933077+0000","last_peered":"2024-03-16T09:15:00.933077+0000","last_clean":"2024-03-16T09:15:00.933077+0000","last_became_active":"2024-03-16T09:13:37.126442+0000","last_became_peered":"2024-03-16T09:13:37.126442+0000","last_unstale":"2024-03-16T09:15:00.933077+0000","last_undegraded":"2024-03-16T09:15:00.933077+0000","last_fullsized":"2024-03-16T09:15:00.933077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:04:39.216835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.004330+0000","last_change":"2024-03-16T09:14:32.004330+0000","last_active":"2024-03-16T09:14:32.004330+0000","last_peered":"2024-03-16T09:14:32.004330+0000","last_clean":"2024-03-16T09:14:32.004330+0000","last_became_active":"2024-03-16T09:14:32.001741+0000","last_became_peered":"2024-03-16T09:14:32.001741+0000","last_unstale":"2024-03-16T09:14:32.004330+0000","last_undegraded":"2024-03-16T09:14:32.004330+0000","last_fullsized":"2024-03-16T09:14:32.004330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:08:19.002873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.921799+0000","last_change":"2024-03-16T09:13:37.125833+0000","last_active":"2024-03-16T09:15:00.921799+0000","last_peered":"2024-03-16T09:15:00.921799+0000","last_clean":"2024-03-16T09:15:00.921799+0000","last_became_active":"2024-03-16T09:13:37.125508+0000","last_became_peered":"2024-03-16T09:13:37.125508+0000","last_unstale":"2024-03-16T09:15:00.921799+0000","last_undegraded":"2024-03-16T09:15:00.921799+0000","last_fullsized":"2024-03-16T09:15:00.921799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:14:47.749715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248156+0000","last_change":"2024-03-16T09:13:36.112200+0000","last_active":"2024-03-16T09:14:32.248156+0000","last_peered":"2024-03-16T09:14:32.248156+0000","last_clean":"2024-03-16T09:14:32.248156+0000","last_became_active":"2024-03-16T09:13:36.111830+0000","last_became_peered":"2024-03-16T09:13:36.111830+0000","last_unstale":"2024-03-16T09:14:32.248156+0000","last_undegraded":"2024-03-16T09:14:32.248156+0000","last_fullsized":"2024-03-16T09:14:32.248156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:07:57.147960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.934446+0000","last_change":"2024-03-16T09:13:37.118434+0000","last_active":"2024-03-16T09:15:00.934446+0000","last_peered":"2024-03-16T09:15:00.934446+0000","last_clean":"2024-03-16T09:15:00.934446+0000","last_became_active":"2024-03-16T09:13:37.118069+0000","last_became_peered":"2024-03-16T09:13:37.118069+0000","last_unstale":"2024-03-16T09:15:00.934446+0000","last_undegraded":"2024-03-16T09:15:00.934446+0000","last_fullsized":"2024-03-16T09:15:00.934446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:02:56.559384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'32","reported_seq":292,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:35.447044+0000","last_change":"2024-03-16T09:13:36.114591+0000","last_active":"2024-03-16T09:15:35.447044+0000","last_peered":"2024-03-16T09:15:35.447044+0000","last_clean":"2024-03-16T09:15:35.447044+0000","last_became_active":"2024-03-16T09:13:36.114252+0000","last_became_peered":"2024-03-16T09:13:36.114252+0000","last_unstale":"2024-03-16T09:15:35.447044+0000","last_undegraded":"2024-03-16T09:15:35.447044+0000","last_fullsized":"2024-03-16T09:15:35.447044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:37:52.637394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60022,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":241,"num_read_kb":179,"num_write":33,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.913494+0000","last_change":"2024-03-16T09:13:37.133102+0000","last_active":"2024-03-16T09:15:00.913494+0000","last_peered":"2024-03-16T09:15:00.913494+0000","last_clean":"2024-03-16T09:15:00.913494+0000","last_became_active":"2024-03-16T09:13:37.132728+0000","last_became_peered":"2024-03-16T09:13:37.132728+0000","last_unstale":"2024-03-16T09:15:00.913494+0000","last_undegraded":"2024-03-16T09:15:00.913494+0000","last_fullsized":"2024-03-16T09:15:00.913494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:54:27.511997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998770+0000","last_change":"2024-03-16T09:13:36.114195+0000","last_active":"2024-03-16T09:14:31.998770+0000","last_peered":"2024-03-16T09:14:31.998770+0000","last_clean":"2024-03-16T09:14:31.998770+0000","last_became_active":"2024-03-16T09:13:36.113829+0000","last_became_peered":"2024-03-16T09:13:36.113829+0000","last_unstale":"2024-03-16T09:14:31.998770+0000","last_undegraded":"2024-03-16T09:14:31.998770+0000","last_fullsized":"2024-03-16T09:14:31.998770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:28:40.509877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.912363+0000","last_change":"2024-03-16T09:13:37.132974+0000","last_active":"2024-03-16T09:15:00.912363+0000","last_peered":"2024-03-16T09:15:00.912363+0000","last_clean":"2024-03-16T09:15:00.912363+0000","last_became_active":"2024-03-16T09:13:37.132474+0000","last_became_peered":"2024-03-16T09:13:37.132474+0000","last_unstale":"2024-03-16T09:15:00.912363+0000","last_undegraded":"2024-03-16T09:15:00.912363+0000","last_fullsized":"2024-03-16T09:15:00.912363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:58:04.480441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.247992+0000","last_change":"2024-03-16T09:13:36.115242+0000","last_active":"2024-03-16T09:14:32.247992+0000","last_peered":"2024-03-16T09:14:32.247992+0000","last_clean":"2024-03-16T09:14:32.247992+0000","last_became_active":"2024-03-16T09:13:36.113566+0000","last_became_peered":"2024-03-16T09:13:36.113566+0000","last_unstale":"2024-03-16T09:14:32.247992+0000","last_undegraded":"2024-03-16T09:14:32.247992+0000","last_fullsized":"2024-03-16T09:14:32.247992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:47:50.950841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.927334+0000","last_change":"2024-03-16T09:14:32.386530+0000","last_active":"2024-03-16T09:15:00.927334+0000","last_peered":"2024-03-16T09:15:00.927334+0000","last_clean":"2024-03-16T09:15:00.927334+0000","last_became_active":"2024-03-16T09:14:32.386088+0000","last_became_peered":"2024-03-16T09:14:32.386088+0000","last_unstale":"2024-03-16T09:15:00.927334+0000","last_undegraded":"2024-03-16T09:15:00.927334+0000","last_fullsized":"2024-03-16T09:15:00.927334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:44:57.427154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":144,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:36.457841+0000","last_change":"2024-03-16T09:13:36.109585+0000","last_active":"2024-03-16T09:15:36.457841+0000","last_peered":"2024-03-16T09:15:36.457841+0000","last_clean":"2024-03-16T09:15:36.457841+0000","last_became_active":"2024-03-16T09:13:36.109267+0000","last_became_peered":"2024-03-16T09:13:36.109267+0000","last_unstale":"2024-03-16T09:15:36.457841+0000","last_undegraded":"2024-03-16T09:15:36.457841+0000","last_fullsized":"2024-03-16T09:15:36.457841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:26:55.700791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.926195+0000","last_change":"2024-03-16T09:13:37.136465+0000","last_active":"2024-03-16T09:15:00.926195+0000","last_peered":"2024-03-16T09:15:00.926195+0000","last_clean":"2024-03-16T09:15:00.926195+0000","last_became_active":"2024-03-16T09:13:37.135933+0000","last_became_peered":"2024-03-16T09:13:37.135933+0000","last_unstale":"2024-03-16T09:15:00.926195+0000","last_undegraded":"2024-03-16T09:15:00.926195+0000","last_fullsized":"2024-03-16T09:15:00.926195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:08:28.490533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994732+0000","last_change":"2024-03-16T09:13:36.119428+0000","last_active":"2024-03-16T09:14:31.994732+0000","last_peered":"2024-03-16T09:14:31.994732+0000","last_clean":"2024-03-16T09:14:31.994732+0000","last_became_active":"2024-03-16T09:13:36.119083+0000","last_became_peered":"2024-03-16T09:13:36.119083+0000","last_unstale":"2024-03-16T09:14:31.994732+0000","last_undegraded":"2024-03-16T09:14:31.994732+0000","last_fullsized":"2024-03-16T09:14:31.994732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:58:56.065151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.920460+0000","last_change":"2024-03-16T09:13:37.118082+0000","last_active":"2024-03-16T09:15:00.920460+0000","last_peered":"2024-03-16T09:15:00.920460+0000","last_clean":"2024-03-16T09:15:00.920460+0000","last_became_active":"2024-03-16T09:13:37.117671+0000","last_became_peered":"2024-03-16T09:13:37.117671+0000","last_unstale":"2024-03-16T09:15:00.920460+0000","last_undegraded":"2024-03-16T09:15:00.920460+0000","last_fullsized":"2024-03-16T09:15:00.920460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:14:05.352191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.995286+0000","last_change":"2024-03-16T09:13:36.121077+0000","last_active":"2024-03-16T09:14:31.995286+0000","last_peered":"2024-03-16T09:14:31.995286+0000","last_clean":"2024-03-16T09:14:31.995286+0000","last_became_active":"2024-03-16T09:13:36.120723+0000","last_became_peered":"2024-03-16T09:13:36.120723+0000","last_unstale":"2024-03-16T09:14:31.995286+0000","last_undegraded":"2024-03-16T09:14:31.995286+0000","last_fullsized":"2024-03-16T09:14:31.995286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:58:14.746215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.922037+0000","last_change":"2024-03-16T09:13:37.116725+0000","last_active":"2024-03-16T09:15:00.922037+0000","last_peered":"2024-03-16T09:15:00.922037+0000","last_clean":"2024-03-16T09:15:00.922037+0000","last_became_active":"2024-03-16T09:13:37.116406+0000","last_became_peered":"2024-03-16T09:13:37.116406+0000","last_unstale":"2024-03-16T09:15:00.922037+0000","last_undegraded":"2024-03-16T09:15:00.922037+0000","last_fullsized":"2024-03-16T09:15:00.922037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:56:59.136649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996706+0000","last_change":"2024-03-16T09:13:36.110568+0000","last_active":"2024-03-16T09:14:31.996706+0000","last_peered":"2024-03-16T09:14:31.996706+0000","last_clean":"2024-03-16T09:14:31.996706+0000","last_became_active":"2024-03-16T09:13:36.110246+0000","last_became_peered":"2024-03-16T09:13:36.110246+0000","last_unstale":"2024-03-16T09:14:31.996706+0000","last_undegraded":"2024-03-16T09:14:31.996706+0000","last_fullsized":"2024-03-16T09:14:31.996706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:36:50.011941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.921527+0000","last_change":"2024-03-16T09:13:37.122296+0000","last_active":"2024-03-16T09:15:00.921527+0000","last_peered":"2024-03-16T09:15:00.921527+0000","last_clean":"2024-03-16T09:15:00.921527+0000","last_became_active":"2024-03-16T09:13:37.121968+0000","last_became_peered":"2024-03-16T09:13:37.121968+0000","last_unstale":"2024-03-16T09:15:00.921527+0000","last_undegraded":"2024-03-16T09:15:00.921527+0000","last_fullsized":"2024-03-16T09:15:00.921527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:10:43.449397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997895+0000","last_change":"2024-03-16T09:13:36.112262+0000","last_active":"2024-03-16T09:14:31.997895+0000","last_peered":"2024-03-16T09:14:31.997895+0000","last_clean":"2024-03-16T09:14:31.997895+0000","last_became_active":"2024-03-16T09:13:36.111916+0000","last_became_peered":"2024-03-16T09:13:36.111916+0000","last_unstale":"2024-03-16T09:14:31.997895+0000","last_undegraded":"2024-03-16T09:14:31.997895+0000","last_fullsized":"2024-03-16T09:14:31.997895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:27:41.163508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.906629+0000","last_change":"2024-03-16T09:13:37.119814+0000","last_active":"2024-03-16T09:15:00.906629+0000","last_peered":"2024-03-16T09:15:00.906629+0000","last_clean":"2024-03-16T09:15:00.906629+0000","last_became_active":"2024-03-16T09:13:37.119454+0000","last_became_peered":"2024-03-16T09:13:37.119454+0000","last_unstale":"2024-03-16T09:15:00.906629+0000","last_undegraded":"2024-03-16T09:15:00.906629+0000","last_fullsized":"2024-03-16T09:15:00.906629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:57:25.988044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.904133+0000","last_change":"2024-03-16T09:14:31.999902+0000","last_active":"2024-03-16T09:15:00.904133+0000","last_peered":"2024-03-16T09:15:00.904133+0000","last_clean":"2024-03-16T09:15:00.904133+0000","last_became_active":"2024-03-16T09:14:31.999609+0000","last_became_peered":"2024-03-16T09:14:31.999609+0000","last_unstale":"2024-03-16T09:15:00.904133+0000","last_undegraded":"2024-03-16T09:15:00.904133+0000","last_fullsized":"2024-03-16T09:15:00.904133+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:58:47.495746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":142,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:36.456168+0000","last_change":"2024-03-16T09:13:36.111300+0000","last_active":"2024-03-16T09:15:36.456168+0000","last_peered":"2024-03-16T09:15:36.456168+0000","last_clean":"2024-03-16T09:15:36.456168+0000","last_became_active":"2024-03-16T09:13:36.110966+0000","last_became_peered":"2024-03-16T09:13:36.110966+0000","last_unstale":"2024-03-16T09:15:36.456168+0000","last_undegraded":"2024-03-16T09:15:36.456168+0000","last_fullsized":"2024-03-16T09:15:36.456168+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:18:33.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.931071+0000","last_change":"2024-03-16T09:13:37.125791+0000","last_active":"2024-03-16T09:15:00.931071+0000","last_peered":"2024-03-16T09:15:00.931071+0000","last_clean":"2024-03-16T09:15:00.931071+0000","last_became_active":"2024-03-16T09:13:37.125448+0000","last_became_peered":"2024-03-16T09:13:37.125448+0000","last_unstale":"2024-03-16T09:15:00.931071+0000","last_undegraded":"2024-03-16T09:15:00.931071+0000","last_fullsized":"2024-03-16T09:15:00.931071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:51:30.767750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996257+0000","last_change":"2024-03-16T09:13:36.107462+0000","last_active":"2024-03-16T09:14:31.996257+0000","last_peered":"2024-03-16T09:14:31.996257+0000","last_clean":"2024-03-16T09:14:31.996257+0000","last_became_active":"2024-03-16T09:13:36.107115+0000","last_became_peered":"2024-03-16T09:13:36.107115+0000","last_unstale":"2024-03-16T09:14:31.996257+0000","last_undegraded":"2024-03-16T09:14:31.996257+0000","last_fullsized":"2024-03-16T09:14:31.996257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:37.796719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997730+0000","last_change":"2024-03-16T09:12:24.331159+0000","last_active":"2024-03-16T09:14:31.997730+0000","last_peered":"2024-03-16T09:14:31.997730+0000","last_clean":"2024-03-16T09:14:31.997730+0000","last_became_active":"2024-03-16T09:12:24.330554+0000","last_became_peered":"2024-03-16T09:12:24.330554+0000","last_unstale":"2024-03-16T09:14:31.997730+0000","last_undegraded":"2024-03-16T09:14:31.997730+0000","last_fullsized":"2024-03-16T09:14:31.997730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:08:57.697012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:08:57.697012+0000","last_clean_scrub_stamp":"2024-03-16T09:08:57.697012+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:43:02.986865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.909651+0000","last_change":"2024-03-16T09:13:37.114299+0000","last_active":"2024-03-16T09:15:00.909651+0000","last_peered":"2024-03-16T09:15:00.909651+0000","last_clean":"2024-03-16T09:15:00.909651+0000","last_became_active":"2024-03-16T09:13:37.113981+0000","last_became_peered":"2024-03-16T09:13:37.113981+0000","last_unstale":"2024-03-16T09:15:00.909651+0000","last_undegraded":"2024-03-16T09:15:00.909651+0000","last_fullsized":"2024-03-16T09:15:00.909651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:17:15.882130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.997873+0000","last_change":"2024-03-16T09:13:36.113962+0000","last_active":"2024-03-16T09:14:31.997873+0000","last_peered":"2024-03-16T09:14:31.997873+0000","last_clean":"2024-03-16T09:14:31.997873+0000","last_became_active":"2024-03-16T09:13:36.113616+0000","last_became_peered":"2024-03-16T09:13:36.113616+0000","last_unstale":"2024-03-16T09:14:31.997873+0000","last_undegraded":"2024-03-16T09:14:31.997873+0000","last_fullsized":"2024-03-16T09:14:31.997873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:15:15.004360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.995031+0000","last_change":"2024-03-16T09:13:36.121946+0000","last_active":"2024-03-16T09:14:31.995031+0000","last_peered":"2024-03-16T09:14:31.995031+0000","last_clean":"2024-03-16T09:14:31.995031+0000","last_became_active":"2024-03-16T09:13:36.121209+0000","last_became_peered":"2024-03-16T09:13:36.121209+0000","last_unstale":"2024-03-16T09:14:31.995031+0000","last_undegraded":"2024-03-16T09:14:31.995031+0000","last_fullsized":"2024-03-16T09:14:31.995031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:09:51.139468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.908991+0000","last_change":"2024-03-16T09:13:37.128931+0000","last_active":"2024-03-16T09:15:00.908991+0000","last_peered":"2024-03-16T09:15:00.908991+0000","last_clean":"2024-03-16T09:15:00.908991+0000","last_became_active":"2024-03-16T09:13:37.128515+0000","last_became_peered":"2024-03-16T09:13:37.128515+0000","last_unstale":"2024-03-16T09:15:00.908991+0000","last_undegraded":"2024-03-16T09:15:00.908991+0000","last_fullsized":"2024-03-16T09:15:00.908991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:58:46.429144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:01.126760+0000","last_change":"2024-03-16T09:13:36.118831+0000","last_active":"2024-03-16T09:15:01.126760+0000","last_peered":"2024-03-16T09:15:01.126760+0000","last_clean":"2024-03-16T09:15:01.126760+0000","last_became_active":"2024-03-16T09:13:36.118489+0000","last_became_peered":"2024-03-16T09:13:36.118489+0000","last_unstale":"2024-03-16T09:15:01.126760+0000","last_undegraded":"2024-03-16T09:15:01.126760+0000","last_fullsized":"2024-03-16T09:15:01.126760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:55:34.089451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.923760+0000","last_change":"2024-03-16T09:13:37.120581+0000","last_active":"2024-03-16T09:15:00.923760+0000","last_peered":"2024-03-16T09:15:00.923760+0000","last_clean":"2024-03-16T09:15:00.923760+0000","last_became_active":"2024-03-16T09:13:37.120266+0000","last_became_peered":"2024-03-16T09:13:37.120266+0000","last_unstale":"2024-03-16T09:15:00.923760+0000","last_undegraded":"2024-03-16T09:15:00.923760+0000","last_fullsized":"2024-03-16T09:15:00.923760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:16:10.784229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.996414+0000","last_change":"2024-03-16T09:13:36.107125+0000","last_active":"2024-03-16T09:14:31.996414+0000","last_peered":"2024-03-16T09:14:31.996414+0000","last_clean":"2024-03-16T09:14:31.996414+0000","last_became_active":"2024-03-16T09:13:36.106795+0000","last_became_peered":"2024-03-16T09:13:36.106795+0000","last_unstale":"2024-03-16T09:14:31.996414+0000","last_undegraded":"2024-03-16T09:14:31.996414+0000","last_fullsized":"2024-03-16T09:14:31.996414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:29:10.130565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.922341+0000","last_change":"2024-03-16T09:13:37.123013+0000","last_active":"2024-03-16T09:15:00.922341+0000","last_peered":"2024-03-16T09:15:00.922341+0000","last_clean":"2024-03-16T09:15:00.922341+0000","last_became_active":"2024-03-16T09:13:37.122642+0000","last_became_peered":"2024-03-16T09:13:37.122642+0000","last_unstale":"2024-03-16T09:15:00.922341+0000","last_undegraded":"2024-03-16T09:15:00.922341+0000","last_fullsized":"2024-03-16T09:15:00.922341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:35:05.297337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.000335+0000","last_change":"2024-03-16T09:13:36.110868+0000","last_active":"2024-03-16T09:14:32.000335+0000","last_peered":"2024-03-16T09:14:32.000335+0000","last_clean":"2024-03-16T09:14:32.000335+0000","last_became_active":"2024-03-16T09:13:36.110536+0000","last_became_peered":"2024-03-16T09:13:36.110536+0000","last_unstale":"2024-03-16T09:14:32.000335+0000","last_undegraded":"2024-03-16T09:14:32.000335+0000","last_fullsized":"2024-03-16T09:14:32.000335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:07:06.829458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.882568+0000","last_change":"2024-03-16T09:13:37.116579+0000","last_active":"2024-03-16T09:15:00.882568+0000","last_peered":"2024-03-16T09:15:00.882568+0000","last_clean":"2024-03-16T09:15:00.882568+0000","last_became_active":"2024-03-16T09:13:37.116226+0000","last_became_peered":"2024-03-16T09:13:37.116226+0000","last_unstale":"2024-03-16T09:15:00.882568+0000","last_undegraded":"2024-03-16T09:15:00.882568+0000","last_fullsized":"2024-03-16T09:15:00.882568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:08:39.015162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.517053+0000","last_change":"2024-03-16T09:13:36.107881+0000","last_active":"2024-03-16T09:14:32.517053+0000","last_peered":"2024-03-16T09:14:32.517053+0000","last_clean":"2024-03-16T09:14:32.517053+0000","last_became_active":"2024-03-16T09:13:36.107538+0000","last_became_peered":"2024-03-16T09:13:36.107538+0000","last_unstale":"2024-03-16T09:14:32.517053+0000","last_undegraded":"2024-03-16T09:14:32.517053+0000","last_fullsized":"2024-03-16T09:14:32.517053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:49:41.932089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.923202+0000","last_change":"2024-03-16T09:13:37.118580+0000","last_active":"2024-03-16T09:15:00.923202+0000","last_peered":"2024-03-16T09:15:00.923202+0000","last_clean":"2024-03-16T09:15:00.923202+0000","last_became_active":"2024-03-16T09:13:37.118166+0000","last_became_peered":"2024-03-16T09:13:37.118166+0000","last_unstale":"2024-03-16T09:15:00.923202+0000","last_undegraded":"2024-03-16T09:15:00.923202+0000","last_fullsized":"2024-03-16T09:15:00.923202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:10:13.957439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.998331+0000","last_change":"2024-03-16T09:13:36.122307+0000","last_active":"2024-03-16T09:14:31.998331+0000","last_peered":"2024-03-16T09:14:31.998331+0000","last_clean":"2024-03-16T09:14:31.998331+0000","last_became_active":"2024-03-16T09:13:36.121961+0000","last_became_peered":"2024-03-16T09:13:36.121961+0000","last_unstale":"2024-03-16T09:14:31.998331+0000","last_undegraded":"2024-03-16T09:14:31.998331+0000","last_fullsized":"2024-03-16T09:14:31.998331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T09:50:03.482300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.921973+0000","last_change":"2024-03-16T09:13:37.131918+0000","last_active":"2024-03-16T09:15:00.921973+0000","last_peered":"2024-03-16T09:15:00.921973+0000","last_clean":"2024-03-16T09:15:00.921973+0000","last_became_active":"2024-03-16T09:13:37.131560+0000","last_became_peered":"2024-03-16T09:13:37.131560+0000","last_unstale":"2024-03-16T09:15:00.921973+0000","last_undegraded":"2024-03-16T09:15:00.921973+0000","last_fullsized":"2024-03-16T09:15:00.921973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:57.315125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:31.994873+0000","last_change":"2024-03-16T09:13:36.120192+0000","last_active":"2024-03-16T09:14:31.994873+0000","last_peered":"2024-03-16T09:14:31.994873+0000","last_clean":"2024-03-16T09:14:31.994873+0000","last_became_active":"2024-03-16T09:13:36.119063+0000","last_became_peered":"2024-03-16T09:13:36.119063+0000","last_unstale":"2024-03-16T09:14:31.994873+0000","last_undegraded":"2024-03-16T09:14:31.994873+0000","last_fullsized":"2024-03-16T09:14:31.994873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:18:56.616624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.935639+0000","last_change":"2024-03-16T09:13:37.129597+0000","last_active":"2024-03-16T09:15:00.935639+0000","last_peered":"2024-03-16T09:15:00.935639+0000","last_clean":"2024-03-16T09:15:00.935639+0000","last_became_active":"2024-03-16T09:13:37.129248+0000","last_became_peered":"2024-03-16T09:13:37.129248+0000","last_unstale":"2024-03-16T09:15:00.935639+0000","last_undegraded":"2024-03-16T09:15:00.935639+0000","last_fullsized":"2024-03-16T09:15:00.935639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:41:11.118192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.010274+0000","last_change":"2024-03-16T09:14:32.009577+0000","last_active":"2024-03-16T09:14:32.010274+0000","last_peered":"2024-03-16T09:14:32.010274+0000","last_clean":"2024-03-16T09:14:32.010274+0000","last_became_active":"2024-03-16T09:14:32.000521+0000","last_became_peered":"2024-03-16T09:14:32.000521+0000","last_unstale":"2024-03-16T09:14:32.010274+0000","last_undegraded":"2024-03-16T09:14:32.010274+0000","last_fullsized":"2024-03-16T09:14:32.010274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:29:07.221252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.935373+0000","last_change":"2024-03-16T09:13:37.120317+0000","last_active":"2024-03-16T09:15:00.935373+0000","last_peered":"2024-03-16T09:15:00.935373+0000","last_clean":"2024-03-16T09:15:00.935373+0000","last_became_active":"2024-03-16T09:13:37.119974+0000","last_became_peered":"2024-03-16T09:13:37.119974+0000","last_unstale":"2024-03-16T09:15:00.935373+0000","last_undegraded":"2024-03-16T09:15:00.935373+0000","last_fullsized":"2024-03-16T09:15:00.935373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:25:34.774377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.248531+0000","last_change":"2024-03-16T09:13:36.112395+0000","last_active":"2024-03-16T09:14:32.248531+0000","last_peered":"2024-03-16T09:14:32.248531+0000","last_clean":"2024-03-16T09:14:32.248531+0000","last_became_active":"2024-03-16T09:13:36.112036+0000","last_became_peered":"2024-03-16T09:13:36.112036+0000","last_unstale":"2024-03-16T09:14:32.248531+0000","last_undegraded":"2024-03-16T09:14:32.248531+0000","last_fullsized":"2024-03-16T09:14:32.248531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:41:02.895820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.923449+0000","last_change":"2024-03-16T09:13:37.133845+0000","last_active":"2024-03-16T09:15:00.923449+0000","last_peered":"2024-03-16T09:15:00.923449+0000","last_clean":"2024-03-16T09:15:00.923449+0000","last_became_active":"2024-03-16T09:13:37.132833+0000","last_became_peered":"2024-03-16T09:13:37.132833+0000","last_unstale":"2024-03-16T09:15:00.923449+0000","last_undegraded":"2024-03-16T09:15:00.923449+0000","last_fullsized":"2024-03-16T09:15:00.923449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:12:18.049210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:14:32.385012+0000","last_change":"2024-03-16T09:13:36.114830+0000","last_active":"2024-03-16T09:14:32.385012+0000","last_peered":"2024-03-16T09:14:32.385012+0000","last_clean":"2024-03-16T09:14:32.385012+0000","last_became_active":"2024-03-16T09:13:36.113898+0000","last_became_peered":"2024-03-16T09:13:36.113898+0000","last_unstale":"2024-03-16T09:14:32.385012+0000","last_undegraded":"2024-03-16T09:14:32.385012+0000","last_fullsized":"2024-03-16T09:14:32.385012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:35.090729+0000","last_clean_scrub_stamp":"2024-03-16T09:13:35.090729+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:27:38.910819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.925474+0000","last_change":"2024-03-16T09:13:37.115991+0000","last_active":"2024-03-16T09:15:00.925474+0000","last_peered":"2024-03-16T09:15:00.925474+0000","last_clean":"2024-03-16T09:15:00.925474+0000","last_became_active":"2024-03-16T09:13:37.115671+0000","last_became_peered":"2024-03-16T09:13:37.115671+0000","last_unstale":"2024-03-16T09:15:00.925474+0000","last_undegraded":"2024-03-16T09:15:00.925474+0000","last_fullsized":"2024-03-16T09:15:00.925474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:05:59.703675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.919368+0000","last_change":"2024-03-16T09:13:37.115122+0000","last_active":"2024-03-16T09:15:00.919368+0000","last_peered":"2024-03-16T09:15:00.919368+0000","last_clean":"2024-03-16T09:15:00.919368+0000","last_became_active":"2024-03-16T09:13:37.114779+0000","last_became_peered":"2024-03-16T09:13:37.114779+0000","last_unstale":"2024-03-16T09:15:00.919368+0000","last_undegraded":"2024-03-16T09:15:00.919368+0000","last_fullsized":"2024-03-16T09:15:00.919368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:50:02.051104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.920923+0000","last_change":"2024-03-16T09:13:37.116542+0000","last_active":"2024-03-16T09:15:00.920923+0000","last_peered":"2024-03-16T09:15:00.920923+0000","last_clean":"2024-03-16T09:15:00.920923+0000","last_became_active":"2024-03-16T09:13:37.116224+0000","last_became_peered":"2024-03-16T09:13:37.116224+0000","last_unstale":"2024-03-16T09:15:00.920923+0000","last_undegraded":"2024-03-16T09:15:00.920923+0000","last_fullsized":"2024-03-16T09:15:00.920923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:32:40.106979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.923128+0000","last_change":"2024-03-16T09:13:37.128750+0000","last_active":"2024-03-16T09:15:00.923128+0000","last_peered":"2024-03-16T09:15:00.923128+0000","last_clean":"2024-03-16T09:15:00.923128+0000","last_became_active":"2024-03-16T09:13:37.128411+0000","last_became_peered":"2024-03-16T09:13:37.128411+0000","last_unstale":"2024-03-16T09:15:00.923128+0000","last_undegraded":"2024-03-16T09:15:00.923128+0000","last_fullsized":"2024-03-16T09:15:00.923128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:01:02.119578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.919187+0000","last_change":"2024-03-16T09:13:37.124024+0000","last_active":"2024-03-16T09:15:00.919187+0000","last_peered":"2024-03-16T09:15:00.919187+0000","last_clean":"2024-03-16T09:15:00.919187+0000","last_became_active":"2024-03-16T09:13:37.123721+0000","last_became_peered":"2024-03-16T09:13:37.123721+0000","last_unstale":"2024-03-16T09:15:00.919187+0000","last_undegraded":"2024-03-16T09:15:00.919187+0000","last_fullsized":"2024-03-16T09:15:00.919187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:29:28.079163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.910102+0000","last_change":"2024-03-16T09:13:37.116603+0000","last_active":"2024-03-16T09:15:00.910102+0000","last_peered":"2024-03-16T09:15:00.910102+0000","last_clean":"2024-03-16T09:15:00.910102+0000","last_became_active":"2024-03-16T09:13:37.116279+0000","last_became_peered":"2024-03-16T09:13:37.116279+0000","last_unstale":"2024-03-16T09:15:00.910102+0000","last_undegraded":"2024-03-16T09:15:00.910102+0000","last_fullsized":"2024-03-16T09:15:00.910102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:25:13.001817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.918731+0000","last_change":"2024-03-16T09:13:37.116863+0000","last_active":"2024-03-16T09:15:00.918731+0000","last_peered":"2024-03-16T09:15:00.918731+0000","last_clean":"2024-03-16T09:15:00.918731+0000","last_became_active":"2024-03-16T09:13:37.116543+0000","last_became_peered":"2024-03-16T09:13:37.116543+0000","last_unstale":"2024-03-16T09:15:00.918731+0000","last_undegraded":"2024-03-16T09:15:00.918731+0000","last_fullsized":"2024-03-16T09:15:00.918731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:06:35.501566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.915096+0000","last_change":"2024-03-16T09:13:37.120325+0000","last_active":"2024-03-16T09:15:00.915096+0000","last_peered":"2024-03-16T09:15:00.915096+0000","last_clean":"2024-03-16T09:15:00.915096+0000","last_became_active":"2024-03-16T09:13:37.119985+0000","last_became_peered":"2024-03-16T09:13:37.119985+0000","last_unstale":"2024-03-16T09:15:00.915096+0000","last_undegraded":"2024-03-16T09:15:00.915096+0000","last_fullsized":"2024-03-16T09:15:00.915096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:35.419225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.908253+0000","last_change":"2024-03-16T09:13:37.119604+0000","last_active":"2024-03-16T09:15:00.908253+0000","last_peered":"2024-03-16T09:15:00.908253+0000","last_clean":"2024-03-16T09:15:00.908253+0000","last_became_active":"2024-03-16T09:13:37.119019+0000","last_became_peered":"2024-03-16T09:13:37.119019+0000","last_unstale":"2024-03-16T09:15:00.908253+0000","last_undegraded":"2024-03-16T09:15:00.908253+0000","last_fullsized":"2024-03-16T09:15:00.908253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:42:57.902370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.915549+0000","last_change":"2024-03-16T09:13:37.123845+0000","last_active":"2024-03-16T09:15:00.915549+0000","last_peered":"2024-03-16T09:15:00.915549+0000","last_clean":"2024-03-16T09:15:00.915549+0000","last_became_active":"2024-03-16T09:13:37.121753+0000","last_became_peered":"2024-03-16T09:13:37.121753+0000","last_unstale":"2024-03-16T09:15:00.915549+0000","last_undegraded":"2024-03-16T09:15:00.915549+0000","last_fullsized":"2024-03-16T09:15:00.915549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:23:01.289378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.910994+0000","last_change":"2024-03-16T09:13:37.118182+0000","last_active":"2024-03-16T09:15:00.910994+0000","last_peered":"2024-03-16T09:15:00.910994+0000","last_clean":"2024-03-16T09:15:00.910994+0000","last_became_active":"2024-03-16T09:13:37.117820+0000","last_became_peered":"2024-03-16T09:13:37.117820+0000","last_unstale":"2024-03-16T09:15:00.910994+0000","last_undegraded":"2024-03-16T09:15:00.910994+0000","last_fullsized":"2024-03-16T09:15:00.910994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:27:31.393266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.935406+0000","last_change":"2024-03-16T09:13:37.121348+0000","last_active":"2024-03-16T09:15:00.935406+0000","last_peered":"2024-03-16T09:15:00.935406+0000","last_clean":"2024-03-16T09:15:00.935406+0000","last_became_active":"2024-03-16T09:13:37.120826+0000","last_became_peered":"2024-03-16T09:13:37.120826+0000","last_unstale":"2024-03-16T09:15:00.935406+0000","last_undegraded":"2024-03-16T09:15:00.935406+0000","last_fullsized":"2024-03-16T09:15:00.935406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:36:33.456230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.919550+0000","last_change":"2024-03-16T09:13:37.131889+0000","last_active":"2024-03-16T09:15:00.919550+0000","last_peered":"2024-03-16T09:15:00.919550+0000","last_clean":"2024-03-16T09:15:00.919550+0000","last_became_active":"2024-03-16T09:13:37.131564+0000","last_became_peered":"2024-03-16T09:13:37.131564+0000","last_unstale":"2024-03-16T09:15:00.919550+0000","last_undegraded":"2024-03-16T09:15:00.919550+0000","last_fullsized":"2024-03-16T09:15:00.919550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:43:34.255378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-16T09:15:00.913691+0000","last_change":"2024-03-16T09:14:31.998395+0000","last_active":"2024-03-16T09:15:00.913691+0000","last_peered":"2024-03-16T09:15:00.913691+0000","last_clean":"2024-03-16T09:15:00.913691+0000","last_became_active":"2024-03-16T09:14:31.998063+0000","last_became_peered":"2024-03-16T09:14:31.998063+0000","last_unstale":"2024-03-16T09:15:00.913691+0000","last_undegraded":"2024-03-16T09:15:00.913691+0000","last_fullsized":"2024-03-16T09:15:00.913691+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T09:13:36.094072+0000","last_clean_scrub_stamp":"2024-03-16T09:13:36.094072+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:50:57.125765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280795,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64023,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":728,"num_read_kb":431,"num_write":165,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1164,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":207855,"data_compressed":5547,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383718,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":456779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645354,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":1107700,"data_compressed":37563,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070516,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332154,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":1047310,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":455822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":523921,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":457093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":524689,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":456133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":457232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":1047576,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34075,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35714,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":68306,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69074,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":61600,"data_compressed":1849,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T09:15:40.145 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:15:40.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:39 smithi159 ceph-mon[36739]: pgmap v351: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T09:15:40.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:39 smithi159 ceph-mon[36739]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-16T09:15:40.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:39 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2979006223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.280+0000 7f492c2e8640 1 -- 172.21.15.112:0/4179530056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49241018e0 msgr2=0x7f4924101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.280+0000 7f492c2e8640 1 --2- 172.21.15.112:0/4179530056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49241018e0 0x7f4924101d60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4914009900 tx=0x7f491402bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.280+0000 7f492c2e8640 1 -- 172.21.15.112:0/4179530056 shutdown_connections 2024-03-16T09:15:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.280+0000 7f492c2e8640 1 --2- 172.21.15.112:0/4179530056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49241022a0 0x7f4924108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.280+0000 7f492c2e8640 1 --2- 172.21.15.112:0/4179530056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49241018e0 0x7f4924101d60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.280+0000 7f492c2e8640 1 --2- 172.21.15.112:0/4179530056 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49241006f0 0x7f4924100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.280+0000 7f492c2e8640 1 -- 172.21.15.112:0/4179530056 >> 172.21.15.112:0/4179530056 conn(0x7f49240fbc80 msgr2=0x7f49240fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.280+0000 7f492c2e8640 1 -- 172.21.15.112:0/4179530056 shutdown_connections 2024-03-16T09:15:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.280+0000 7f492c2e8640 1 -- 172.21.15.112:0/4179530056 wait complete. 2024-03-16T09:15:40.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f492c2e8640 1 Processor -- start 2024-03-16T09:15:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f492c2e8640 1 -- start start 2024-03-16T09:15:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f492c2e8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49241006f0 0x7f4924197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f492c2e8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49241018e0 0x7f49241981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f492c2e8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49241022a0 0x7f492419f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f492c2e8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f492410c450 con 0x7f49241022a0 2024-03-16T09:15:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f492c2e8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f492410c2d0 con 0x7f49241018e0 2024-03-16T09:15:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f492c2e8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f492410c5d0 con 0x7f49241006f0 2024-03-16T09:15:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f492a885640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49241022a0 0x7f492419f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f492a885640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49241022a0 0x7f492419f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57288/0 (socket says 172.21.15.112:57288) 2024-03-16T09:15:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f492a885640 1 -- 172.21.15.112:0/2740289474 learned_addr learned my addr 172.21.15.112:0/2740289474 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f492a084640 1 --2- 172.21.15.112:0/2740289474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49241006f0 0x7f4924197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f4929883640 1 --2- 172.21.15.112:0/2740289474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49241018e0 0x7f49241981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f492a885640 1 -- 172.21.15.112:0/2740289474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49241006f0 msgr2=0x7f4924197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:40.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f492a885640 1 --2- 172.21.15.112:0/2740289474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49241006f0 0x7f4924197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f492a885640 1 -- 172.21.15.112:0/2740289474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49241018e0 msgr2=0x7f49241981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:40.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f492a885640 1 --2- 172.21.15.112:0/2740289474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49241018e0 0x7f49241981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f492a885640 1 -- 172.21.15.112:0/2740289474 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492419f870 con 0x7f49241022a0 2024-03-16T09:15:40.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f492a885640 1 --2- 172.21.15.112:0/2740289474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49241022a0 0x7f492419f1d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f491c00efd0 tx=0x7f491c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:40.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f491b7fe640 1 -- 172.21.15.112:0/2740289474 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c003b70 con 0x7f49241022a0 2024-03-16T09:15:40.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f491b7fe640 1 -- 172.21.15.112:0/2740289474 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f491c003d10 con 0x7f49241022a0 2024-03-16T09:15:40.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.281+0000 7f492c2e8640 1 -- 172.21.15.112:0/2740289474 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f492419fb60 con 0x7f49241022a0 2024-03-16T09:15:40.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.282+0000 7f491b7fe640 1 -- 172.21.15.112:0/2740289474 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c010a90 con 0x7f49241022a0 2024-03-16T09:15:40.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.282+0000 7f492c2e8640 1 -- 172.21.15.112:0/2740289474 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49241a0060 con 0x7f49241022a0 2024-03-16T09:15:40.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.282+0000 7f492c2e8640 1 -- 172.21.15.112:0/2740289474 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49241a0230 con 0x7f49241022a0 2024-03-16T09:15:40.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.283+0000 7f491b7fe640 1 -- 172.21.15.112:0/2740289474 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7f491c004020 con 0x7f49241022a0 2024-03-16T09:15:40.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.284+0000 7f491b7fe640 1 --2- 172.21.15.112:0/2740289474 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f49100a2890 0x7f49100a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:40.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.284+0000 7f492a084640 1 --2- 172.21.15.112:0/2740289474 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f49100a2890 0x7f49100a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:40.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.284+0000 7f491b7fe640 1 -- 172.21.15.112:0/2740289474 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f491c0cf0b0 con 0x7f49241022a0 2024-03-16T09:15:40.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.284+0000 7f492a084640 1 --2- 172.21.15.112:0/2740289474 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f49100a2890 0x7f49100a4d50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4924199130 tx=0x7f4920005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:40.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.288+0000 7f491b7fe640 1 -- 172.21.15.112:0/2740289474 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f491c09c000 con 0x7f49241022a0 2024-03-16T09:15:40.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:39 smithi194 ceph-mon[36729]: pgmap v351: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T09:15:40.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:39 smithi194 ceph-mon[36729]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-16T09:15:40.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:39 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2979006223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:40.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.465+0000 7f492c2e8640 1 -- 172.21.15.112:0/2740289474 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49241a04a0 con 0x7f49241022a0 2024-03-16T09:15:40.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.466+0000 7f491b7fe640 1 -- 172.21.15.112:0/2740289474 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22725 (secure 0 0 0) 0x7f491c09b900 con 0x7f49241022a0 2024-03-16T09:15:40.469 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:40.469 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":87,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:14:31.991998+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T09:13:35.040628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T09:13:35.689419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:15:40.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.469+0000 7f492c2e8640 1 -- 172.21.15.112:0/2740289474 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f49100a2890 msgr2=0x7f49100a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:40.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.469+0000 7f492c2e8640 1 --2- 172.21.15.112:0/2740289474 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f49100a2890 0x7f49100a4d50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4924199130 tx=0x7f4920005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:40.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.469+0000 7f492c2e8640 1 -- 172.21.15.112:0/2740289474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49241022a0 msgr2=0x7f492419f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:40.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.469+0000 7f492c2e8640 1 --2- 172.21.15.112:0/2740289474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49241022a0 0x7f492419f1d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f491c00efd0 tx=0x7f491c00c6a0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.469+0000 7f492c2e8640 1 -- 172.21.15.112:0/2740289474 shutdown_connections 2024-03-16T09:15:40.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.469+0000 7f492c2e8640 1 --2- 172.21.15.112:0/2740289474 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f49100a2890 0x7f49100a4d50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.469+0000 7f492c2e8640 1 --2- 172.21.15.112:0/2740289474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49241022a0 0x7f492419f1d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.469+0000 7f492c2e8640 1 --2- 172.21.15.112:0/2740289474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49241018e0 0x7f49241981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.469+0000 7f492c2e8640 1 --2- 172.21.15.112:0/2740289474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49241006f0 0x7f4924197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.469+0000 7f492c2e8640 1 -- 172.21.15.112:0/2740289474 >> 172.21.15.112:0/2740289474 conn(0x7f49240fbc80 msgr2=0x7f49240fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:40.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.469+0000 7f492c2e8640 1 -- 172.21.15.112:0/2740289474 shutdown_connections 2024-03-16T09:15:40.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.469+0000 7f492c2e8640 1 -- 172.21.15.112:0/2740289474 wait complete. 2024-03-16T09:15:40.482 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T09:08:57.135312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T09:13:35.040628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T09:13:35.689419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T09:15:40.483 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T09:15:40.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.586+0000 7fc85059d640 1 -- 172.21.15.112:0/1336736457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8481006f0 msgr2=0x7fc848100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:40.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.586+0000 7fc85059d640 1 --2- 172.21.15.112:0/1336736457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8481006f0 0x7fc848100b10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc838009900 tx=0x7fc83802bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:40.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.586+0000 7fc85059d640 1 -- 172.21.15.112:0/1336736457 shutdown_connections 2024-03-16T09:15:40.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.586+0000 7fc85059d640 1 --2- 172.21.15.112:0/1336736457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8481022a0 0x7fc848108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.586+0000 7fc85059d640 1 --2- 172.21.15.112:0/1336736457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8481018e0 0x7fc848101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.586+0000 7fc85059d640 1 --2- 172.21.15.112:0/1336736457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8481006f0 0x7fc848100b10 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.586+0000 7fc85059d640 1 -- 172.21.15.112:0/1336736457 >> 172.21.15.112:0/1336736457 conn(0x7fc8480fbc80 msgr2=0x7fc8480fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:40.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.586+0000 7fc85059d640 1 -- 172.21.15.112:0/1336736457 shutdown_connections 2024-03-16T09:15:40.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.586+0000 7fc85059d640 1 -- 172.21.15.112:0/1336736457 wait complete. 2024-03-16T09:15:40.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc85059d640 1 Processor -- start 2024-03-16T09:15:40.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc85059d640 1 -- start start 2024-03-16T09:15:40.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc85059d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8481006f0 0x7fc848197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:40.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc85059d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8481018e0 0x7fc848198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:40.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc85059d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8481022a0 0x7fc84819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:40.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc85059d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc84810c450 con 0x7fc8481022a0 2024-03-16T09:15:40.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc85059d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc84810c2d0 con 0x7fc8481006f0 2024-03-16T09:15:40.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc85059d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc84810c5d0 con 0x7fc8481018e0 2024-03-16T09:15:40.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc84eb3a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8481022a0 0x7fc84819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:40.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc84db38640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8481018e0 0x7fc848198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:40.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc84eb3a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8481022a0 0x7fc84819f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57316/0 (socket says 172.21.15.112:57316) 2024-03-16T09:15:40.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc84eb3a640 1 -- 172.21.15.112:0/2034208934 learned_addr learned my addr 172.21.15.112:0/2034208934 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:40.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc84e339640 1 --2- 172.21.15.112:0/2034208934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8481006f0 0x7fc848197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:40.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc84eb3a640 1 -- 172.21.15.112:0/2034208934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8481018e0 msgr2=0x7fc848198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:40.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc84eb3a640 1 --2- 172.21.15.112:0/2034208934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8481018e0 0x7fc848198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc84eb3a640 1 -- 172.21.15.112:0/2034208934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8481006f0 msgr2=0x7fc848197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:40.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc84eb3a640 1 --2- 172.21.15.112:0/2034208934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8481006f0 0x7fc848197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc84eb3a640 1 -- 172.21.15.112:0/2034208934 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84819f860 con 0x7fc8481022a0 2024-03-16T09:15:40.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc84db38640 1 --2- 172.21.15.112:0/2034208934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8481018e0 0x7fc848198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:40.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc84eb3a640 1 --2- 172.21.15.112:0/2034208934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8481022a0 0x7fc84819f1c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc84000eb10 tx=0x7fc84000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:40.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.587+0000 7fc83f7fe640 1 -- 172.21.15.112:0/2034208934 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840004b20 con 0x7fc8481022a0 2024-03-16T09:15:40.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.588+0000 7fc85059d640 1 -- 172.21.15.112:0/2034208934 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84819fb50 con 0x7fc8481022a0 2024-03-16T09:15:40.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.588+0000 7fc83f7fe640 1 -- 172.21.15.112:0/2034208934 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc840004cc0 con 0x7fc8481022a0 2024-03-16T09:15:40.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.588+0000 7fc83f7fe640 1 -- 172.21.15.112:0/2034208934 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8400107f0 con 0x7fc8481022a0 2024-03-16T09:15:40.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.588+0000 7fc85059d640 1 -- 172.21.15.112:0/2034208934 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8481a0070 con 0x7fc8481022a0 2024-03-16T09:15:40.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.589+0000 7fc85059d640 1 -- 172.21.15.112:0/2034208934 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8481a0240 con 0x7fc8481022a0 2024-03-16T09:15:40.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.590+0000 7fc83f7fe640 1 -- 172.21.15.112:0/2034208934 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7fc840005020 con 0x7fc8481022a0 2024-03-16T09:15:40.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.590+0000 7fc83f7fe640 1 --2- 172.21.15.112:0/2034208934 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc8340a2890 0x7fc8340a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:40.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.590+0000 7fc84e339640 1 --2- 172.21.15.112:0/2034208934 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc8340a2890 0x7fc8340a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:40.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.590+0000 7fc83f7fe640 1 -- 172.21.15.112:0/2034208934 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fc8400cf200 con 0x7fc8481022a0 2024-03-16T09:15:40.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.591+0000 7fc84e339640 1 --2- 172.21.15.112:0/2034208934 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc8340a2890 0x7fc8340a4d50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc838000c00 tx=0x7fc838005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:40.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.595+0000 7fc83f7fe640 1 -- 172.21.15.112:0/2034208934 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc84009c150 con 0x7fc8481022a0 2024-03-16T09:15:40.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.767+0000 7fc85059d640 1 -- 172.21.15.112:0/2034208934 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc8481a0410 con 0x7fc8481022a0 2024-03-16T09:15:40.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.768+0000 7fc83f7fe640 1 -- 172.21.15.112:0/2034208934 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fc84009ba50 con 0x7fc8481022a0 2024-03-16T09:15:40.770 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-03-16T09:15:40.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.770+0000 7fc85059d640 1 -- 172.21.15.112:0/2034208934 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc8340a2890 msgr2=0x7fc8340a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:40.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.770+0000 7fc85059d640 1 --2- 172.21.15.112:0/2034208934 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc8340a2890 0x7fc8340a4d50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc838000c00 tx=0x7fc838005fd0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.770+0000 7fc85059d640 1 -- 172.21.15.112:0/2034208934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8481022a0 msgr2=0x7fc84819f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:40.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.770+0000 7fc85059d640 1 --2- 172.21.15.112:0/2034208934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8481022a0 0x7fc84819f1c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc84000eb10 tx=0x7fc84000c2d0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.770+0000 7fc85059d640 1 -- 172.21.15.112:0/2034208934 shutdown_connections 2024-03-16T09:15:40.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.770+0000 7fc85059d640 1 --2- 172.21.15.112:0/2034208934 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc8340a2890 0x7fc8340a4d50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.770+0000 7fc85059d640 1 --2- 172.21.15.112:0/2034208934 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8481022a0 0x7fc84819f1c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.770+0000 7fc85059d640 1 --2- 172.21.15.112:0/2034208934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc8481018e0 0x7fc848198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.770+0000 7fc85059d640 1 --2- 172.21.15.112:0/2034208934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8481006f0 0x7fc848197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.770+0000 7fc85059d640 1 -- 172.21.15.112:0/2034208934 >> 172.21.15.112:0/2034208934 conn(0x7fc8480fbc80 msgr2=0x7fc8480fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:40.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.770+0000 7fc85059d640 1 -- 172.21.15.112:0/2034208934 shutdown_connections 2024-03-16T09:15:40.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.770+0000 7fc85059d640 1 -- 172.21.15.112:0/2034208934 wait complete. 2024-03-16T09:15:40.783 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T09:15:40.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.885+0000 7fd61c9d4640 1 -- 172.21.15.112:0/2673292751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd618104bf0 msgr2=0x7fd618106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:40.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.885+0000 7fd61c9d4640 1 --2- 172.21.15.112:0/2673292751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd618104bf0 0x7fd618106fe0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd600009900 tx=0x7fd60002bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:40.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.885+0000 7fd61c9d4640 1 -- 172.21.15.112:0/2673292751 shutdown_connections 2024-03-16T09:15:40.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.885+0000 7fd61c9d4640 1 --2- 172.21.15.112:0/2673292751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd618107520 0x7fd618109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.885+0000 7fd61c9d4640 1 --2- 172.21.15.112:0/2673292751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd618104bf0 0x7fd618106fe0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.885+0000 7fd61c9d4640 1 --2- 172.21.15.112:0/2673292751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6181022c0 0x7fd6181046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.885+0000 7fd61c9d4640 1 -- 172.21.15.112:0/2673292751 >> 172.21.15.112:0/2673292751 conn(0x7fd6180fbc70 msgr2=0x7fd6180fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:40.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.885+0000 7fd61c9d4640 1 -- 172.21.15.112:0/2673292751 shutdown_connections 2024-03-16T09:15:40.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.885+0000 7fd61c9d4640 1 -- 172.21.15.112:0/2673292751 wait complete. 2024-03-16T09:15:40.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.886+0000 7fd61c9d4640 1 Processor -- start 2024-03-16T09:15:40.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.886+0000 7fd61c9d4640 1 -- start start 2024-03-16T09:15:40.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.886+0000 7fd61c9d4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6181022c0 0x7fd618197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:40.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.886+0000 7fd61c9d4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd618104bf0 0x7fd618198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:40.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.886+0000 7fd61c9d4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd618107520 0x7fd61819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:40.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.886+0000 7fd61c9d4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd61810d520 con 0x7fd6181022c0 2024-03-16T09:15:40.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.886+0000 7fd61c9d4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd61810d3a0 con 0x7fd618104bf0 2024-03-16T09:15:40.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.886+0000 7fd61c9d4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd61810d6a0 con 0x7fd618107520 2024-03-16T09:15:40.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.886+0000 7fd61659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6181022c0 0x7fd618197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:40.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.886+0000 7fd61659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6181022c0 0x7fd618197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57330/0 (socket says 172.21.15.112:57330) 2024-03-16T09:15:40.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.886+0000 7fd61659c640 1 -- 172.21.15.112:0/277762392 learned_addr learned my addr 172.21.15.112:0/277762392 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:40.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.886+0000 7fd616d9d640 1 --2- 172.21.15.112:0/277762392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd618107520 0x7fd61819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:40.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.886+0000 7fd60ffff640 1 --2- 172.21.15.112:0/277762392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd618104bf0 0x7fd618198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:40.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.886+0000 7fd61659c640 1 -- 172.21.15.112:0/277762392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd618107520 msgr2=0x7fd61819f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:40.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.886+0000 7fd61659c640 1 --2- 172.21.15.112:0/277762392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd618107520 0x7fd61819f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.886+0000 7fd61659c640 1 -- 172.21.15.112:0/277762392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd618104bf0 msgr2=0x7fd618198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:40.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.886+0000 7fd61659c640 1 --2- 172.21.15.112:0/277762392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd618104bf0 0x7fd618198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:40.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.886+0000 7fd61659c640 1 -- 172.21.15.112:0/277762392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61819f850 con 0x7fd6181022c0 2024-03-16T09:15:40.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.886+0000 7fd61659c640 1 --2- 172.21.15.112:0/277762392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6181022c0 0x7fd618197c40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd60800e9f0 tx=0x7fd60800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:40.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.887+0000 7fd60f7fe640 1 -- 172.21.15.112:0/277762392 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60800cd80 con 0x7fd6181022c0 2024-03-16T09:15:40.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.887+0000 7fd60f7fe640 1 -- 172.21.15.112:0/277762392 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd608010930 con 0x7fd6181022c0 2024-03-16T09:15:40.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.887+0000 7fd60f7fe640 1 -- 172.21.15.112:0/277762392 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608018b00 con 0x7fd6181022c0 2024-03-16T09:15:40.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.887+0000 7fd61c9d4640 1 -- 172.21.15.112:0/277762392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61819fb40 con 0x7fd6181022c0 2024-03-16T09:15:40.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.887+0000 7fd61c9d4640 1 -- 172.21.15.112:0/277762392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6181a00e0 con 0x7fd6181022c0 2024-03-16T09:15:40.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.888+0000 7fd61c9d4640 1 -- 172.21.15.112:0/277762392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6180fd760 con 0x7fd6181022c0 2024-03-16T09:15:40.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.889+0000 7fd60f7fe640 1 -- 172.21.15.112:0/277762392 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7fd608018ca0 con 0x7fd6181022c0 2024-03-16T09:15:40.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.889+0000 7fd60f7fe640 1 --2- 172.21.15.112:0/277762392 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6040a2910 0x7fd6040a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:40.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.889+0000 7fd60ffff640 1 --2- 172.21.15.112:0/277762392 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6040a2910 0x7fd6040a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:40.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.889+0000 7fd60f7fe640 1 -- 172.21.15.112:0/277762392 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd6080cef60 con 0x7fd6181022c0 2024-03-16T09:15:40.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.890+0000 7fd60ffff640 1 --2- 172.21.15.112:0/277762392 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6040a2910 0x7fd6040a4dd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd600000c00 tx=0x7fd600005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:40.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:40.893+0000 7fd60f7fe640 1 -- 172.21.15.112:0/277762392 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd60809b860 con 0x7fd6181022c0 2024-03-16T09:15:41.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.071+0000 7fd61c9d4640 1 -- 172.21.15.112:0/277762392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd61802d420 con 0x7fd6181022c0 2024-03-16T09:15:41.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.071+0000 7fd60f7fe640 1 -- 172.21.15.112:0/277762392 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7fd60809f900 con 0x7fd6181022c0 2024-03-16T09:15:41.073 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-03-16T09:15:41.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.073+0000 7fd61c9d4640 1 -- 172.21.15.112:0/277762392 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6040a2910 msgr2=0x7fd6040a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:41.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.073+0000 7fd61c9d4640 1 --2- 172.21.15.112:0/277762392 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6040a2910 0x7fd6040a4dd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd600000c00 tx=0x7fd600005fd0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.073+0000 7fd61c9d4640 1 -- 172.21.15.112:0/277762392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6181022c0 msgr2=0x7fd618197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:41.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.073+0000 7fd61c9d4640 1 --2- 172.21.15.112:0/277762392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6181022c0 0x7fd618197c40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd60800e9f0 tx=0x7fd60800eeb0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.073+0000 7fd61c9d4640 1 -- 172.21.15.112:0/277762392 shutdown_connections 2024-03-16T09:15:41.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.073+0000 7fd61c9d4640 1 --2- 172.21.15.112:0/277762392 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6040a2910 0x7fd6040a4dd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.073+0000 7fd61c9d4640 1 --2- 172.21.15.112:0/277762392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd618107520 0x7fd61819f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.073+0000 7fd61c9d4640 1 --2- 172.21.15.112:0/277762392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd618104bf0 0x7fd618198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.073+0000 7fd61c9d4640 1 --2- 172.21.15.112:0/277762392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6181022c0 0x7fd618197c40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.073+0000 7fd61c9d4640 1 -- 172.21.15.112:0/277762392 >> 172.21.15.112:0/277762392 conn(0x7fd6180fbc70 msgr2=0x7fd618102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:41.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.073+0000 7fd61c9d4640 1 -- 172.21.15.112:0/277762392 shutdown_connections 2024-03-16T09:15:41.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.073+0000 7fd61c9d4640 1 -- 172.21.15.112:0/277762392 wait complete. 2024-03-16T09:15:41.085 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:40 smithi112 ceph-mon[32307]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:15:41.085 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:40 smithi112 ceph-mon[32307]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:15:41.085 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:40 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2740289474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:41.085 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:40 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2034208934' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:41.086 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T09:15:41.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.189+0000 7f0666eb2640 1 -- 172.21.15.112:0/1406507322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0660107520 msgr2=0x7f0660109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:41.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.189+0000 7f0666eb2640 1 --2- 172.21.15.112:0/1406507322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0660107520 0x7f0660109910 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0658009ca0 tx=0x7f065802c0f0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.189+0000 7f0666eb2640 1 -- 172.21.15.112:0/1406507322 shutdown_connections 2024-03-16T09:15:41.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.189+0000 7f0666eb2640 1 --2- 172.21.15.112:0/1406507322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0660107520 0x7f0660109910 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.189+0000 7f0666eb2640 1 --2- 172.21.15.112:0/1406507322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0660104bf0 0x7f0660106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.189+0000 7f0666eb2640 1 --2- 172.21.15.112:0/1406507322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06601022c0 0x7f06601046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.190+0000 7f0666eb2640 1 -- 172.21.15.112:0/1406507322 >> 172.21.15.112:0/1406507322 conn(0x7f06600fbc70 msgr2=0x7f06600fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:41.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.190+0000 7f0666eb2640 1 -- 172.21.15.112:0/1406507322 shutdown_connections 2024-03-16T09:15:41.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.190+0000 7f0666eb2640 1 -- 172.21.15.112:0/1406507322 wait complete. 2024-03-16T09:15:41.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.190+0000 7f0666eb2640 1 Processor -- start 2024-03-16T09:15:41.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.190+0000 7f0666eb2640 1 -- start start 2024-03-16T09:15:41.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.190+0000 7f0666eb2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06601022c0 0x7f0660197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:41.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.190+0000 7f0666eb2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0660104bf0 0x7f06601980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:41.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.190+0000 7f0666eb2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0660107520 0x7f066019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:41.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.190+0000 7f0666eb2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f066010d5b0 con 0x7f0660104bf0 2024-03-16T09:15:41.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.190+0000 7f0666eb2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f066010d430 con 0x7f06601022c0 2024-03-16T09:15:41.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.190+0000 7f0666eb2640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f066010d730 con 0x7f0660107520 2024-03-16T09:15:41.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.190+0000 7f065ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0660104bf0 0x7f06601980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:41.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.190+0000 7f0664c4e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06601022c0 0x7f0660197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:41.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.190+0000 7f065ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0660104bf0 0x7f06601980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57346/0 (socket says 172.21.15.112:57346) 2024-03-16T09:15:41.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.190+0000 7f0664c4e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06601022c0 0x7f0660197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:39748/0 (socket says 172.21.15.112:39748) 2024-03-16T09:15:41.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.190+0000 7f065ffff640 1 -- 172.21.15.112:0/4123697086 learned_addr learned my addr 172.21.15.112:0/4123697086 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:41.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.190+0000 7f066544f640 1 --2- 172.21.15.112:0/4123697086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0660107520 0x7f066019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:41.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.191+0000 7f065ffff640 1 -- 172.21.15.112:0/4123697086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0660107520 msgr2=0x7f066019f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:41.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.191+0000 7f065ffff640 1 --2- 172.21.15.112:0/4123697086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0660107520 0x7f066019f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.191+0000 7f065ffff640 1 -- 172.21.15.112:0/4123697086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06601022c0 msgr2=0x7f0660197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:41.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.191+0000 7f065ffff640 1 --2- 172.21.15.112:0/4123697086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06601022c0 0x7f0660197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.191+0000 7f065ffff640 1 -- 172.21.15.112:0/4123697086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066019f7b0 con 0x7f0660104bf0 2024-03-16T09:15:41.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.191+0000 7f065ffff640 1 --2- 172.21.15.112:0/4123697086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0660104bf0 0x7f06601980d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f065400e9c0 tx=0x7f065400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:41.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.191+0000 7f065dffb640 1 -- 172.21.15.112:0/4123697086 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065400cd80 con 0x7f0660104bf0 2024-03-16T09:15:41.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.191+0000 7f065dffb640 1 -- 172.21.15.112:0/4123697086 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0654005520 con 0x7f0660104bf0 2024-03-16T09:15:41.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.191+0000 7f0666eb2640 1 -- 172.21.15.112:0/4123697086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066019faa0 con 0x7f0660104bf0 2024-03-16T09:15:41.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.191+0000 7f065dffb640 1 -- 172.21.15.112:0/4123697086 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654010690 con 0x7f0660104bf0 2024-03-16T09:15:41.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.191+0000 7f0666eb2640 1 -- 172.21.15.112:0/4123697086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f066019fff0 con 0x7f0660104bf0 2024-03-16T09:15:41.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.192+0000 7f0666eb2640 1 -- 172.21.15.112:0/4123697086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06600fd760 con 0x7f0660104bf0 2024-03-16T09:15:41.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.193+0000 7f065dffb640 1 -- 172.21.15.112:0/4123697086 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7f06540046c0 con 0x7f0660104bf0 2024-03-16T09:15:41.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.194+0000 7f065dffb640 1 --2- 172.21.15.112:0/4123697086 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f06480a2890 0x7f06480a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:41.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.194+0000 7f0664c4e640 1 --2- 172.21.15.112:0/4123697086 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f06480a2890 0x7f06480a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:41.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.194+0000 7f065dffb640 1 -- 172.21.15.112:0/4123697086 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f06540cef40 con 0x7f0660104bf0 2024-03-16T09:15:41.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.194+0000 7f0664c4e640 1 --2- 172.21.15.112:0/4123697086 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f06480a2890 0x7f06480a4d50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0650009bd0 tx=0x7f0650005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:41.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.198+0000 7f065dffb640 1 -- 172.21.15.112:0/4123697086 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f065409be90 con 0x7f0660104bf0 2024-03-16T09:15:41.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:40 smithi159 ceph-mon[36739]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:15:41.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:40 smithi159 ceph-mon[36739]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:15:41.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:40 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2740289474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:41.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:40 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2034208934' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:41.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.373+0000 7f0666eb2640 1 -- 172.21.15.112:0/4123697086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f066002d3c0 con 0x7f0660104bf0 2024-03-16T09:15:41.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.374+0000 7f065dffb640 1 -- 172.21.15.112:0/4123697086 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f065409b790 con 0x7f0660104bf0 2024-03-16T09:15:41.376 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-03-16T09:15:41.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.376+0000 7f0666eb2640 1 -- 172.21.15.112:0/4123697086 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f06480a2890 msgr2=0x7f06480a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:41.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.376+0000 7f0666eb2640 1 --2- 172.21.15.112:0/4123697086 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f06480a2890 0x7f06480a4d50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0650009bd0 tx=0x7f0650005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:41.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.376+0000 7f0666eb2640 1 -- 172.21.15.112:0/4123697086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0660104bf0 msgr2=0x7f06601980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:41.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.376+0000 7f0666eb2640 1 --2- 172.21.15.112:0/4123697086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0660104bf0 0x7f06601980d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f065400e9c0 tx=0x7f065400ee80 comp rx=0 tx=0).stop 2024-03-16T09:15:41.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.376+0000 7f0666eb2640 1 -- 172.21.15.112:0/4123697086 shutdown_connections 2024-03-16T09:15:41.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.376+0000 7f0666eb2640 1 --2- 172.21.15.112:0/4123697086 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f06480a2890 0x7f06480a4d50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.376+0000 7f0666eb2640 1 --2- 172.21.15.112:0/4123697086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0660107520 0x7f066019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.376+0000 7f0666eb2640 1 --2- 172.21.15.112:0/4123697086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0660104bf0 0x7f06601980d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.376+0000 7f0666eb2640 1 --2- 172.21.15.112:0/4123697086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06601022c0 0x7f0660197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.376+0000 7f0666eb2640 1 -- 172.21.15.112:0/4123697086 >> 172.21.15.112:0/4123697086 conn(0x7f06600fbc70 msgr2=0x7f0660102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:41.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.376+0000 7f0666eb2640 1 -- 172.21.15.112:0/4123697086 shutdown_connections 2024-03-16T09:15:41.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.376+0000 7f0666eb2640 1 -- 172.21.15.112:0/4123697086 wait complete. 2024-03-16T09:15:41.389 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:15:41.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:40 smithi194 ceph-mon[36729]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:15:41.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:40 smithi194 ceph-mon[36729]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:15:41.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:40 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2740289474' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:41.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:40 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2034208934' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:15:41.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.489+0000 7fa966197640 1 -- 172.21.15.112:0/155202798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9600ffaf0 msgr2=0x7fa9600fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:41.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.489+0000 7fa966197640 1 --2- 172.21.15.112:0/155202798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9600ffaf0 0x7fa9600fff10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa950009930 tx=0x7fa95002bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:41.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.489+0000 7fa966197640 1 -- 172.21.15.112:0/155202798 shutdown_connections 2024-03-16T09:15:41.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.489+0000 7fa966197640 1 --2- 172.21.15.112:0/155202798 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa960103480 0x7fa960109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.489+0000 7fa966197640 1 --2- 172.21.15.112:0/155202798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa960102ac0 0x7fa960102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.489+0000 7fa966197640 1 --2- 172.21.15.112:0/155202798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9600ffaf0 0x7fa9600fff10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.489+0000 7fa966197640 1 -- 172.21.15.112:0/155202798 >> 172.21.15.112:0/155202798 conn(0x7fa9600fb420 msgr2=0x7fa9600fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:41.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.489+0000 7fa966197640 1 -- 172.21.15.112:0/155202798 shutdown_connections 2024-03-16T09:15:41.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.489+0000 7fa966197640 1 -- 172.21.15.112:0/155202798 wait complete. 2024-03-16T09:15:41.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.490+0000 7fa966197640 1 Processor -- start 2024-03-16T09:15:41.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.490+0000 7fa966197640 1 -- start start 2024-03-16T09:15:41.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.490+0000 7fa966197640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9600ffaf0 0x7fa960197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:41.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.490+0000 7fa966197640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa960102ac0 0x7fa9601981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:41.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.490+0000 7fa966197640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa960103480 0x7fa96019f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:41.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.490+0000 7fa966197640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa96010d590 con 0x7fa9600ffaf0 2024-03-16T09:15:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.490+0000 7fa966197640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa96010d410 con 0x7fa960103480 2024-03-16T09:15:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.490+0000 7fa966197640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa96010d710 con 0x7fa960102ac0 2024-03-16T09:15:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.490+0000 7fa95f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9600ffaf0 0x7fa960197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.490+0000 7fa95f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9600ffaf0 0x7fa960197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52150/0 (socket says 172.21.15.112:52150) 2024-03-16T09:15:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.490+0000 7fa95f7fe640 1 -- 172.21.15.112:0/1406019233 learned_addr learned my addr 172.21.15.112:0/1406019233 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.490+0000 7fa95effd640 1 --2- 172.21.15.112:0/1406019233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa960102ac0 0x7fa9601981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.490+0000 7fa95ffff640 1 --2- 172.21.15.112:0/1406019233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa960103480 0x7fa96019f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.490+0000 7fa95f7fe640 1 -- 172.21.15.112:0/1406019233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa960102ac0 msgr2=0x7fa9601981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.490+0000 7fa95f7fe640 1 --2- 172.21.15.112:0/1406019233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa960102ac0 0x7fa9601981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.490+0000 7fa95f7fe640 1 -- 172.21.15.112:0/1406019233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa960103480 msgr2=0x7fa96019f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.490+0000 7fa95f7fe640 1 --2- 172.21.15.112:0/1406019233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa960103480 0x7fa96019f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.490+0000 7fa95f7fe640 1 -- 172.21.15.112:0/1406019233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96019f870 con 0x7fa9600ffaf0 2024-03-16T09:15:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.491+0000 7fa95f7fe640 1 --2- 172.21.15.112:0/1406019233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9600ffaf0 0x7fa960197c60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa95002c900 tx=0x7fa950004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.491+0000 7fa95cff9640 1 -- 172.21.15.112:0/1406019233 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950043020 con 0x7fa9600ffaf0 2024-03-16T09:15:41.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.491+0000 7fa966197640 1 -- 172.21.15.112:0/1406019233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96019fb00 con 0x7fa9600ffaf0 2024-03-16T09:15:41.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.491+0000 7fa95cff9640 1 -- 172.21.15.112:0/1406019233 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa950004250 con 0x7fa9600ffaf0 2024-03-16T09:15:41.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.491+0000 7fa95cff9640 1 -- 172.21.15.112:0/1406019233 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950003a50 con 0x7fa9600ffaf0 2024-03-16T09:15:41.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.491+0000 7fa966197640 1 -- 172.21.15.112:0/1406019233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9601a0050 con 0x7fa9600ffaf0 2024-03-16T09:15:41.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.492+0000 7fa966197640 1 -- 172.21.15.112:0/1406019233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa960068a20 con 0x7fa9600ffaf0 2024-03-16T09:15:41.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.493+0000 7fa95cff9640 1 -- 172.21.15.112:0/1406019233 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7fa950046020 con 0x7fa9600ffaf0 2024-03-16T09:15:41.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.493+0000 7fa95cff9640 1 --2- 172.21.15.112:0/1406019233 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa9480a2890 0x7fa9480a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:41.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.493+0000 7fa95effd640 1 --2- 172.21.15.112:0/1406019233 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa9480a2890 0x7fa9480a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:41.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.494+0000 7fa95cff9640 1 -- 172.21.15.112:0/1406019233 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa9500f0bd0 con 0x7fa9600ffaf0 2024-03-16T09:15:41.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.494+0000 7fa95effd640 1 --2- 172.21.15.112:0/1406019233 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa9480a2890 0x7fa9480a4d50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa954009bd0 tx=0x7fa954005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:41.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.498+0000 7fa95cff9640 1 -- 172.21.15.112:0/1406019233 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa95004d020 con 0x7fa9600ffaf0 2024-03-16T09:15:41.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.708+0000 7fa966197640 1 -- 172.21.15.112:0/1406019233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa96002d1b0 con 0x7fa9600ffaf0 2024-03-16T09:15:41.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.709+0000 7fa95cff9640 1 -- 172.21.15.112:0/1406019233 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11881 (secure 0 0 0) 0x7fa9500bdc30 con 0x7fa9600ffaf0 2024-03-16T09:15:41.711 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:41.711 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3939748854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3939748854},{"type":"v1","addr":"172.21.15.112:6839","nonce":3939748854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3118523258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3118523258},{"type":"v1","addr":"172.21.15.112:6841","nonce":3118523258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T09:13:36.450216+0000","modified":"2024-03-16T09:15:34.982470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/1682076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":1682076},{"type":"v1","addr":"172.21.15.194:6837","nonce":1682076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.194:6839/2528943309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2528943309},{"type":"v1","addr":"172.21.15.194:6839","nonce":2528943309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/272895965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":272895965},{"type":"v1","addr":"172.21.15.159:6839","nonce":272895965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24452,24466]},"id":1}]} 2024-03-16T09:15:41.711 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 22 2024-03-16T09:15:41.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.710+0000 7fa966197640 1 -- 172.21.15.112:0/1406019233 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa9480a2890 msgr2=0x7fa9480a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:41.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.710+0000 7fa966197640 1 --2- 172.21.15.112:0/1406019233 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa9480a2890 0x7fa9480a4d50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa954009bd0 tx=0x7fa954005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:41.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.710+0000 7fa966197640 1 -- 172.21.15.112:0/1406019233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9600ffaf0 msgr2=0x7fa960197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:41.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.710+0000 7fa966197640 1 --2- 172.21.15.112:0/1406019233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9600ffaf0 0x7fa960197c60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa95002c900 tx=0x7fa950004a30 comp rx=0 tx=0).stop 2024-03-16T09:15:41.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.711+0000 7fa966197640 1 -- 172.21.15.112:0/1406019233 shutdown_connections 2024-03-16T09:15:41.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.711+0000 7fa966197640 1 --2- 172.21.15.112:0/1406019233 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa9480a2890 0x7fa9480a4d50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.711+0000 7fa966197640 1 --2- 172.21.15.112:0/1406019233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa960103480 0x7fa96019f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.711+0000 7fa966197640 1 --2- 172.21.15.112:0/1406019233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa960102ac0 0x7fa9601981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.711+0000 7fa966197640 1 --2- 172.21.15.112:0/1406019233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9600ffaf0 0x7fa960197c60 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.711+0000 7fa966197640 1 -- 172.21.15.112:0/1406019233 >> 172.21.15.112:0/1406019233 conn(0x7fa9600fb420 msgr2=0x7fa960105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:41.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.711+0000 7fa966197640 1 -- 172.21.15.112:0/1406019233 shutdown_connections 2024-03-16T09:15:41.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.711+0000 7fa966197640 1 -- 172.21.15.112:0/1406019233 wait complete. 2024-03-16T09:15:41.724 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:15:41.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.823+0000 7f6adc3eb640 1 -- 172.21.15.112:0/2019620818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ad41022b0 msgr2=0x7f6ad4108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:41.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.823+0000 7f6adc3eb640 1 --2- 172.21.15.112:0/2019620818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ad41022b0 0x7f6ad4108b40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009ca0 tx=0x7f6acc02c100 comp rx=0 tx=0).stop 2024-03-16T09:15:41.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.824+0000 7f6adc3eb640 1 -- 172.21.15.112:0/2019620818 shutdown_connections 2024-03-16T09:15:41.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.824+0000 7f6adc3eb640 1 --2- 172.21.15.112:0/2019620818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ad41022b0 0x7f6ad4108b40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.824+0000 7f6adc3eb640 1 --2- 172.21.15.112:0/2019620818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad41018f0 0x7f6ad4101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.824+0000 7f6adc3eb640 1 --2- 172.21.15.112:0/2019620818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ad4100700 0x7f6ad4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.824+0000 7f6adc3eb640 1 -- 172.21.15.112:0/2019620818 >> 172.21.15.112:0/2019620818 conn(0x7f6ad40fbc70 msgr2=0x7f6ad40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:41.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.824+0000 7f6adc3eb640 1 -- 172.21.15.112:0/2019620818 shutdown_connections 2024-03-16T09:15:41.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.824+0000 7f6adc3eb640 1 -- 172.21.15.112:0/2019620818 wait complete. 2024-03-16T09:15:41.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.824+0000 7f6adc3eb640 1 Processor -- start 2024-03-16T09:15:41.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.824+0000 7f6adc3eb640 1 -- start start 2024-03-16T09:15:41.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.824+0000 7f6adc3eb640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ad4100700 0x7f6ad4197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:41.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.824+0000 7f6adc3eb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ad41018f0 0x7f6ad41980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:41.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.824+0000 7f6adc3eb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad41022b0 0x7f6ad419f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:41.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.824+0000 7f6adc3eb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad410d5c0 con 0x7f6ad41022b0 2024-03-16T09:15:41.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.824+0000 7f6adc3eb640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad410d440 con 0x7f6ad41018f0 2024-03-16T09:15:41.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.824+0000 7f6adc3eb640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad410d740 con 0x7f6ad4100700 2024-03-16T09:15:41.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.824+0000 7f6ada988640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad41022b0 0x7f6ad419f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:41.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.824+0000 7f6ada988640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad41022b0 0x7f6ad419f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52178/0 (socket says 172.21.15.112:52178) 2024-03-16T09:15:41.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.824+0000 7f6ada988640 1 -- 172.21.15.112:0/2830838829 learned_addr learned my addr 172.21.15.112:0/2830838829 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:41.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.824+0000 7f6ad9986640 1 --2- 172.21.15.112:0/2830838829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ad41018f0 0x7f6ad41980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:41.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.825+0000 7f6ada187640 1 --2- 172.21.15.112:0/2830838829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ad4100700 0x7f6ad4197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:41.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.825+0000 7f6ada988640 1 -- 172.21.15.112:0/2830838829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ad4100700 msgr2=0x7f6ad4197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:41.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.825+0000 7f6ada988640 1 --2- 172.21.15.112:0/2830838829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ad4100700 0x7f6ad4197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.825+0000 7f6ada988640 1 -- 172.21.15.112:0/2830838829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ad41018f0 msgr2=0x7f6ad41980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:41.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.825+0000 7f6ada988640 1 --2- 172.21.15.112:0/2830838829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ad41018f0 0x7f6ad41980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:41.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.825+0000 7f6ada988640 1 -- 172.21.15.112:0/2830838829 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad419f7c0 con 0x7f6ad41022b0 2024-03-16T09:15:41.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.825+0000 7f6ada988640 1 --2- 172.21.15.112:0/2830838829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad41022b0 0x7f6ad419f150 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6acc02c710 tx=0x7f6acc0346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:41.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.825+0000 7f6ac77fe640 1 -- 172.21.15.112:0/2830838829 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc039040 con 0x7f6ad41022b0 2024-03-16T09:15:41.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.825+0000 7f6ac77fe640 1 -- 172.21.15.112:0/2830838829 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6acc044070 con 0x7f6ad41022b0 2024-03-16T09:15:41.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.825+0000 7f6adc3eb640 1 -- 172.21.15.112:0/2830838829 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad419fa50 con 0x7f6ad41022b0 2024-03-16T09:15:41.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.825+0000 7f6ac77fe640 1 -- 172.21.15.112:0/2830838829 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc004480 con 0x7f6ad41022b0 2024-03-16T09:15:41.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.825+0000 7f6adc3eb640 1 -- 172.21.15.112:0/2830838829 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad419ff40 con 0x7f6ad41022b0 2024-03-16T09:15:41.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.826+0000 7f6adc3eb640 1 -- 172.21.15.112:0/2830838829 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad4107950 con 0x7f6ad41022b0 2024-03-16T09:15:41.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.827+0000 7f6ac77fe640 1 -- 172.21.15.112:0/2830838829 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7f6acc004620 con 0x7f6ad41022b0 2024-03-16T09:15:41.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.827+0000 7f6ac77fe640 1 --2- 172.21.15.112:0/2830838829 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ac00a2890 0x7f6ac00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:41.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.828+0000 7f6ada187640 1 --2- 172.21.15.112:0/2830838829 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ac00a2890 0x7f6ac00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:41.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.828+0000 7f6ac77fe640 1 -- 172.21.15.112:0/2830838829 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f6acc0f1090 con 0x7f6ad41022b0 2024-03-16T09:15:41.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.828+0000 7f6ada187640 1 --2- 172.21.15.112:0/2830838829 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ac00a2890 0x7f6ac00a4d50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4101750 tx=0x7f6ad0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:41.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:41.832+0000 7f6ac77fe640 1 -- 172.21.15.112:0/2830838829 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6acc0be0f0 con 0x7f6ad41022b0 2024-03-16T09:15:42.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.009+0000 7f6adc3eb640 1 -- 172.21.15.112:0/2830838829 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ad40623e0 con 0x7f6ad41022b0 2024-03-16T09:15:42.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.011+0000 7f6ac77fe640 1 -- 172.21.15.112:0/2830838829 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22725 (secure 0 0 0) 0x7f6acc0bd9f0 con 0x7f6ad41022b0 2024-03-16T09:15:42.013 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:15:42.013 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":87,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:14:31.991998+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T09:13:35.040628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T09:13:35.689419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:15:42.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.013+0000 7f6adc3eb640 1 -- 172.21.15.112:0/2830838829 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ac00a2890 msgr2=0x7f6ac00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.013+0000 7f6adc3eb640 1 --2- 172.21.15.112:0/2830838829 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ac00a2890 0x7f6ac00a4d50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4101750 tx=0x7f6ad0005f90 comp rx=0 tx=0).stop 2024-03-16T09:15:42.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.013+0000 7f6adc3eb640 1 -- 172.21.15.112:0/2830838829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad41022b0 msgr2=0x7f6ad419f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.013+0000 7f6adc3eb640 1 --2- 172.21.15.112:0/2830838829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad41022b0 0x7f6ad419f150 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6acc02c710 tx=0x7f6acc0346f0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.013+0000 7f6adc3eb640 1 -- 172.21.15.112:0/2830838829 shutdown_connections 2024-03-16T09:15:42.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.013+0000 7f6adc3eb640 1 --2- 172.21.15.112:0/2830838829 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6ac00a2890 0x7f6ac00a4d50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.013+0000 7f6adc3eb640 1 --2- 172.21.15.112:0/2830838829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ad41022b0 0x7f6ad419f150 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.013+0000 7f6adc3eb640 1 --2- 172.21.15.112:0/2830838829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ad41018f0 0x7f6ad41980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.013+0000 7f6adc3eb640 1 --2- 172.21.15.112:0/2830838829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ad4100700 0x7f6ad4197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.013+0000 7f6adc3eb640 1 -- 172.21.15.112:0/2830838829 >> 172.21.15.112:0/2830838829 conn(0x7f6ad40fbc70 msgr2=0x7f6ad4104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:42.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.013+0000 7f6adc3eb640 1 -- 172.21.15.112:0/2830838829 shutdown_connections 2024-03-16T09:15:42.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.013+0000 7f6adc3eb640 1 -- 172.21.15.112:0/2830838829 wait complete. 2024-03-16T09:15:42.026 DEBUG:tasks.fwd_scrub:Yielding 2024-03-16T09:15:42.027 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-16T09:15:42.037 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-03-16T09:15:42.037 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-16T09:15:42.037 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-16T09:15:42.037 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T09:15:42.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:15:42.038 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T09:15:42.040 INFO:teuthology.run_tasks:Running task workunit... 2024-03-16T09:15:42.065 INFO:tasks.workunit:Pulling workunits from ref 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:15:42.065 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-16T09:15:42.065 INFO:tasks.workunit:timeout=3h 2024-03-16T09:15:42.065 INFO:tasks.workunit:cleanup=True 2024-03-16T09:15:42.066 DEBUG:teuthology.orchestra.run.smithi112:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-16T09:15:42.080 INFO:teuthology.orchestra.run.smithi112.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-16T09:15:42.080 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-16T09:15:42.080 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-03-16T09:15:42.080 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T09:15:42.081 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-16T09:15:42.081 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-03-16 09:13:42.692754116 +0000 2024-03-16T09:15:42.081 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-03-16 09:13:42.692754116 +0000 2024-03-16T09:15:42.081 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-03-16 09:15:34.302435060 +0000 2024-03-16T09:15:42.081 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-03-16 09:13:42.692754116 +0000 2024-03-16T09:15:42.082 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-16T09:15:42.083 DEBUG:teuthology.orchestra.run.smithi112:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-16T09:15:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.143+0000 7fa6d48a1640 1 -- 172.21.15.112:0/1782140669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6d01006e0 msgr2=0x7fa6d0100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.143+0000 7fa6d48a1640 1 --2- 172.21.15.112:0/1782140669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6d01006e0 0x7fa6d0100b00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009900 tx=0x7fa6c002bc30 comp rx=0 tx=0).stop 2024-03-16T09:15:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.143+0000 7fa6d48a1640 1 -- 172.21.15.112:0/1782140669 shutdown_connections 2024-03-16T09:15:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.143+0000 7fa6d48a1640 1 --2- 172.21.15.112:0/1782140669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d0102290 0x7fa6d0108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.143+0000 7fa6d48a1640 1 --2- 172.21.15.112:0/1782140669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6d01018d0 0x7fa6d0101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.143+0000 7fa6d48a1640 1 --2- 172.21.15.112:0/1782140669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6d01006e0 0x7fa6d0100b00 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.143+0000 7fa6d48a1640 1 -- 172.21.15.112:0/1782140669 >> 172.21.15.112:0/1782140669 conn(0x7fa6d00fbc70 msgr2=0x7fa6d00fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:42.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.143+0000 7fa6d48a1640 1 -- 172.21.15.112:0/1782140669 shutdown_connections 2024-03-16T09:15:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.143+0000 7fa6d48a1640 1 -- 172.21.15.112:0/1782140669 wait complete. 2024-03-16T09:15:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.144+0000 7fa6d48a1640 1 Processor -- start 2024-03-16T09:15:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.144+0000 7fa6d48a1640 1 -- start start 2024-03-16T09:15:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.144+0000 7fa6d48a1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6d01006e0 0x7fa6d0197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.144+0000 7fa6d48a1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d01018d0 0x7fa6d01981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.144+0000 7fa6d48a1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6d0102290 0x7fa6d019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.144+0000 7fa6d48a1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d010c440 con 0x7fa6d01006e0 2024-03-16T09:15:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.144+0000 7fa6d48a1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d010c2c0 con 0x7fa6d01018d0 2024-03-16T09:15:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.144+0000 7fa6d48a1640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d010c5c0 con 0x7fa6d0102290 2024-03-16T09:15:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.144+0000 7fa6ce59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6d01006e0 0x7fa6d0197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.145+0000 7fa6c5d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d01018d0 0x7fa6d01981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.145+0000 7fa6c5d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d01018d0 0x7fa6d01981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:58586/0 (socket says 172.21.15.112:58586) 2024-03-16T09:15:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.145+0000 7fa6ced9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6d0102290 0x7fa6d019f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.145+0000 7fa6c5d9b640 1 -- 172.21.15.112:0/1280263546 learned_addr learned my addr 172.21.15.112:0/1280263546 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:42.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.145+0000 7fa6c5d9b640 1 -- 172.21.15.112:0/1280263546 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6d0102290 msgr2=0x7fa6d019f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.145+0000 7fa6c5d9b640 1 --2- 172.21.15.112:0/1280263546 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6d0102290 0x7fa6d019f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.145+0000 7fa6c5d9b640 1 -- 172.21.15.112:0/1280263546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6d01006e0 msgr2=0x7fa6d0197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.145+0000 7fa6c5d9b640 1 --2- 172.21.15.112:0/1280263546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6d01006e0 0x7fa6d0197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.145+0000 7fa6c5d9b640 1 -- 172.21.15.112:0/1280263546 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d019f8d0 con 0x7fa6d01018d0 2024-03-16T09:15:42.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.145+0000 7fa6ce59c640 1 --2- 172.21.15.112:0/1280263546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6d01006e0 0x7fa6d0197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:42.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.145+0000 7fa6c5d9b640 1 --2- 172.21.15.112:0/1280263546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d01018d0 0x7fa6d01981d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800e9f0 tx=0x7fa6b800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:42.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.145+0000 7fa6c7fff640 1 -- 172.21.15.112:0/1280263546 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b800ccc0 con 0x7fa6d01018d0 2024-03-16T09:15:42.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.145+0000 7fa6c7fff640 1 -- 172.21.15.112:0/1280263546 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa6b800ce60 con 0x7fa6d01018d0 2024-03-16T09:15:42.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.145+0000 7fa6d48a1640 1 -- 172.21.15.112:0/1280263546 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d019fbc0 con 0x7fa6d01018d0 2024-03-16T09:15:42.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.145+0000 7fa6c7fff640 1 -- 172.21.15.112:0/1280263546 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8010690 con 0x7fa6d01018d0 2024-03-16T09:15:42.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.145+0000 7fa6d48a1640 1 -- 172.21.15.112:0/1280263546 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d01a00e0 con 0x7fa6d01018d0 2024-03-16T09:15:42.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.148+0000 7fa6c7fff640 1 -- 172.21.15.112:0/1280263546 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7fa6b80046c0 con 0x7fa6d01018d0 2024-03-16T09:15:42.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.148+0000 7fa6d756c740 1 Processor -- start 2024-03-16T09:15:42.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.148+0000 7fa6d756c740 1 -- start start 2024-03-16T09:15:42.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.148+0000 7fa6c7fff640 1 --2- 172.21.15.112:0/1280263546 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa6bc0a2910 0x7fa6bc0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.148+0000 7fa6d756c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa6ceda0 0x5561aa6cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.148+0000 7fa6d756c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6cf790 0x5561aa6d4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.148+0000 7fa6d756c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6d4900 0x5561aa6d6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.148+0000 7fa6d756c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5561aa62f0d0 con 0x5561aa6cf790 2024-03-16T09:15:42.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.148+0000 7fa6d756c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5561aa5d7c70 con 0x5561aa6d4900 2024-03-16T09:15:42.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.148+0000 7fa6d756c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5561aa62b170 con 0x5561aa6ceda0 2024-03-16T09:15:42.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.149+0000 7fa6c5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6cf790 0x5561aa6d4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.149+0000 7fa6ced9d640 1 --2- 172.21.15.112:0/1280263546 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa6bc0a2910 0x7fa6bc0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.149+0000 7fa6c5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6cf790 0x5561aa6d4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52190/0 (socket says 172.21.15.112:52190) 2024-03-16T09:15:42.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.149+0000 7fa6c5d9b640 1 -- 172.21.15.112:0/1171018566 learned_addr learned my addr 172.21.15.112:0/1171018566 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:42.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.149+0000 7fa6ce59c640 1 --2- 172.21.15.112:0/1171018566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa6ceda0 0x5561aa6cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.149+0000 7fa6ced9d640 1 --2- 172.21.15.112:0/1171018566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6d4900 0x5561aa6d6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.149+0000 7fa6c7fff640 1 -- 172.21.15.112:0/1280263546 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa6b80cef80 con 0x7fa6d01018d0 2024-03-16T09:15:42.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.149+0000 7fa6c5d9b640 1 -- 172.21.15.112:0/1171018566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa6ceda0 msgr2=0x5561aa6cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:42.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.149+0000 7fa6c5d9b640 1 --2- 172.21.15.112:0/1171018566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa6ceda0 0x5561aa6cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.149+0000 7fa6c5d9b640 1 -- 172.21.15.112:0/1171018566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6d4900 msgr2=0x5561aa6d6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:42.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.149+0000 7fa6c5d9b640 1 --2- 172.21.15.112:0/1171018566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6d4900 0x5561aa6d6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.149+0000 7fa6c5d9b640 1 -- 172.21.15.112:0/1171018566 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561aa40b7e0 con 0x5561aa6cf790 2024-03-16T09:15:42.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.149+0000 7fa6c5d9b640 1 --2- 172.21.15.112:0/1171018566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6cf790 0x5561aa6d4290 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80cc910 tx=0x7fa6b8004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb4e9027388035f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:42.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.149+0000 7fa6ced9d640 1 --2- 172.21.15.112:0/1280263546 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa6bc0a2910 0x7fa6bc0a4dd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8005fd0 tx=0x7fa6c8009e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:42.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.149+0000 7fa6c4d99640 1 -- 172.21.15.112:0/1171018566 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b80ae530 con 0x5561aa6cf790 2024-03-16T09:15:42.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.149+0000 7fa6c4d99640 1 -- 172.21.15.112:0/1171018566 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa6b809be70 con 0x5561aa6cf790 2024-03-16T09:15:42.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.149+0000 7fa6c4d99640 1 -- 172.21.15.112:0/1171018566 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b80ae170 con 0x5561aa6cf790 2024-03-16T09:15:42.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.150+0000 7fa6d756c740 1 -- 172.21.15.112:0/1171018566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6cf790 msgr2=0x5561aa6d4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.150+0000 7fa6d756c740 1 --2- 172.21.15.112:0/1171018566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6cf790 0x5561aa6d4290 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80cc910 tx=0x7fa6b8004ba0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.150+0000 7fa6d756c740 1 -- 172.21.15.112:0/1171018566 shutdown_connections 2024-03-16T09:15:42.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.150+0000 7fa6d756c740 1 --2- 172.21.15.112:0/1171018566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6d4900 0x5561aa6d6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.150+0000 7fa6d756c740 1 --2- 172.21.15.112:0/1171018566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6cf790 0x5561aa6d4290 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.150+0000 7fa6d756c740 1 --2- 172.21.15.112:0/1171018566 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa6ceda0 0x5561aa6cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.150+0000 7fa6d756c740 1 -- 172.21.15.112:0/1171018566 >> 172.21.15.112:0/1171018566 conn(0x5561aa692c50 msgr2=0x5561aa6ca710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:42.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.150+0000 7fa6d756c740 1 -- 172.21.15.112:0/1171018566 shutdown_connections 2024-03-16T09:15:42.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.150+0000 7fa6d756c740 1 -- 172.21.15.112:0/1171018566 wait complete. 2024-03-16T09:15:42.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.150+0000 7fa6d756c740 1 -- start start 2024-03-16T09:15:42.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6d756c740 1 Processor -- start 2024-03-16T09:15:42.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6d756c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa6ceda0 0x5561aa6d9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6d756c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6cf790 0x5561aa6d97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6d756c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6d4900 0x5561aa6e0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6d756c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5561aa6d78b0 con 0x5561aa6cf790 2024-03-16T09:15:42.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6d756c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5561aa5eb0d0 con 0x5561aa6d4900 2024-03-16T09:15:42.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6d756c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5561aa6d7a30 con 0x5561aa6ceda0 2024-03-16T09:15:42.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6d756c740 10 client.0 _ll_register_callbacks cb 0x5561aa5b4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:15:42.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6c5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6cf790 0x5561aa6d97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6c5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6cf790 0x5561aa6d97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52204/0 (socket says 172.21.15.112:52204) 2024-03-16T09:15:42.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6c5d9b640 1 -- 172.21.15.112:0/746433405 learned_addr learned my addr 172.21.15.112:0/746433405 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:42.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6ce59c640 1 --2- 172.21.15.112:0/746433405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa6ceda0 0x5561aa6d9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6ced9d640 1 --2- 172.21.15.112:0/746433405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6d4900 0x5561aa6e0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6c5d9b640 1 -- 172.21.15.112:0/746433405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa6ceda0 msgr2=0x5561aa6d9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:42.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6c5d9b640 1 --2- 172.21.15.112:0/746433405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa6ceda0 0x5561aa6d9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6c5d9b640 1 -- 172.21.15.112:0/746433405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6d4900 msgr2=0x5561aa6e0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:42.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6c5d9b640 1 --2- 172.21.15.112:0/746433405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6d4900 0x5561aa6e0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6c5d9b640 1 -- 172.21.15.112:0/746433405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561aa46bf70 con 0x5561aa6cf790 2024-03-16T09:15:42.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6ce59c640 1 --2- 172.21.15.112:0/746433405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa6ceda0 0x5561aa6d9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:42.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6c5d9b640 1 --2- 172.21.15.112:0/746433405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6cf790 0x5561aa6d97c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800e9c0 tx=0x7fa6b8098010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8dd042d292e87f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:42.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:15:42.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.151+0000 7fa6b5ffb640 1 -- 172.21.15.112:0/746433405 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b80ae350 con 0x5561aa6cf790 2024-03-16T09:15:42.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.152+0000 7fa6b5ffb640 1 -- 172.21.15.112:0/746433405 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa6b809b9d0 con 0x5561aa6cf790 2024-03-16T09:15:42.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.152+0000 7fa6d756c740 1 -- 172.21.15.112:0/746433405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6b809be70 con 0x5561aa6cf790 2024-03-16T09:15:42.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.152+0000 7fa6b5ffb640 1 -- 172.21.15.112:0/746433405 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa6b80ae170 con 0x5561aa6cf790 2024-03-16T09:15:42.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.152+0000 7fa6d756c740 10 client.15384 fetch_fsmap learned FSMap version 22 2024-03-16T09:15:42.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.152+0000 7fa6d756c740 1 -- 172.21.15.112:0/746433405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5561aa303170 con 0x5561aa6cf790 2024-03-16T09:15:42.165 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:15:42.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.152+0000 7fa6b5ffb640 1 -- 172.21.15.112:0/746433405 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa6b809be70 con 0x5561aa6cf790 2024-03-16T09:15:42.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.152+0000 7fa6b5ffb640 1 -- 172.21.15.112:0/746433405 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa6b80afb50 con 0x5561aa6cf790 2024-03-16T09:15:42.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.152+0000 7fa6d756c740 10 client.15384 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:15:42.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.152+0000 7fa6d756c740 10 client.15384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:15:42.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.152+0000 7fa6d756c740 20 client.15384 populate_metadata read hostname 'smithi112' 2024-03-16T09:15:42.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.152+0000 7fa6d756c740 1 --2- 172.21.15.112:0/746433405 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5561aa6e4960 0x5561aa6e6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.152+0000 7fa6d756c740 4 client.15384 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:15:42.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.152+0000 7fa6d756c740 1 -- 172.21.15.112:0/746433405 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561aa63c610 con 0x5561aa6e4960 2024-03-16T09:15:42.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.152+0000 7fa6ce59c640 1 --2- 172.21.15.112:0/746433405 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5561aa6e4960 0x5561aa6e6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.153+0000 7fa6ce59c640 1 --2- 172.21.15.112:0/746433405 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5561aa6e4960 0x5561aa6e6d50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf78ecc2ba1f0af6 server_cookie=c52d9053d009cd61 in_seq=0 out_seq=0 2024-03-16T09:15:42.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.153+0000 7fa6b5ffb640 10 client.15384 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:42.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.153+0000 7fa6b5ffb640 1 -- 172.21.15.112:0/746433405 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5561aa63c610 con 0x5561aa6e4960 2024-03-16T09:15:42.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.153+0000 7fa6b5ffb640 10 client.15384 handle_command_reply: tid=0 2024-03-16T09:15:42.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.153+0000 7fa6d756c740 1 client.15384 shutdown 2024-03-16T09:15:42.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.153+0000 7fa6d756c740 1 -- 172.21.15.112:0/746433405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6cf790 msgr2=0x5561aa6d97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.153+0000 7fa6d756c740 1 --2- 172.21.15.112:0/746433405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6cf790 0x5561aa6d97c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800e9c0 tx=0x7fa6b8098010 comp rx=0 tx=0).stop 2024-03-16T09:15:42.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.153+0000 7fa6d756c740 1 -- 172.21.15.112:0/746433405 shutdown_connections 2024-03-16T09:15:42.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.153+0000 7fa6d756c740 1 --2- 172.21.15.112:0/746433405 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5561aa6e4960 0x5561aa6e6d50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.153+0000 7fa6d756c740 1 --2- 172.21.15.112:0/746433405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6d4900 0x5561aa6e0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.153+0000 7fa6d756c740 1 --2- 172.21.15.112:0/746433405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6cf790 0x5561aa6d97c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.153+0000 7fa6d756c740 1 --2- 172.21.15.112:0/746433405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa6ceda0 0x5561aa6d9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.153+0000 7fa6b5ffb640 0 client.15384 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:42.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.153+0000 7fa6d756c740 1 -- 172.21.15.112:0/746433405 >> 172.21.15.112:0/746433405 conn(0x5561aa692c50 msgr2=0x5561aa6c54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:42.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.153+0000 7fa6d756c740 1 -- 172.21.15.112:0/746433405 shutdown_connections 2024-03-16T09:15:42.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.153+0000 7fa6d756c740 1 -- 172.21.15.112:0/746433405 wait complete. 2024-03-16T09:15:42.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.154+0000 7fa6d756c740 20 client.15384 trim_cache size 0 max 16384 2024-03-16T09:15:42.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.171+0000 7fa6d756c740 1 Processor -- start 2024-03-16T09:15:42.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.171+0000 7fa6d756c740 1 -- start start 2024-03-16T09:15:42.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.171+0000 7fa6d756c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6d0570 0x5561aa6ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.171+0000 7fa6d756c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa692c50 0x5561aa6e07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.171+0000 7fa6d756c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6ceda0 0x5561aa6eb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.171+0000 7fa6d756c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d01a9400 con 0x5561aa6d0570 2024-03-16T09:15:42.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.171+0000 7fa6d756c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d01a6cd0 con 0x5561aa6ceda0 2024-03-16T09:15:42.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.171+0000 7fa6d756c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d01a4b20 con 0x5561aa692c50 2024-03-16T09:15:42.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.171+0000 7fa6ce59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6d0570 0x5561aa6ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.171+0000 7fa6ce59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6d0570 0x5561aa6ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52220/0 (socket says 172.21.15.112:52220) 2024-03-16T09:15:42.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.171+0000 7fa6ce59c640 1 -- 172.21.15.112:0/1927163504 learned_addr learned my addr 172.21.15.112:0/1927163504 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:42.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.171+0000 7fa6c5d9b640 1 --2- 172.21.15.112:0/1927163504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa692c50 0x5561aa6e07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.171+0000 7fa6ced9d640 1 --2- 172.21.15.112:0/1927163504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6ceda0 0x5561aa6eb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.171+0000 7fa6ce59c640 1 -- 172.21.15.112:0/1927163504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa692c50 msgr2=0x5561aa6e07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:42.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.171+0000 7fa6ce59c640 1 --2- 172.21.15.112:0/1927163504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa692c50 0x5561aa6e07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.171+0000 7fa6ce59c640 1 -- 172.21.15.112:0/1927163504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6ceda0 msgr2=0x5561aa6eb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:42.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.171+0000 7fa6ce59c640 1 --2- 172.21.15.112:0/1927163504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6ceda0 0x5561aa6eb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.171+0000 7fa6ce59c640 1 -- 172.21.15.112:0/1927163504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561aa6c08f0 con 0x5561aa6d0570 2024-03-16T09:15:42.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.171+0000 7fa6ced9d640 1 --2- 172.21.15.112:0/1927163504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6ceda0 0x5561aa6eb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:15:42.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.172+0000 7fa6ce59c640 1 --2- 172.21.15.112:0/1927163504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6d0570 0x5561aa6ce060 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009900 tx=0x7fa6c002c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab12633bdc1b201 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:42.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.172+0000 7fa6b7fff640 1 -- 172.21.15.112:0/1927163504 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0004490 con 0x5561aa6d0570 2024-03-16T09:15:42.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.172+0000 7fa6b7fff640 1 -- 172.21.15.112:0/1927163504 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa6c0005540 con 0x5561aa6d0570 2024-03-16T09:15:42.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.172+0000 7fa6b7fff640 1 -- 172.21.15.112:0/1927163504 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0003850 con 0x5561aa6d0570 2024-03-16T09:15:42.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.172+0000 7fa6d756c740 1 -- 172.21.15.112:0/1927163504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6d0570 msgr2=0x5561aa6ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.172+0000 7fa6d756c740 1 --2- 172.21.15.112:0/1927163504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6d0570 0x5561aa6ce060 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009900 tx=0x7fa6c002c3e0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.172+0000 7fa6d756c740 1 -- 172.21.15.112:0/1927163504 shutdown_connections 2024-03-16T09:15:42.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.172+0000 7fa6d756c740 1 --2- 172.21.15.112:0/1927163504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6ceda0 0x5561aa6eb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.172+0000 7fa6d756c740 1 --2- 172.21.15.112:0/1927163504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa692c50 0x5561aa6e07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.172+0000 7fa6d756c740 1 --2- 172.21.15.112:0/1927163504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa6d0570 0x5561aa6ce060 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.172+0000 7fa6d756c740 1 -- 172.21.15.112:0/1927163504 >> 172.21.15.112:0/1927163504 conn(0x5561aa6d8e40 msgr2=0x5561aa6cd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:42.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.172+0000 7fa6d756c740 1 -- 172.21.15.112:0/1927163504 shutdown_connections 2024-03-16T09:15:42.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.172+0000 7fa6d756c740 1 -- 172.21.15.112:0/1927163504 wait complete. 2024-03-16T09:15:42.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6d756c740 1 -- start start 2024-03-16T09:15:42.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6d756c740 1 Processor -- start 2024-03-16T09:15:42.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6d756c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa692c50 0x5561aa6e0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6d756c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6ceda0 0x5561aa6de9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6d756c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa6d0570 0x5561aa6def10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6d756c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d01a5190 con 0x5561aa692c50 2024-03-16T09:15:42.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6d756c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d01a6600 con 0x5561aa6ceda0 2024-03-16T09:15:42.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6d756c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d01a5860 con 0x5561aa6d0570 2024-03-16T09:15:42.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6d756c740 10 client.0 _ll_register_callbacks cb 0x7fa6c80137c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:15:42.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6ce59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa692c50 0x5561aa6e0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6ce59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa692c50 0x5561aa6e0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52222/0 (socket says 172.21.15.112:52222) 2024-03-16T09:15:42.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6ce59c640 1 -- 172.21.15.112:0/157178518 learned_addr learned my addr 172.21.15.112:0/157178518 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:42.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6c5d9b640 1 --2- 172.21.15.112:0/157178518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6ceda0 0x5561aa6de9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6ced9d640 1 --2- 172.21.15.112:0/157178518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa6d0570 0x5561aa6def10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6ce59c640 1 -- 172.21.15.112:0/157178518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa6d0570 msgr2=0x5561aa6def10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:42.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6ce59c640 1 --2- 172.21.15.112:0/157178518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa6d0570 0x5561aa6def10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6ce59c640 1 -- 172.21.15.112:0/157178518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6ceda0 msgr2=0x5561aa6de9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:42.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6ce59c640 1 --2- 172.21.15.112:0/157178518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6ceda0 0x5561aa6de9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6ce59c640 1 -- 172.21.15.112:0/157178518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561aa6df450 con 0x5561aa692c50 2024-03-16T09:15:42.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.173+0000 7fa6ced9d640 1 --2- 172.21.15.112:0/157178518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa6d0570 0x5561aa6def10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:15:42.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.174+0000 7fa6ce59c640 1 --2- 172.21.15.112:0/157178518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa692c50 0x5561aa6e0350 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0006010 tx=0x7fa6c0004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1e306a27836f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:42.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.174+0000 7fa6b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:15:42.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.174+0000 7fa6b5ffb640 1 -- 172.21.15.112:0/157178518 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0057070 con 0x5561aa692c50 2024-03-16T09:15:42.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.174+0000 7fa6b5ffb640 1 -- 172.21.15.112:0/157178518 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa6c00052a0 con 0x5561aa692c50 2024-03-16T09:15:42.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.174+0000 7fa6d756c740 1 -- 172.21.15.112:0/157178518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6c0005540 con 0x5561aa692c50 2024-03-16T09:15:42.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.174+0000 7fa6b5ffb640 1 -- 172.21.15.112:0/157178518 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa6c0063050 con 0x5561aa692c50 2024-03-16T09:15:42.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.174+0000 7fa6d756c740 10 client.15390 fetch_fsmap learned FSMap version 22 2024-03-16T09:15:42.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.174+0000 7fa6d756c740 1 -- 172.21.15.112:0/157178518 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5561aa74b740 con 0x5561aa692c50 2024-03-16T09:15:42.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.174+0000 7fa6b5ffb640 1 -- 172.21.15.112:0/157178518 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa6c0005540 con 0x5561aa692c50 2024-03-16T09:15:42.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.174+0000 7fa6b5ffb640 1 -- 172.21.15.112:0/157178518 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa6c00344e0 con 0x5561aa692c50 2024-03-16T09:15:42.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.174+0000 7fa6d756c740 10 client.15390 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:15:42.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.174+0000 7fa6d756c740 10 client.15390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:15:42.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.174+0000 7fa6d756c740 20 client.15390 populate_metadata read hostname 'smithi112' 2024-03-16T09:15:42.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.174+0000 7fa6d756c740 1 --2- 172.21.15.112:0/157178518 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5561aa6cf790 0x5561aa74e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.174+0000 7fa6d756c740 4 client.15390 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T09:15:42.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.174+0000 7fa6d756c740 1 -- 172.21.15.112:0/157178518 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5561aa636960 con 0x5561aa6cf790 2024-03-16T09:15:42.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.174+0000 7fa6c5d9b640 1 --2- 172.21.15.112:0/157178518 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5561aa6cf790 0x5561aa74e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.175+0000 7fa6c5d9b640 1 --2- 172.21.15.112:0/157178518 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5561aa6cf790 0x5561aa74e0b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=487672970ee1b593 server_cookie=ff4fdb39aa47e4ca in_seq=0 out_seq=0 2024-03-16T09:15:42.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.175+0000 7fa6b5ffb640 10 client.15390 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:42.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.176+0000 7fa6b5ffb640 1 -- 172.21.15.112:0/157178518 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5561aa636960 con 0x5561aa6cf790 2024-03-16T09:15:42.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.176+0000 7fa6b5ffb640 10 client.15390 handle_command_reply: tid=0 2024-03-16T09:15:42.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.176+0000 7fa6d756c740 1 client.15390 shutdown 2024-03-16T09:15:42.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.176+0000 7fa6d756c740 1 -- 172.21.15.112:0/157178518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa692c50 msgr2=0x5561aa6e0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.176+0000 7fa6d756c740 1 --2- 172.21.15.112:0/157178518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa692c50 0x5561aa6e0350 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0006010 tx=0x7fa6c0004510 comp rx=0 tx=0).stop 2024-03-16T09:15:42.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.176+0000 7fa6d756c740 1 -- 172.21.15.112:0/157178518 shutdown_connections 2024-03-16T09:15:42.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.176+0000 7fa6d756c740 1 --2- 172.21.15.112:0/157178518 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5561aa6d0570 0x5561aa6def10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.176+0000 7fa6d756c740 1 --2- 172.21.15.112:0/157178518 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5561aa6cf790 0x5561aa74e0b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.176+0000 7fa6d756c740 1 --2- 172.21.15.112:0/157178518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5561aa6ceda0 0x5561aa6de9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.176+0000 7fa6d756c740 1 --2- 172.21.15.112:0/157178518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561aa692c50 0x5561aa6e0350 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.176+0000 7fa6d756c740 1 -- 172.21.15.112:0/157178518 >> 172.21.15.112:0/157178518 conn(0x5561aa6d8e40 msgr2=0x5561aa6ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:42.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.176+0000 7fa6b5ffb640 0 client.15390 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:42.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.176+0000 7fa6d756c740 1 -- 172.21.15.112:0/157178518 shutdown_connections 2024-03-16T09:15:42.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.176+0000 7fa6d756c740 1 -- 172.21.15.112:0/157178518 wait complete. 2024-03-16T09:15:42.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.176+0000 7fa6d756c740 20 client.15390 trim_cache size 0 max 16384 2024-03-16T09:15:42.186 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:15:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-03-16T09:15:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "022a6cbe-bceb-4836-9fbd-830d467bedcc", 2024-03-16T09:15:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-03-16T09:15:42.186 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:15:42.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.177+0000 7fa6d48a1640 1 -- 172.21.15.112:0/1280263546 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa6bc0a2910 msgr2=0x7fa6bc0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.177+0000 7fa6d48a1640 1 --2- 172.21.15.112:0/1280263546 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa6bc0a2910 0x7fa6bc0a4dd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8005fd0 tx=0x7fa6c8009e80 comp rx=0 tx=0).stop 2024-03-16T09:15:42.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.177+0000 7fa6d48a1640 1 -- 172.21.15.112:0/1280263546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d01018d0 msgr2=0x7fa6d01981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.177+0000 7fa6d48a1640 1 --2- 172.21.15.112:0/1280263546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d01018d0 0x7fa6d01981d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800e9f0 tx=0x7fa6b800edf0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.177+0000 7fa6d48a1640 1 -- 172.21.15.112:0/1280263546 shutdown_connections 2024-03-16T09:15:42.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.177+0000 7fa6d48a1640 1 --2- 172.21.15.112:0/1280263546 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa6bc0a2910 0x7fa6bc0a4dd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.177+0000 7fa6d48a1640 1 --2- 172.21.15.112:0/1280263546 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6d0102290 0x7fa6d019f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.177+0000 7fa6d48a1640 1 --2- 172.21.15.112:0/1280263546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d01018d0 0x7fa6d01981d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.177+0000 7fa6d48a1640 1 --2- 172.21.15.112:0/1280263546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6d01006e0 0x7fa6d0197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.177+0000 7fa6d48a1640 1 -- 172.21.15.112:0/1280263546 >> 172.21.15.112:0/1280263546 conn(0x7fa6d00fbc70 msgr2=0x7fa6d00689f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:42.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.177+0000 7fa6d48a1640 1 -- 172.21.15.112:0/1280263546 shutdown_connections 2024-03-16T09:15:42.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.177+0000 7fa6d48a1640 1 -- 172.21.15.112:0/1280263546 wait complete. 2024-03-16T09:15:42.188 INFO:tasks.workunit.client.0.smithi112.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-16T09:15:42.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T09:15:42.190 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T09:15:42.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:41 smithi159 ceph-mon[36739]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:15:42.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:41 smithi159 ceph-mon[36739]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-16T09:15:42.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:41 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/277762392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:42.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:41 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/4123697086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:42.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:41 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1406019233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:42.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:41 smithi112 ceph-mon[32307]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:15:42.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:41 smithi112 ceph-mon[32307]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-16T09:15:42.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:41 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/277762392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:42.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:41 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/4123697086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:42.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:41 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1406019233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:42.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.340+0000 7fdeb851d640 1 -- 172.21.15.112:0/3288676631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdeb01018d0 msgr2=0x7fdeb0101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.340+0000 7fdeb851d640 1 --2- 172.21.15.112:0/3288676631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdeb01018d0 0x7fdeb0101d50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009860 tx=0x7fdea002bb70 comp rx=0 tx=0).stop 2024-03-16T09:15:42.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.340+0000 7fdeb851d640 1 -- 172.21.15.112:0/3288676631 shutdown_connections 2024-03-16T09:15:42.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.340+0000 7fdeb851d640 1 --2- 172.21.15.112:0/3288676631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdeb0102290 0x7fdeb0108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.340+0000 7fdeb851d640 1 --2- 172.21.15.112:0/3288676631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdeb01018d0 0x7fdeb0101d50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.340+0000 7fdeb851d640 1 --2- 172.21.15.112:0/3288676631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdeb01006e0 0x7fdeb0100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.340+0000 7fdeb851d640 1 -- 172.21.15.112:0/3288676631 >> 172.21.15.112:0/3288676631 conn(0x7fdeb00fbc70 msgr2=0x7fdeb00fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:42.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.340+0000 7fdeb851d640 1 -- 172.21.15.112:0/3288676631 shutdown_connections 2024-03-16T09:15:42.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.340+0000 7fdeb851d640 1 -- 172.21.15.112:0/3288676631 wait complete. 2024-03-16T09:15:42.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb851d640 1 Processor -- start 2024-03-16T09:15:42.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb851d640 1 -- start start 2024-03-16T09:15:42.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb851d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdeb01006e0 0x7fdeb0197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb851d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdeb01018d0 0x7fdeb01981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb851d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdeb0102290 0x7fdeb019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb851d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb010c490 con 0x7fdeb01018d0 2024-03-16T09:15:42.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb851d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb010c310 con 0x7fdeb01006e0 2024-03-16T09:15:42.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb851d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb010c610 con 0x7fdeb0102290 2024-03-16T09:15:42.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb5ab8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdeb01018d0 0x7fdeb01981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb5ab8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdeb01018d0 0x7fdeb01981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52236/0 (socket says 172.21.15.112:52236) 2024-03-16T09:15:42.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb5ab8640 1 -- 172.21.15.112:0/2732342762 learned_addr learned my addr 172.21.15.112:0/2732342762 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:42.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb62b9640 1 --2- 172.21.15.112:0/2732342762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdeb01006e0 0x7fdeb0197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb6aba640 1 --2- 172.21.15.112:0/2732342762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdeb0102290 0x7fdeb019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb5ab8640 1 -- 172.21.15.112:0/2732342762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdeb0102290 msgr2=0x7fdeb019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb5ab8640 1 --2- 172.21.15.112:0/2732342762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdeb0102290 0x7fdeb019f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb5ab8640 1 -- 172.21.15.112:0/2732342762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdeb01006e0 msgr2=0x7fdeb0197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb5ab8640 1 --2- 172.21.15.112:0/2732342762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdeb01006e0 0x7fdeb0197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb5ab8640 1 -- 172.21.15.112:0/2732342762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb019f830 con 0x7fdeb01018d0 2024-03-16T09:15:42.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb62b9640 1 --2- 172.21.15.112:0/2732342762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdeb01006e0 0x7fdeb0197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:42.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb6aba640 1 --2- 172.21.15.112:0/2732342762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdeb0102290 0x7fdeb019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:42.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.341+0000 7fdeb5ab8640 1 --2- 172.21.15.112:0/2732342762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdeb01018d0 0x7fdeb01981b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdea0034bb0 tx=0x7fdea00349a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:42.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.342+0000 7fdea77fe640 1 -- 172.21.15.112:0/2732342762 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0041020 con 0x7fdeb01018d0 2024-03-16T09:15:42.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.342+0000 7fdeb851d640 1 -- 172.21.15.112:0/2732342762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb019fac0 con 0x7fdeb01018d0 2024-03-16T09:15:42.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.342+0000 7fdea77fe640 1 -- 172.21.15.112:0/2732342762 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdea002ca10 con 0x7fdeb01018d0 2024-03-16T09:15:42.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.342+0000 7fdea77fe640 1 -- 172.21.15.112:0/2732342762 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0039040 con 0x7fdeb01018d0 2024-03-16T09:15:42.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.342+0000 7fdeb851d640 1 -- 172.21.15.112:0/2732342762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb01a0000 con 0x7fdeb01018d0 2024-03-16T09:15:42.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.344+0000 7fdea77fe640 1 -- 172.21.15.112:0/2732342762 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7fdea0044020 con 0x7fdeb01018d0 2024-03-16T09:15:42.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.344+0000 7fdebb1df740 1 Processor -- start 2024-03-16T09:15:42.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.344+0000 7fdebb1df740 1 -- start start 2024-03-16T09:15:42.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdebb1df740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e2170 0x55accd2e2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdebb1df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e2b60 0x55accd2e7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdebb1df740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e7cd0 0x55accd2ea160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdebb1df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55accd247500 con 0x55accd2e2b60 2024-03-16T09:15:42.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdebb1df740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55accd1efbe0 con 0x55accd2e2170 2024-03-16T09:15:42.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdebb1df740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55accd2435a0 con 0x55accd2e7cd0 2024-03-16T09:15:42.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdea77fe640 1 --2- 172.21.15.112:0/2732342762 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fde9c0a2890 0x7fde9c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdeb62b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e2b60 0x55accd2e7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdeb62b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e2b60 0x55accd2e7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52244/0 (socket says 172.21.15.112:52244) 2024-03-16T09:15:42.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdeb62b9640 1 -- 172.21.15.112:0/2321772581 learned_addr learned my addr 172.21.15.112:0/2321772581 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:42.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdeb5ab8640 1 --2- 172.21.15.112:0/2321772581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e7cd0 0x55accd2ea160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdeb6aba640 1 --2- 172.21.15.112:0/2321772581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e2170 0x55accd2e2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdeb6aba640 1 --2- 172.21.15.112:0/2732342762 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fde9c0a2890 0x7fde9c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdea77fe640 1 -- 172.21.15.112:0/2732342762 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdea00f04b0 con 0x7fdeb01018d0 2024-03-16T09:15:42.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdeb62b9640 1 -- 172.21.15.112:0/2321772581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e7cd0 msgr2=0x55accd2ea160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:42.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdeb62b9640 1 --2- 172.21.15.112:0/2321772581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e7cd0 0x55accd2ea160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdeb62b9640 1 -- 172.21.15.112:0/2321772581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e2170 msgr2=0x55accd2e2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:42.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdeb62b9640 1 --2- 172.21.15.112:0/2321772581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e2170 0x55accd2e2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdeb62b9640 1 -- 172.21.15.112:0/2321772581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55accd2034b0 con 0x55accd2e2b60 2024-03-16T09:15:42.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdeb62b9640 1 --2- 172.21.15.112:0/2321772581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e2b60 0x55accd2e7660 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0114f0 tx=0x7fdeac0118f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b7510fd5795906 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:42.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdeb6aba640 1 --2- 172.21.15.112:0/2732342762 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fde9c0a2890 0x7fde9c0a4d50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009fd0 tx=0x7fdea80095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:42.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.345+0000 7fdea4bfe640 1 -- 172.21.15.112:0/2321772581 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac004310 con 0x55accd2e2b60 2024-03-16T09:15:42.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.346+0000 7fdea4bfe640 1 -- 172.21.15.112:0/2321772581 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdeac0044b0 con 0x55accd2e2b60 2024-03-16T09:15:42.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.346+0000 7fdea4bfe640 1 -- 172.21.15.112:0/2321772581 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac003850 con 0x55accd2e2b60 2024-03-16T09:15:42.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.346+0000 7fdebb1df740 1 -- 172.21.15.112:0/2321772581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e2b60 msgr2=0x55accd2e7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.346+0000 7fdebb1df740 1 --2- 172.21.15.112:0/2321772581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e2b60 0x55accd2e7660 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0114f0 tx=0x7fdeac0118f0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.346+0000 7fdebb1df740 1 -- 172.21.15.112:0/2321772581 shutdown_connections 2024-03-16T09:15:42.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.346+0000 7fdebb1df740 1 --2- 172.21.15.112:0/2321772581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e7cd0 0x55accd2ea160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.346+0000 7fdebb1df740 1 --2- 172.21.15.112:0/2321772581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e2b60 0x55accd2e7660 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.346+0000 7fdebb1df740 1 --2- 172.21.15.112:0/2321772581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e2170 0x55accd2e2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.346+0000 7fdebb1df740 1 -- 172.21.15.112:0/2321772581 >> 172.21.15.112:0/2321772581 conn(0x55accd2dd6e0 msgr2=0x55accd2ddae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:42.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.346+0000 7fdebb1df740 1 -- 172.21.15.112:0/2321772581 shutdown_connections 2024-03-16T09:15:42.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.346+0000 7fdebb1df740 1 -- 172.21.15.112:0/2321772581 wait complete. 2024-03-16T09:15:42.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdebb1df740 1 -- start start 2024-03-16T09:15:42.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdebb1df740 1 Processor -- start 2024-03-16T09:15:42.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdebb1df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e2170 0x55accd2ec660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdebb1df740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e2b60 0x55accd2ecba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdebb1df740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e7cd0 0x55accd2f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdebb1df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55accd2eac80 con 0x55accd2e2170 2024-03-16T09:15:42.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdebb1df740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55accd203160 con 0x55accd2e7cd0 2024-03-16T09:15:42.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdebb1df740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55accd2eae00 con 0x55accd2e2b60 2024-03-16T09:15:42.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdebb1df740 10 client.0 _ll_register_callbacks cb 0x55accd1e7150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:15:42.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdeb62b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e2170 0x55accd2ec660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdeb5ab8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e2b60 0x55accd2ecba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdeb62b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e2170 0x55accd2ec660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52258/0 (socket says 172.21.15.112:52258) 2024-03-16T09:15:42.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdeb62b9640 1 -- 172.21.15.112:0/602133336 learned_addr learned my addr 172.21.15.112:0/602133336 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:42.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdeb6aba640 1 --2- 172.21.15.112:0/602133336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e7cd0 0x55accd2f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdeb5ab8640 1 -- 172.21.15.112:0/602133336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e7cd0 msgr2=0x55accd2f3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:42.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdeb5ab8640 1 --2- 172.21.15.112:0/602133336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e7cd0 0x55accd2f3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdeb5ab8640 1 -- 172.21.15.112:0/602133336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e2170 msgr2=0x55accd2ec660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:42.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdeb5ab8640 1 --2- 172.21.15.112:0/602133336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e2170 0x55accd2ec660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdeb5ab8640 1 -- 172.21.15.112:0/602133336 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55accd0257d0 con 0x55accd2e2b60 2024-03-16T09:15:42.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fdeb5ab8640 1 --2- 172.21.15.112:0/602133336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e2b60 0x55accd2ecba0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdea00edf80 tx=0x7fdea008bbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c8a17c2aa156f5c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:42.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.347+0000 7fde99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T09:15:42.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.348+0000 7fde99ffb640 1 -- 172.21.15.112:0/602133336 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea00bc770 con 0x55accd2e2b60 2024-03-16T09:15:42.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.348+0000 7fde99ffb640 1 -- 172.21.15.112:0/602133336 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdea00f00b0 con 0x55accd2e2b60 2024-03-16T09:15:42.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.348+0000 7fdebb1df740 1 -- 172.21.15.112:0/602133336 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeac0044b0 con 0x55accd2e2b60 2024-03-16T09:15:42.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.348+0000 7fde99ffb640 1 -- 172.21.15.112:0/602133336 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fdea00baf50 con 0x55accd2e2b60 2024-03-16T09:15:42.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.348+0000 7fdebb1df740 10 client.25267 fetch_fsmap learned FSMap version 22 2024-03-16T09:15:42.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.348+0000 7fdebb1df740 1 -- 172.21.15.112:0/602133336 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55acccf5b3f0 con 0x55accd2e2b60 2024-03-16T09:15:42.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.348+0000 7fde99ffb640 1 -- 172.21.15.112:0/602133336 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdeac0044b0 con 0x55accd2e2b60 2024-03-16T09:15:42.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.348+0000 7fde99ffb640 1 -- 172.21.15.112:0/602133336 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fdea003c030 con 0x55accd2e2b60 2024-03-16T09:15:42.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.348+0000 7fdebb1df740 10 client.25267 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:15:42.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.348+0000 7fdebb1df740 10 client.25267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:15:42.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.348+0000 7fdebb1df740 20 client.25267 populate_metadata read hostname 'smithi112' 2024-03-16T09:15:42.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.348+0000 7fdebb1df740 1 --2- 172.21.15.112:0/602133336 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55accd2f81b0 0x55accd2fa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.348+0000 7fdebb1df740 4 client.25267 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:15:42.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.348+0000 7fdebb1df740 1 -- 172.21.15.112:0/602133336 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55accd254a40 con 0x55accd2f81b0 2024-03-16T09:15:42.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.348+0000 7fdeb62b9640 1 --2- 172.21.15.112:0/602133336 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55accd2f81b0 0x55accd2fa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.349+0000 7fdeb62b9640 1 --2- 172.21.15.112:0/602133336 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55accd2f81b0 0x55accd2fa650 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d80e8285e1024b6 server_cookie=5d4e44318586711d in_seq=0 out_seq=0 2024-03-16T09:15:42.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.349+0000 7fde99ffb640 10 client.25267 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:42.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.350+0000 7fde99ffb640 1 -- 172.21.15.112:0/602133336 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55accd254a40 con 0x55accd2f81b0 2024-03-16T09:15:42.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.350+0000 7fde99ffb640 10 client.25267 handle_command_reply: tid=0 2024-03-16T09:15:42.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.350+0000 7fdebb1df740 1 client.25267 shutdown 2024-03-16T09:15:42.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.350+0000 7fdebb1df740 1 -- 172.21.15.112:0/602133336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e2b60 msgr2=0x55accd2ecba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.350+0000 7fdebb1df740 1 --2- 172.21.15.112:0/602133336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e2b60 0x55accd2ecba0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdea00edf80 tx=0x7fdea008bbe0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.350+0000 7fdebb1df740 1 -- 172.21.15.112:0/602133336 shutdown_connections 2024-03-16T09:15:42.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.350+0000 7fdebb1df740 1 --2- 172.21.15.112:0/602133336 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55accd2f81b0 0x55accd2fa650 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.350+0000 7fdebb1df740 1 --2- 172.21.15.112:0/602133336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e7cd0 0x55accd2f3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.350+0000 7fdebb1df740 1 --2- 172.21.15.112:0/602133336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e2b60 0x55accd2ecba0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.350+0000 7fde99ffb640 0 client.25267 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:42.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.350+0000 7fdebb1df740 1 --2- 172.21.15.112:0/602133336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e2170 0x55accd2ec660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.350+0000 7fdebb1df740 1 -- 172.21.15.112:0/602133336 >> 172.21.15.112:0/602133336 conn(0x55accd2dd6e0 msgr2=0x55accd2dc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:42.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.350+0000 7fdebb1df740 1 -- 172.21.15.112:0/602133336 shutdown_connections 2024-03-16T09:15:42.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.350+0000 7fdebb1df740 1 -- 172.21.15.112:0/602133336 wait complete. 2024-03-16T09:15:42.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.350+0000 7fdebb1df740 20 client.25267 trim_cache size 0 max 16384 2024-03-16T09:15:42.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.367+0000 7fdebb1df740 1 Processor -- start 2024-03-16T09:15:42.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.367+0000 7fdebb1df740 1 -- start start 2024-03-16T09:15:42.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.367+0000 7fdebb1df740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e2170 0x55accd2ec660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.367+0000 7fdebb1df740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e2b60 0x55accd2f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.367+0000 7fdebb1df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e7cd0 0x55accd2ecba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.367+0000 7fdebb1df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01ab520 con 0x55accd2e7cd0 2024-03-16T09:15:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.367+0000 7fdebb1df740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01a8df0 con 0x55accd2e2b60 2024-03-16T09:15:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.367+0000 7fdebb1df740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01a6c40 con 0x55accd2e2170 2024-03-16T09:15:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.368+0000 7fdeb6aba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e7cd0 0x55accd2ecba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.368+0000 7fdeb62b9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e2170 0x55accd2ec660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.368+0000 7fdeb6aba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e7cd0 0x55accd2ecba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52274/0 (socket says 172.21.15.112:52274) 2024-03-16T09:15:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.368+0000 7fdeb62b9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e2170 0x55accd2ec660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:59016/0 (socket says 172.21.15.112:59016) 2024-03-16T09:15:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.368+0000 7fdeb6aba640 1 -- 172.21.15.112:0/3858162642 learned_addr learned my addr 172.21.15.112:0/3858162642 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.368+0000 7fdeb5ab8640 1 --2- 172.21.15.112:0/3858162642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e2b60 0x55accd2f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.368+0000 7fdeb6aba640 1 -- 172.21.15.112:0/3858162642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e2170 msgr2=0x55accd2ec660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.368+0000 7fdeb6aba640 1 --2- 172.21.15.112:0/3858162642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e2170 0x55accd2ec660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.368+0000 7fdeb6aba640 1 -- 172.21.15.112:0/3858162642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e2b60 msgr2=0x55accd2f3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.368+0000 7fdeb6aba640 1 --2- 172.21.15.112:0/3858162642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e2b60 0x55accd2f3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.368+0000 7fdeb6aba640 1 -- 172.21.15.112:0/3858162642 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acccf1b0f0 con 0x55accd2e7cd0 2024-03-16T09:15:42.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.368+0000 7fdeb6aba640 1 --2- 172.21.15.112:0/3858162642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e7cd0 0x55accd2ecba0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdea800ee00 tx=0x7fdea8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eef3160f76aa313 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:42.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.368+0000 7fde9bfff640 1 -- 172.21.15.112:0/3858162642 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea802e070 con 0x55accd2e7cd0 2024-03-16T09:15:42.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.368+0000 7fde9bfff640 1 -- 172.21.15.112:0/3858162642 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdea8003e00 con 0x55accd2e7cd0 2024-03-16T09:15:42.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.368+0000 7fde9bfff640 1 -- 172.21.15.112:0/3858162642 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea801d830 con 0x55accd2e7cd0 2024-03-16T09:15:42.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.369+0000 7fdebb1df740 1 -- 172.21.15.112:0/3858162642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e7cd0 msgr2=0x55accd2ecba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.369+0000 7fdebb1df740 1 --2- 172.21.15.112:0/3858162642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e7cd0 0x55accd2ecba0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdea800ee00 tx=0x7fdea8018000 comp rx=0 tx=0).stop 2024-03-16T09:15:42.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.369+0000 7fdebb1df740 1 -- 172.21.15.112:0/3858162642 shutdown_connections 2024-03-16T09:15:42.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.369+0000 7fdebb1df740 1 --2- 172.21.15.112:0/3858162642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e7cd0 0x55accd2ecba0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.369+0000 7fdebb1df740 1 --2- 172.21.15.112:0/3858162642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e2b60 0x55accd2f3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.369+0000 7fdebb1df740 1 --2- 172.21.15.112:0/3858162642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e2170 0x55accd2ec660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.369+0000 7fdebb1df740 1 -- 172.21.15.112:0/3858162642 >> 172.21.15.112:0/3858162642 conn(0x55accd2dd6e0 msgr2=0x55accd2e4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.369+0000 7fdebb1df740 1 -- 172.21.15.112:0/3858162642 shutdown_connections 2024-03-16T09:15:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.369+0000 7fdebb1df740 1 -- 172.21.15.112:0/3858162642 wait complete. 2024-03-16T09:15:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.369+0000 7fdebb1df740 1 -- start start 2024-03-16T09:15:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.369+0000 7fdebb1df740 1 Processor -- start 2024-03-16T09:15:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.369+0000 7fdebb1df740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e2170 0x55accd2f1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.369+0000 7fdebb1df740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e2b60 0x55accd2f15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.369+0000 7fdebb1df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e7cd0 0x55accd2f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.369+0000 7fdebb1df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01a72b0 con 0x55accd2e7cd0 2024-03-16T09:15:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.369+0000 7fdebb1df740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01a8720 con 0x55accd2e2170 2024-03-16T09:15:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.369+0000 7fdebb1df740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb01a7980 con 0x55accd2e2b60 2024-03-16T09:15:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.369+0000 7fdebb1df740 10 client.0 _ll_register_callbacks cb 0x7fdea00b9090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:15:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fdeb6aba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e7cd0 0x55accd2f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fdeb62b9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e2170 0x55accd2f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fdeb62b9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e2170 0x55accd2f1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:58654/0 (socket says 172.21.15.112:58654) 2024-03-16T09:15:42.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fdeb6aba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e7cd0 0x55accd2f25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52278/0 (socket says 172.21.15.112:52278) 2024-03-16T09:15:42.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fdeb62b9640 1 -- 172.21.15.112:0/4020023235 learned_addr learned my addr 172.21.15.112:0/4020023235 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:42.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fdeb5ab8640 1 --2- 172.21.15.112:0/4020023235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e2b60 0x55accd2f15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fdeb6aba640 1 -- 172.21.15.112:0/4020023235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e2b60 msgr2=0x55accd2f15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:42.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fdeb6aba640 1 --2- 172.21.15.112:0/4020023235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e2b60 0x55accd2f15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fdeb6aba640 1 -- 172.21.15.112:0/4020023235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e2170 msgr2=0x55accd2f1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:42.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fdeb6aba640 1 --2- 172.21.15.112:0/4020023235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e2170 0x55accd2f1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fdeb6aba640 1 -- 172.21.15.112:0/4020023235 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55accd20e9d0 con 0x55accd2e7cd0 2024-03-16T09:15:42.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fdeb62b9640 1 --2- 172.21.15.112:0/4020023235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e2170 0x55accd2f1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:42.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fdeb6aba640 1 --2- 172.21.15.112:0/4020023235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e7cd0 0x55accd2f25b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdea8004400 tx=0x7fdea8029b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8c94553a8e776e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:42.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fde99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:15:42.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fde99ffb640 1 -- 172.21.15.112:0/4020023235 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8037060 con 0x55accd2e7cd0 2024-03-16T09:15:42.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fde99ffb640 1 -- 172.21.15.112:0/4020023235 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdea8004c40 con 0x55accd2e7cd0 2024-03-16T09:15:42.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fdebb1df740 1 -- 172.21.15.112:0/4020023235 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea8003e00 con 0x55accd2e7cd0 2024-03-16T09:15:42.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fde99ffb640 1 -- 172.21.15.112:0/4020023235 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fdea802e070 con 0x55accd2e7cd0 2024-03-16T09:15:42.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fdebb1df740 10 client.15408 fetch_fsmap learned FSMap version 22 2024-03-16T09:15:42.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fdebb1df740 1 -- 172.21.15.112:0/4020023235 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55accd2f3ce0 con 0x55accd2e7cd0 2024-03-16T09:15:42.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.370+0000 7fde99ffb640 1 -- 172.21.15.112:0/4020023235 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdea8003e00 con 0x55accd2e7cd0 2024-03-16T09:15:42.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.371+0000 7fde99ffb640 1 -- 172.21.15.112:0/4020023235 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fdea800b040 con 0x55accd2e7cd0 2024-03-16T09:15:42.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.371+0000 7fdebb1df740 10 client.15408 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:15:42.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.371+0000 7fdebb1df740 10 client.15408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:15:42.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.371+0000 7fdebb1df740 20 client.15408 populate_metadata read hostname 'smithi112' 2024-03-16T09:15:42.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.371+0000 7fdebb1df740 1 --2- 172.21.15.112:0/4020023235 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55accd35eaf0 0x55accd360f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:42.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.371+0000 7fdebb1df740 4 client.15408 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T09:15:42.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.371+0000 7fdebb1df740 1 -- 172.21.15.112:0/4020023235 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55accd24ed90 con 0x55accd35eaf0 2024-03-16T09:15:42.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.371+0000 7fdeb62b9640 1 --2- 172.21.15.112:0/4020023235 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55accd35eaf0 0x55accd360f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:42.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.371+0000 7fde99ffb640 10 client.15408 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:42.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.371+0000 7fdeb62b9640 1 --2- 172.21.15.112:0/4020023235 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55accd35eaf0 0x55accd360f10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1738d64ccc505f1 server_cookie=447b80b0fe8f1302 in_seq=0 out_seq=0 2024-03-16T09:15:42.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.371+0000 7fde99ffb640 1 -- 172.21.15.112:0/4020023235 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55accd24ed90 con 0x55accd35eaf0 2024-03-16T09:15:42.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.371+0000 7fde99ffb640 10 client.15408 handle_command_reply: tid=0 2024-03-16T09:15:42.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.371+0000 7fdebb1df740 1 client.15408 shutdown 2024-03-16T09:15:42.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdebb1df740 1 -- 172.21.15.112:0/4020023235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e7cd0 msgr2=0x55accd2f25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdebb1df740 1 --2- 172.21.15.112:0/4020023235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e7cd0 0x55accd2f25b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdea8004400 tx=0x7fdea8029b00 comp rx=0 tx=0).stop 2024-03-16T09:15:42.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdebb1df740 1 -- 172.21.15.112:0/4020023235 shutdown_connections 2024-03-16T09:15:42.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdebb1df740 1 --2- 172.21.15.112:0/4020023235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55accd2e7cd0 0x55accd2f25b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdebb1df740 1 --2- 172.21.15.112:0/4020023235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55accd2e2b60 0x55accd2f15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdebb1df740 1 --2- 172.21.15.112:0/4020023235 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55accd35eaf0 0x55accd360f10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdebb1df740 1 --2- 172.21.15.112:0/4020023235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55accd2e2170 0x55accd2f1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdebb1df740 1 -- 172.21.15.112:0/4020023235 >> 172.21.15.112:0/4020023235 conn(0x55accd2dd6e0 msgr2=0x55accd2ec620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:42.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fde99ffb640 0 client.15408 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:42.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdebb1df740 1 -- 172.21.15.112:0/4020023235 shutdown_connections 2024-03-16T09:15:42.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdebb1df740 1 -- 172.21.15.112:0/4020023235 wait complete. 2024-03-16T09:15:42.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdebb1df740 20 client.15408 trim_cache size 0 max 16384 2024-03-16T09:15:42.383 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:15:42.383 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T09:15:42.383 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-03-16T09:15:42.383 INFO:teuthology.orchestra.run.smithi112.stdout: "022a6cbe-bceb-4836-9fbd-830d467bedcc": { 2024-03-16T09:15:42.383 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-03-16T09:15:42.383 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "022a6cbe-bceb-4836-9fbd-830d467bedcc", 2024-03-16T09:15:42.384 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-03-16T09:15:42.384 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:15:42.384 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:15:42.384 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:15:42.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdeb851d640 1 -- 172.21.15.112:0/2732342762 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fde9c0a2890 msgr2=0x7fde9c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdeb851d640 1 --2- 172.21.15.112:0/2732342762 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fde9c0a2890 0x7fde9c0a4d50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdea8009fd0 tx=0x7fdea80095c0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdeb851d640 1 -- 172.21.15.112:0/2732342762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdeb01018d0 msgr2=0x7fdeb01981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:42.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdeb851d640 1 --2- 172.21.15.112:0/2732342762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdeb01018d0 0x7fdeb01981b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdea0034bb0 tx=0x7fdea00349a0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdeb851d640 1 -- 172.21.15.112:0/2732342762 shutdown_connections 2024-03-16T09:15:42.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdeb851d640 1 --2- 172.21.15.112:0/2732342762 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fde9c0a2890 0x7fde9c0a4d50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdeb851d640 1 --2- 172.21.15.112:0/2732342762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdeb0102290 0x7fdeb019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdeb851d640 1 --2- 172.21.15.112:0/2732342762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdeb01018d0 0x7fdeb01981b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdeb851d640 1 --2- 172.21.15.112:0/2732342762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdeb01006e0 0x7fdeb0197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:42.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdeb851d640 1 -- 172.21.15.112:0/2732342762 >> 172.21.15.112:0/2732342762 conn(0x7fdeb00fbc70 msgr2=0x7fdeb0068940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:42.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdeb851d640 1 -- 172.21.15.112:0/2732342762 shutdown_connections 2024-03-16T09:15:42.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:15:42.372+0000 7fdeb851d640 1 -- 172.21.15.112:0/2732342762 wait complete. 2024-03-16T09:15:42.387 INFO:tasks.cephfs.filesystem:scrub status for tag:022a6cbe-bceb-4836-9fbd-830d467bedcc - {'path': '/', 'tag': '022a6cbe-bceb-4836-9fbd-830d467bedcc', 'options': 'recursive,force'} 2024-03-16T09:15:42.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:41 smithi194 ceph-mon[36729]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-16T09:15:42.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:41 smithi194 ceph-mon[36729]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-16T09:15:42.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:41 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/277762392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:15:42.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:41 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/4123697086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:15:42.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:41 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1406019233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:15:43.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:42 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2830838829' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:43.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:42 smithi112 ceph-mon[32307]: scrub queued for path: / 2024-03-16T09:15:43.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:42 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:15:43.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:42 smithi112 ceph-mon[32307]: scrub summary: active paths [/] 2024-03-16T09:15:43.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:42 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:15:43.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:42 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2830838829' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:43.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:42 smithi194 ceph-mon[36729]: scrub queued for path: / 2024-03-16T09:15:43.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:42 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:15:43.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:42 smithi194 ceph-mon[36729]: scrub summary: active paths [/] 2024-03-16T09:15:43.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:42 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:15:43.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:42 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2830838829' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:15:43.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:42 smithi159 ceph-mon[36739]: scrub queued for path: / 2024-03-16T09:15:43.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:42 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:15:43.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:42 smithi159 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-16T09:15:43.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:42 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:15:44.303 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:43 smithi194 ceph-mon[36729]: pgmap v353: 97 pgs: 97 active+clean; 647 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-03-16T09:15:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:43 smithi112 ceph-mon[32307]: pgmap v353: 97 pgs: 97 active+clean; 647 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-03-16T09:15:44.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:43 smithi159 ceph-mon[36739]: pgmap v353: 97 pgs: 97 active+clean; 647 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-03-16T09:15:46.288 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:45 smithi159 ceph-mon[36739]: pgmap v354: 97 pgs: 97 active+clean; 648 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-16T09:15:46.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:45 smithi112 ceph-mon[32307]: pgmap v354: 97 pgs: 97 active+clean; 648 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-16T09:15:46.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:45 smithi194 ceph-mon[36729]: pgmap v354: 97 pgs: 97 active+clean; 648 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-03-16T09:15:47.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:47 smithi112 ceph-mon[32307]: scrub summary: idle 2024-03-16T09:15:47.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:47 smithi194 ceph-mon[36729]: scrub summary: idle 2024-03-16T09:15:47.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:47 smithi159 ceph-mon[36739]: scrub summary: idle 2024-03-16T09:15:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:48 smithi112 ceph-mon[32307]: pgmap v355: 97 pgs: 97 active+clean; 653 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-03-16T09:15:48.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:48 smithi194 ceph-mon[36729]: pgmap v355: 97 pgs: 97 active+clean; 653 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-03-16T09:15:48.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:48 smithi159 ceph-mon[36739]: pgmap v355: 97 pgs: 97 active+clean; 653 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-03-16T09:15:50.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:50 smithi112 ceph-mon[32307]: pgmap v356: 97 pgs: 97 active+clean; 653 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-16T09:15:50.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:50 smithi194 ceph-mon[36729]: pgmap v356: 97 pgs: 97 active+clean; 653 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-16T09:15:50.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:50 smithi159 ceph-mon[36739]: pgmap v356: 97 pgs: 97 active+clean; 653 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-16T09:15:52.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:52 smithi112 ceph-mon[32307]: pgmap v357: 97 pgs: 97 active+clean; 653 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-03-16T09:15:52.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:52 smithi194 ceph-mon[36729]: pgmap v357: 97 pgs: 97 active+clean; 653 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-03-16T09:15:52.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:52 smithi159 ceph-mon[36739]: pgmap v357: 97 pgs: 97 active+clean; 653 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-03-16T09:15:54.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:54 smithi112 ceph-mon[32307]: pgmap v358: 97 pgs: 97 active+clean; 653 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-03-16T09:15:54.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:54 smithi194 ceph-mon[36729]: pgmap v358: 97 pgs: 97 active+clean; 653 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-03-16T09:15:54.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:54 smithi159 ceph-mon[36739]: pgmap v358: 97 pgs: 97 active+clean; 653 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-03-16T09:15:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.169+0000 7ff0ec309640 1 -- 172.21.15.112:0/804592204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e41022b0 msgr2=0x7ff0e4108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.169+0000 7ff0ec309640 1 --2- 172.21.15.112:0/804592204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e41022b0 0x7ff0e4108b40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009ca0 tx=0x7ff0dc02c0f0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.169+0000 7ff0ec309640 1 -- 172.21.15.112:0/804592204 shutdown_connections 2024-03-16T09:15:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.169+0000 7ff0ec309640 1 --2- 172.21.15.112:0/804592204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e41022b0 0x7ff0e4108b40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.169+0000 7ff0ec309640 1 --2- 172.21.15.112:0/804592204 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0e41018f0 0x7ff0e4101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.169+0000 7ff0ec309640 1 --2- 172.21.15.112:0/804592204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0e4100700 0x7ff0e4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.169+0000 7ff0ec309640 1 -- 172.21.15.112:0/804592204 >> 172.21.15.112:0/804592204 conn(0x7ff0e40fbc70 msgr2=0x7ff0e40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.169+0000 7ff0ec309640 1 -- 172.21.15.112:0/804592204 shutdown_connections 2024-03-16T09:15:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.169+0000 7ff0ec309640 1 -- 172.21.15.112:0/804592204 wait complete. 2024-03-16T09:15:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.169+0000 7ff0ec309640 1 Processor -- start 2024-03-16T09:15:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0ec309640 1 -- start start 2024-03-16T09:15:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0ec309640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e4100700 0x7ff0e40699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0ec309640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0e41018f0 0x7ff0e419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0ec309640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0e41022b0 0x7ff0e419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0ec309640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e410c4b0 con 0x7ff0e4100700 2024-03-16T09:15:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0ec309640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e410c330 con 0x7ff0e41018f0 2024-03-16T09:15:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0ec309640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e410c630 con 0x7ff0e41022b0 2024-03-16T09:15:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0ea0a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e4100700 0x7ff0e40699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0ea0a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e4100700 0x7ff0e40699d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52844/0 (socket says 172.21.15.112:52844) 2024-03-16T09:15:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0ea0a5640 1 -- 172.21.15.112:0/2110930644 learned_addr learned my addr 172.21.15.112:0/2110930644 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0e98a4640 1 --2- 172.21.15.112:0/2110930644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0e41018f0 0x7ff0e419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0ea8a6640 1 --2- 172.21.15.112:0/2110930644 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0e41022b0 0x7ff0e419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0ea0a5640 1 -- 172.21.15.112:0/2110930644 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0e41022b0 msgr2=0x7ff0e419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0ea0a5640 1 --2- 172.21.15.112:0/2110930644 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0e41022b0 0x7ff0e419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0ea0a5640 1 -- 172.21.15.112:0/2110930644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0e41018f0 msgr2=0x7ff0e419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0ea0a5640 1 --2- 172.21.15.112:0/2110930644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0e41018f0 0x7ff0e419c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0ea0a5640 1 -- 172.21.15.112:0/2110930644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e419f780 con 0x7ff0e4100700 2024-03-16T09:15:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0e98a4640 1 --2- 172.21.15.112:0/2110930644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0e41018f0 0x7ff0e419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0ea0a5640 1 --2- 172.21.15.112:0/2110930644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e4100700 0x7ff0e40699d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000e9c0 tx=0x7ff0e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0db7fe640 1 -- 172.21.15.112:0/2110930644 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e000cd80 con 0x7ff0e4100700 2024-03-16T09:15:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0db7fe640 1 -- 172.21.15.112:0/2110930644 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0e00035f0 con 0x7ff0e4100700 2024-03-16T09:15:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0ec309640 1 -- 172.21.15.112:0/2110930644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e419fa10 con 0x7ff0e4100700 2024-03-16T09:15:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.170+0000 7ff0db7fe640 1 -- 172.21.15.112:0/2110930644 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0010690 con 0x7ff0e4100700 2024-03-16T09:15:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.171+0000 7ff0ec309640 1 -- 172.21.15.112:0/2110930644 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e419ff60 con 0x7ff0e4100700 2024-03-16T09:15:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.173+0000 7ff0db7fe640 1 -- 172.21.15.112:0/2110930644 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7ff0e00037a0 con 0x7ff0e4100700 2024-03-16T09:15:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.173+0000 7ff0eefd1740 1 Processor -- start 2024-03-16T09:15:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.173+0000 7ff0eefd1740 1 -- start start 2024-03-16T09:15:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.173+0000 7ff0eefd1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6340 0x5558c9db6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.173+0000 7ff0eefd1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6d30 0x5558c9dbb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.173+0000 7ff0eefd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9dbe330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0eefd1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5558c9d1ba40 con 0x5558c9dbbea0 2024-03-16T09:15:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0eefd1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5558c9cc4920 con 0x5558c9db6d30 2024-03-16T09:15:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0eefd1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5558c9d17ae0 con 0x5558c9db6340 2024-03-16T09:15:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0ea0a5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6d30 0x5558c9dbb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0e98a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9dbe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0ea8a6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6340 0x5558c9db6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0ea8a6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6340 0x5558c9db6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:49882/0 (socket says 172.21.15.112:49882) 2024-03-16T09:15:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0ea0a5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6d30 0x5558c9dbb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:49908/0 (socket says 172.21.15.112:49908) 2024-03-16T09:15:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0ea8a6640 1 -- 172.21.15.112:0/4021254569 learned_addr learned my addr 172.21.15.112:0/4021254569 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0db7fe640 1 --2- 172.21.15.112:0/2110930644 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff0d00a2940 0x7ff0d00a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0ea8a6640 1 --2- 172.21.15.112:0/2110930644 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff0d00a2940 0x7ff0d00a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0ea0a5640 1 -- 172.21.15.112:0/4021254569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6340 msgr2=0x5558c9db6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0ea0a5640 1 --2- 172.21.15.112:0/4021254569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6340 0x5558c9db6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0ea0a5640 1 -- 172.21.15.112:0/4021254569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 msgr2=0x5558c9dbe330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0ea0a5640 1 --2- 172.21.15.112:0/4021254569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9dbe330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0db7fe640 1 -- 172.21.15.112:0/2110930644 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff0e00cf060 con 0x7ff0e4100700 2024-03-16T09:15:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0ea0a5640 1 -- 172.21.15.112:0/4021254569 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558c9cdc460 con 0x5558c9db6d30 2024-03-16T09:15:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0ea8a6640 1 --2- 172.21.15.112:0/4021254569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6340 0x5558c9db6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0e98a4640 1 --2- 172.21.15.112:0/4021254569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9dbe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0ea0a5640 1 --2- 172.21.15.112:0/4021254569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6d30 0x5558c9dbb830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00cc9c0 tx=0x7ff0e0003ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8539f2b5a9f55d2a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:55.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.174+0000 7ff0ea8a6640 1 --2- 172.21.15.112:0/2110930644 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff0d00a2940 0x7ff0d00a4e00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5558c9db7590 tx=0x7ff0dc02c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.175+0000 7ff0d8bfe640 1 -- 172.21.15.112:0/4021254569 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e009b390 con 0x5558c9db6d30 2024-03-16T09:15:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.175+0000 7ff0d8bfe640 1 -- 172.21.15.112:0/4021254569 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0e0003bc0 con 0x5558c9db6d30 2024-03-16T09:15:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.175+0000 7ff0d8bfe640 1 -- 172.21.15.112:0/4021254569 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e009b1b0 con 0x5558c9db6d30 2024-03-16T09:15:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.175+0000 7ff0eefd1740 1 -- 172.21.15.112:0/4021254569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6d30 msgr2=0x5558c9dbb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.175+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/4021254569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6d30 0x5558c9dbb830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00cc9c0 tx=0x7ff0e0003ec0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.175+0000 7ff0eefd1740 1 -- 172.21.15.112:0/4021254569 shutdown_connections 2024-03-16T09:15:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.175+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/4021254569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9dbe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.175+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/4021254569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6d30 0x5558c9dbb830 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.175+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/4021254569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6340 0x5558c9db6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.175+0000 7ff0eefd1740 1 -- 172.21.15.112:0/4021254569 >> 172.21.15.112:0/4021254569 conn(0x5558c9db18b0 msgr2=0x5558c9db1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.175+0000 7ff0eefd1740 1 -- 172.21.15.112:0/4021254569 shutdown_connections 2024-03-16T09:15:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.176+0000 7ff0eefd1740 1 -- 172.21.15.112:0/4021254569 wait complete. 2024-03-16T09:15:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.176+0000 7ff0eefd1740 1 -- start start 2024-03-16T09:15:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.176+0000 7ff0eefd1740 1 Processor -- start 2024-03-16T09:15:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.176+0000 7ff0eefd1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6340 0x5558c9dc0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.176+0000 7ff0eefd1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6d30 0x5558c9dc0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.176+0000 7ff0eefd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9dc7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.176+0000 7ff0eefd1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5558c9dbedf0 con 0x5558c9dbbea0 2024-03-16T09:15:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.176+0000 7ff0eefd1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5558c9cd7d70 con 0x5558c9db6d30 2024-03-16T09:15:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.176+0000 7ff0eefd1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5558c9dbef70 con 0x5558c9db6340 2024-03-16T09:15:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.176+0000 7ff0eefd1740 10 client.0 _ll_register_callbacks cb 0x5558c9a2fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:15:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.176+0000 7ff0ea8a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9dc7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.176+0000 7ff0e98a4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6d30 0x5558c9dc0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.176+0000 7ff0ea0a5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6340 0x5558c9dc0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.176+0000 7ff0ea8a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9dc7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52870/0 (socket says 172.21.15.112:52870) 2024-03-16T09:15:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.176+0000 7ff0ea8a6640 1 -- 172.21.15.112:0/1977145542 learned_addr learned my addr 172.21.15.112:0/1977145542 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0ea8a6640 1 -- 172.21.15.112:0/1977145542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6340 msgr2=0x5558c9dc0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0ea8a6640 1 --2- 172.21.15.112:0/1977145542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6340 0x5558c9dc0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0ea8a6640 1 -- 172.21.15.112:0/1977145542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6d30 msgr2=0x5558c9dc0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0ea8a6640 1 --2- 172.21.15.112:0/1977145542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6d30 0x5558c9dc0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0ea8a6640 1 -- 172.21.15.112:0/1977145542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558c9a30210 con 0x5558c9dbbea0 2024-03-16T09:15:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0ea0a5640 1 --2- 172.21.15.112:0/1977145542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6340 0x5558c9dc0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0e98a4640 1 --2- 172.21.15.112:0/1977145542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6d30 0x5558c9dc0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0ea8a6640 1 --2- 172.21.15.112:0/1977145542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9dc7b90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc037040 tx=0x7ff0dc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d505cb398d9b22 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:15:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0cdffb640 1 -- 172.21.15.112:0/1977145542 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc052060 con 0x5558c9dbbea0 2024-03-16T09:15:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0cdffb640 1 -- 172.21.15.112:0/1977145542 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff0dc004450 con 0x5558c9dbbea0 2024-03-16T09:15:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0eefd1740 1 -- 172.21.15.112:0/1977145542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5558c9d28f80 con 0x5558c9dbbea0 2024-03-16T09:15:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0cdffb640 1 -- 172.21.15.112:0/1977145542 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff0dc039040 con 0x5558c9dbbea0 2024-03-16T09:15:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0eefd1740 10 client.15420 fetch_fsmap learned FSMap version 22 2024-03-16T09:15:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0eefd1740 1 -- 172.21.15.112:0/1977145542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5558c9afc620 con 0x5558c9dbbea0 2024-03-16T09:15:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0cdffb640 1 -- 172.21.15.112:0/1977145542 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x5558c9d28f80 con 0x5558c9dbbea0 2024-03-16T09:15:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0cdffb640 1 -- 172.21.15.112:0/1977145542 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ff0dc03f070 con 0x5558c9dbbea0 2024-03-16T09:15:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0eefd1740 10 client.15420 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:15:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0eefd1740 10 client.15420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:15:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0eefd1740 20 client.15420 populate_metadata read hostname 'smithi112' 2024-03-16T09:15:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.177+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/1977145542 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5558c9dcc4e0 0x5558c9dce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.178+0000 7ff0eefd1740 4 client.15420 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:15:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.178+0000 7ff0eefd1740 1 -- 172.21.15.112:0/1977145542 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558c9d232d0 con 0x5558c9dcc4e0 2024-03-16T09:15:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.178+0000 7ff0ea0a5640 1 --2- 172.21.15.112:0/1977145542 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5558c9dcc4e0 0x5558c9dce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.178+0000 7ff0ea0a5640 1 --2- 172.21.15.112:0/1977145542 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5558c9dcc4e0 0x5558c9dce8d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8f5a7aeebd4749f server_cookie=1adc414cf915a476 in_seq=0 out_seq=0 2024-03-16T09:15:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.178+0000 7ff0cdffb640 10 client.15420 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.179+0000 7ff0cdffb640 1 -- 172.21.15.112:0/1977145542 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5558c9d232d0 con 0x5558c9dcc4e0 2024-03-16T09:15:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.179+0000 7ff0cdffb640 10 client.15420 handle_command_reply: tid=0 2024-03-16T09:15:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.179+0000 7ff0eefd1740 1 client.15420 shutdown 2024-03-16T09:15:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.179+0000 7ff0eefd1740 1 -- 172.21.15.112:0/1977145542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 msgr2=0x5558c9dc7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:55.189 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.179+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/1977145542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9dc7b90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc037040 tx=0x7ff0dc0095c0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.179+0000 7ff0eefd1740 1 -- 172.21.15.112:0/1977145542 shutdown_connections 2024-03-16T09:15:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.179+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/1977145542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9dc7b90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.179+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/1977145542 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5558c9dcc4e0 0x5558c9dce8d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.179+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/1977145542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6d30 0x5558c9dc0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.179+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/1977145542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6340 0x5558c9dc0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.179+0000 7ff0cdffb640 0 client.15420 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.179+0000 7ff0eefd1740 1 -- 172.21.15.112:0/1977145542 >> 172.21.15.112:0/1977145542 conn(0x5558c9db18b0 msgr2=0x5558c9db0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.179+0000 7ff0eefd1740 1 -- 172.21.15.112:0/1977145542 shutdown_connections 2024-03-16T09:15:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.179+0000 7ff0eefd1740 1 -- 172.21.15.112:0/1977145542 wait complete. 2024-03-16T09:15:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.179+0000 7ff0eefd1740 20 client.15420 trim_cache size 0 max 16384 2024-03-16T09:15:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.209+0000 7ff0eefd1740 1 Processor -- start 2024-03-16T09:15:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.209+0000 7ff0eefd1740 1 -- start start 2024-03-16T09:15:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.209+0000 7ff0eefd1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6340 0x5558c9db8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.209+0000 7ff0eefd1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6d30 0x5558c9db8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:55.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.209+0000 7ff0eefd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9dc7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.209+0000 7ff0eefd1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e41ab650 con 0x5558c9dbbea0 2024-03-16T09:15:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.209+0000 7ff0eefd1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e41a8f20 con 0x5558c9db6340 2024-03-16T09:15:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.209+0000 7ff0eefd1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e41a6d70 con 0x5558c9db6d30 2024-03-16T09:15:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.209+0000 7ff0ea8a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9dc7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.209+0000 7ff0ea8a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9dc7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52876/0 (socket says 172.21.15.112:52876) 2024-03-16T09:15:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.209+0000 7ff0ea8a6640 1 -- 172.21.15.112:0/3770447996 learned_addr learned my addr 172.21.15.112:0/3770447996 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.209+0000 7ff0ea0a5640 1 --2- 172.21.15.112:0/3770447996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6340 0x5558c9db8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.209+0000 7ff0e98a4640 1 --2- 172.21.15.112:0/3770447996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6d30 0x5558c9db8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.210+0000 7ff0ea8a6640 1 -- 172.21.15.112:0/3770447996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6d30 msgr2=0x5558c9db8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.210+0000 7ff0ea8a6640 1 --2- 172.21.15.112:0/3770447996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6d30 0x5558c9db8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.210+0000 7ff0ea8a6640 1 -- 172.21.15.112:0/3770447996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6340 msgr2=0x5558c9db8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.210+0000 7ff0ea8a6640 1 --2- 172.21.15.112:0/3770447996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6340 0x5558c9db8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.210+0000 7ff0ea8a6640 1 -- 172.21.15.112:0/3770447996 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558c9db75c0 con 0x5558c9dbbea0 2024-03-16T09:15:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.210+0000 7ff0ea8a6640 1 --2- 172.21.15.112:0/3770447996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9dc7b50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc02c630 tx=0x7ff0dc005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ecc9fea3cff421 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.210+0000 7ff0cffff640 1 -- 172.21.15.112:0/3770447996 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc05d070 con 0x5558c9dbbea0 2024-03-16T09:15:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.210+0000 7ff0cffff640 1 -- 172.21.15.112:0/3770447996 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0dc0346c0 con 0x5558c9dbbea0 2024-03-16T09:15:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.210+0000 7ff0cffff640 1 -- 172.21.15.112:0/3770447996 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc052040 con 0x5558c9dbbea0 2024-03-16T09:15:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0eefd1740 1 -- 172.21.15.112:0/3770447996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 msgr2=0x5558c9dc7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/3770447996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9dc7b50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc02c630 tx=0x7ff0dc005f30 comp rx=0 tx=0).stop 2024-03-16T09:15:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0eefd1740 1 -- 172.21.15.112:0/3770447996 shutdown_connections 2024-03-16T09:15:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/3770447996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9dc7b50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/3770447996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6d30 0x5558c9db8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/3770447996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6340 0x5558c9db8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0eefd1740 1 -- 172.21.15.112:0/3770447996 >> 172.21.15.112:0/3770447996 conn(0x5558c9db18b0 msgr2=0x5558c9dc0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0eefd1740 1 -- 172.21.15.112:0/3770447996 shutdown_connections 2024-03-16T09:15:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0eefd1740 1 -- 172.21.15.112:0/3770447996 wait complete. 2024-03-16T09:15:55.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0eefd1740 1 -- start start 2024-03-16T09:15:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0eefd1740 1 Processor -- start 2024-03-16T09:15:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0eefd1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6340 0x5558c9dc4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0eefd1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6d30 0x5558c9dc4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0eefd1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9e307e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0eefd1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e41a73e0 con 0x5558c9dbbea0 2024-03-16T09:15:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0eefd1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e41a8850 con 0x5558c9db6340 2024-03-16T09:15:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0eefd1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e41a7ab0 con 0x5558c9db6d30 2024-03-16T09:15:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0eefd1740 10 client.0 _ll_register_callbacks cb 0x7ff0d4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:15:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0ea8a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9e307e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0ea8a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9e307e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52892/0 (socket says 172.21.15.112:52892) 2024-03-16T09:15:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0ea8a6640 1 -- 172.21.15.112:0/3897201923 learned_addr learned my addr 172.21.15.112:0/3897201923 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:15:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0e98a4640 1 --2- 172.21.15.112:0/3897201923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6d30 0x5558c9dc4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.211+0000 7ff0ea0a5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6340 0x5558c9dc4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.212+0000 7ff0ea8a6640 1 -- 172.21.15.112:0/3897201923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6d30 msgr2=0x5558c9dc4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.212+0000 7ff0ea8a6640 1 --2- 172.21.15.112:0/3897201923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6d30 0x5558c9dc4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.212+0000 7ff0ea8a6640 1 -- 172.21.15.112:0/3897201923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6340 msgr2=0x5558c9dc4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:15:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.212+0000 7ff0ea8a6640 1 --2- 172.21.15.112:0/3897201923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6340 0x5558c9dc4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.212+0000 7ff0ea8a6640 1 -- 172.21.15.112:0/3897201923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558c9e30d20 con 0x5558c9dbbea0 2024-03-16T09:15:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.212+0000 7ff0e98a4640 1 --2- 172.21.15.112:0/3897201923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6d30 0x5558c9dc4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.212+0000 7ff0ea0a5640 1 --2- 172.21.15.112:0/3897201923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6340 0x5558c9dc4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:15:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.212+0000 7ff0ea8a6640 1 --2- 172.21.15.112:0/3897201923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9e307e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc005e30 tx=0x7ff0dc032a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511fc4d7b4ea869a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:15:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.212+0000 7ff0cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:15:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.212+0000 7ff0cdffb640 1 -- 172.21.15.112:0/3897201923 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc066020 con 0x5558c9dbbea0 2024-03-16T09:15:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.212+0000 7ff0cdffb640 1 -- 172.21.15.112:0/3897201923 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff0dc034870 con 0x5558c9dbbea0 2024-03-16T09:15:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.212+0000 7ff0eefd1740 1 -- 172.21.15.112:0/3897201923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0dc0346c0 con 0x5558c9dbbea0 2024-03-16T09:15:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.212+0000 7ff0cdffb640 1 -- 172.21.15.112:0/3897201923 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff0dc05d030 con 0x5558c9dbbea0 2024-03-16T09:15:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.212+0000 7ff0eefd1740 10 client.15426 fetch_fsmap learned FSMap version 22 2024-03-16T09:15:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.212+0000 7ff0eefd1740 1 -- 172.21.15.112:0/3897201923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5558c9e31f10 con 0x5558c9dbbea0 2024-03-16T09:15:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.212+0000 7ff0cdffb640 1 -- 172.21.15.112:0/3897201923 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0dc0346c0 con 0x5558c9dbbea0 2024-03-16T09:15:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.213+0000 7ff0cdffb640 1 -- 172.21.15.112:0/3897201923 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ff0dc03f070 con 0x5558c9dbbea0 2024-03-16T09:15:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.213+0000 7ff0eefd1740 10 client.15426 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:15:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.213+0000 7ff0eefd1740 10 client.15426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:15:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.213+0000 7ff0eefd1740 20 client.15426 populate_metadata read hostname 'smithi112' 2024-03-16T09:15:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.213+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/3897201923 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5558c9e328c0 0x5558c9e34ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:15:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.213+0000 7ff0eefd1740 4 client.15426 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T09:15:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.213+0000 7ff0eefd1740 1 -- 172.21.15.112:0/3897201923 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558c9ca1770 con 0x5558c9e328c0 2024-03-16T09:15:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.213+0000 7ff0ea0a5640 1 --2- 172.21.15.112:0/3897201923 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5558c9e328c0 0x5558c9e34ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:15:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.213+0000 7ff0cdffb640 10 client.15426 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.213+0000 7ff0ea0a5640 1 --2- 172.21.15.112:0/3897201923 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5558c9e328c0 0x5558c9e34ce0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4cf8f373a4c3753 server_cookie=9345a8d1b788a550 in_seq=0 out_seq=0 2024-03-16T09:15:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.225+0000 7ff0cdffb640 1 -- 172.21.15.112:0/3897201923 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5558c9ca1770 con 0x5558c9e328c0 2024-03-16T09:15:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.225+0000 7ff0cdffb640 10 client.15426 handle_command_reply: tid=0 2024-03-16T09:15:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.225+0000 7ff0eefd1740 1 client.15426 shutdown 2024-03-16T09:15:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.225+0000 7ff0eefd1740 1 -- 172.21.15.112:0/3897201923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 msgr2=0x5558c9e307e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.225+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/3897201923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9e307e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc005e30 tx=0x7ff0dc032a80 comp rx=0 tx=0).stop 2024-03-16T09:15:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0eefd1740 1 -- 172.21.15.112:0/3897201923 shutdown_connections 2024-03-16T09:15:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/3897201923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5558c9dbbea0 0x5558c9e307e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/3897201923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5558c9db6d30 0x5558c9dc4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/3897201923 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5558c9e328c0 0x5558c9e34ce0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0eefd1740 1 --2- 172.21.15.112:0/3897201923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558c9db6340 0x5558c9dc4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0cdffb640 0 client.15426 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:15:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0eefd1740 1 -- 172.21.15.112:0/3897201923 >> 172.21.15.112:0/3897201923 conn(0x5558c9db18b0 msgr2=0x5558c9dc0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0eefd1740 1 -- 172.21.15.112:0/3897201923 shutdown_connections 2024-03-16T09:15:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0eefd1740 1 -- 172.21.15.112:0/3897201923 wait complete. 2024-03-16T09:15:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0eefd1740 20 client.15426 trim_cache size 0 max 16384 2024-03-16T09:15:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-03-16T09:15:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-03-16T09:15:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-03-16T09:15:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-03-16T09:15:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0ec309640 1 -- 172.21.15.112:0/2110930644 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff0d00a2940 msgr2=0x7ff0d00a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0ec309640 1 --2- 172.21.15.112:0/2110930644 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff0d00a2940 0x7ff0d00a4e00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5558c9db7590 tx=0x7ff0dc02c0f0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0ec309640 1 -- 172.21.15.112:0/2110930644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e4100700 msgr2=0x7ff0e40699d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:15:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0ec309640 1 --2- 172.21.15.112:0/2110930644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e4100700 0x7ff0e40699d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000e9c0 tx=0x7ff0e000ee80 comp rx=0 tx=0).stop 2024-03-16T09:15:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0ec309640 1 -- 172.21.15.112:0/2110930644 shutdown_connections 2024-03-16T09:15:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0ec309640 1 --2- 172.21.15.112:0/2110930644 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff0d00a2940 0x7ff0d00a4e00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0ec309640 1 --2- 172.21.15.112:0/2110930644 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0e41022b0 0x7ff0e419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0ec309640 1 --2- 172.21.15.112:0/2110930644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0e41018f0 0x7ff0e419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0ec309640 1 --2- 172.21.15.112:0/2110930644 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0e4100700 0x7ff0e40699d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:15:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0ec309640 1 -- 172.21.15.112:0/2110930644 >> 172.21.15.112:0/2110930644 conn(0x7ff0e40fbc70 msgr2=0x7ff0e4104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:15:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0ec309640 1 -- 172.21.15.112:0/2110930644 shutdown_connections 2024-03-16T09:15:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:15:55.226+0000 7ff0ec309640 1 -- 172.21.15.112:0/2110930644 wait complete. 2024-03-16T09:15:55.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:55 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:15:55.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:55 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:15:55.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:55 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:15:56.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:56 smithi112 ceph-mon[32307]: pgmap v359: 97 pgs: 97 active+clean; 653 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-16T09:15:56.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:56 smithi194 ceph-mon[36729]: pgmap v359: 97 pgs: 97 active+clean; 653 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-16T09:15:56.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:56 smithi159 ceph-mon[36739]: pgmap v359: 97 pgs: 97 active+clean; 653 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-16T09:15:57.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.842 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.843 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.843 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.843 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:57 smithi112 ceph-mon[32307]: pgmap v360: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-03-16T09:15:57.843 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:15:57.843 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:15:57.843 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:57 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:57 smithi194 ceph-mon[36729]: pgmap v360: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-03-16T09:15:57.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:15:57.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:15:57.902 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:57 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:15:57.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:57 smithi159 ceph-mon[36739]: pgmap v360: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-03-16T09:15:57.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:15:57.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:15:57.987 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:57 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:16:00.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:15:59 smithi159 ceph-mon[36739]: pgmap v361: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-03-16T09:16:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:15:59 smithi112 ceph-mon[32307]: pgmap v361: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-03-16T09:16:00.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:15:59 smithi194 ceph-mon[36729]: pgmap v361: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-03-16T09:16:02.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:01 smithi159 ceph-mon[36739]: pgmap v362: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-16T09:16:02.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:01 smithi112 ceph-mon[32307]: pgmap v362: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-16T09:16:02.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:01 smithi194 ceph-mon[36729]: pgmap v362: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-16T09:16:04.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:03 smithi159 ceph-mon[36739]: pgmap v363: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-16T09:16:04.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:03 smithi112 ceph-mon[32307]: pgmap v363: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-16T09:16:04.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:03 smithi194 ceph-mon[36729]: pgmap v363: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-16T09:16:06.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:05 smithi159 ceph-mon[36739]: pgmap v364: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-16T09:16:06.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:05 smithi112 ceph-mon[32307]: pgmap v364: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-16T09:16:06.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:05 smithi194 ceph-mon[36729]: pgmap v364: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-16T09:16:08.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:07 smithi159 ceph-mon[36739]: pgmap v365: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-16T09:16:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.323+0000 7f42c6190640 1 -- 172.21.15.112:0/52121035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42c01033e0 msgr2=0x7f42c0109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.323+0000 7f42c6190640 1 --2- 172.21.15.112:0/52121035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42c01033e0 0x7f42c0109c70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009ca0 tx=0x7f42b802c0f0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.323+0000 7f42c6190640 1 -- 172.21.15.112:0/52121035 shutdown_connections 2024-03-16T09:16:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.323+0000 7f42c6190640 1 --2- 172.21.15.112:0/52121035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42c01033e0 0x7f42c0109c70 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.323+0000 7f42c6190640 1 --2- 172.21.15.112:0/52121035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42c0102a20 0x7f42c0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.323+0000 7f42c6190640 1 --2- 172.21.15.112:0/52121035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42c0101830 0x7f42c0101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.323+0000 7f42c6190640 1 -- 172.21.15.112:0/52121035 >> 172.21.15.112:0/52121035 conn(0x7f42c00fcdc0 msgr2=0x7f42c00ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.323+0000 7f42c6190640 1 -- 172.21.15.112:0/52121035 shutdown_connections 2024-03-16T09:16:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.323+0000 7f42c6190640 1 -- 172.21.15.112:0/52121035 wait complete. 2024-03-16T09:16:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.323+0000 7f42c6190640 1 Processor -- start 2024-03-16T09:16:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.323+0000 7f42c6190640 1 -- start start 2024-03-16T09:16:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.324+0000 7f42c6190640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42c0101830 0x7f42c0197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.324+0000 7f42c6190640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42c0102a20 0x7f42c0197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.324+0000 7f42c6190640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42c01033e0 0x7f42c019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.324+0000 7f42c6190640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f42c010d5e0 con 0x7f42c0101830 2024-03-16T09:16:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.324+0000 7f42c6190640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f42c010d460 con 0x7f42c0102a20 2024-03-16T09:16:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.324+0000 7f42c6190640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f42c010d760 con 0x7f42c01033e0 2024-03-16T09:16:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.324+0000 7f42bf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42c0101830 0x7f42c0197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.324+0000 7f42bf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42c0101830 0x7f42c0197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56214/0 (socket says 172.21.15.112:56214) 2024-03-16T09:16:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.324+0000 7f42bffff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42c01033e0 0x7f42c019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.324+0000 7f42beffd640 1 --2- 172.21.15.112:0/1907524650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42c0102a20 0x7f42c0197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.324+0000 7f42bf7fe640 1 -- 172.21.15.112:0/1907524650 learned_addr learned my addr 172.21.15.112:0/1907524650 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.324+0000 7f42bffff640 1 -- 172.21.15.112:0/1907524650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42c0102a20 msgr2=0x7f42c0197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.324+0000 7f42bffff640 1 --2- 172.21.15.112:0/1907524650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42c0102a20 0x7f42c0197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.324+0000 7f42bffff640 1 -- 172.21.15.112:0/1907524650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42c0101830 msgr2=0x7f42c0197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.324+0000 7f42bffff640 1 --2- 172.21.15.112:0/1907524650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42c0101830 0x7f42c0197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.324+0000 7f42bffff640 1 -- 172.21.15.112:0/1907524650 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c019f6c0 con 0x7f42c01033e0 2024-03-16T09:16:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.324+0000 7f42beffd640 1 --2- 172.21.15.112:0/1907524650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42c0102a20 0x7f42c0197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.324+0000 7f42bffff640 1 --2- 172.21.15.112:0/1907524650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42c01033e0 0x7f42c019f050 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f42b802c630 tx=0x7f42b80346f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.325+0000 7f42bcff9640 1 -- 172.21.15.112:0/1907524650 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8039040 con 0x7f42c01033e0 2024-03-16T09:16:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.325+0000 7f42bcff9640 1 -- 172.21.15.112:0/1907524650 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f42b8044070 con 0x7f42c01033e0 2024-03-16T09:16:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.325+0000 7f42bcff9640 1 -- 172.21.15.112:0/1907524650 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8004500 con 0x7f42c01033e0 2024-03-16T09:16:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.325+0000 7f42c6190640 1 -- 172.21.15.112:0/1907524650 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c019f950 con 0x7f42c01033e0 2024-03-16T09:16:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.325+0000 7f42c6190640 1 -- 172.21.15.112:0/1907524650 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c019fdc0 con 0x7f42c01033e0 2024-03-16T09:16:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.327+0000 7f42bcff9640 1 -- 172.21.15.112:0/1907524650 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7f42b80046a0 con 0x7f42c01033e0 2024-03-16T09:16:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.327+0000 7f42c8d77740 1 Processor -- start 2024-03-16T09:16:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.327+0000 7f42c8d77740 1 -- start start 2024-03-16T09:16:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.327+0000 7f42bcff9640 1 --2- 172.21.15.112:0/1907524650 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f42a80a2940 0x7f42a80a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.327+0000 7f42c8d77740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd923340 0x55bafd923760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.327+0000 7f42c8d77740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923d30 0x55bafd928830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.327+0000 7f42c8d77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd928ea0 0x55bafd92b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.327+0000 7f42c8d77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bafd888a40 con 0x55bafd928ea0 2024-03-16T09:16:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.327+0000 7f42c8d77740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bafd831920 con 0x55bafd923d30 2024-03-16T09:16:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.327+0000 7f42c8d77740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bafd884ae0 con 0x55bafd923340 2024-03-16T09:16:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.328+0000 7f42beffd640 1 --2- 172.21.15.112:0/1907524650 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f42a80a2940 0x7f42a80a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.328+0000 7f42bffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd928ea0 0x55bafd92b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.328+0000 7f42bffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd928ea0 0x55bafd92b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56224/0 (socket says 172.21.15.112:56224) 2024-03-16T09:16:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.328+0000 7f42bf7fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd923340 0x55bafd923760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.328+0000 7f42bffff640 1 -- 172.21.15.112:0/289018426 learned_addr learned my addr 172.21.15.112:0/289018426 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.328+0000 7f42beffd640 1 --2- 172.21.15.112:0/289018426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923d30 0x55bafd928830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.328+0000 7f42bcff9640 1 -- 172.21.15.112:0/1907524650 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f42b80f1050 con 0x7f42c01033e0 2024-03-16T09:16:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.328+0000 7f42bffff640 1 -- 172.21.15.112:0/289018426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd923340 msgr2=0x55bafd923760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.328+0000 7f42bffff640 1 --2- 172.21.15.112:0/289018426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd923340 0x55bafd923760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.328+0000 7f42bffff640 1 -- 172.21.15.112:0/289018426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923d30 msgr2=0x55bafd928830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.328+0000 7f42bffff640 1 --2- 172.21.15.112:0/289018426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923d30 0x55bafd928830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.328+0000 7f42bffff640 1 -- 172.21.15.112:0/289018426 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bafd849460 con 0x55bafd928ea0 2024-03-16T09:16:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.328+0000 7f42bf7fe640 1 --2- 172.21.15.112:0/289018426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd923340 0x55bafd923760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.328+0000 7f42beffd640 1 --2- 172.21.15.112:0/1907524650 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f42a80a2940 0x7f42a80a4e00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009fd0 tx=0x7f42b4009ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.328+0000 7f42bffff640 1 --2- 172.21.15.112:0/289018426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd928ea0 0x55bafd92b330 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f42b80c5c20 tx=0x7f42b8004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7a9c44cd693e55 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.328+0000 7f42a5ffb640 1 -- 172.21.15.112:0/289018426 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b80bd4f0 con 0x55bafd928ea0 2024-03-16T09:16:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.328+0000 7f42a5ffb640 1 -- 172.21.15.112:0/289018426 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f42b80be0b0 con 0x55bafd928ea0 2024-03-16T09:16:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.328+0000 7f42a5ffb640 1 -- 172.21.15.112:0/289018426 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b80d6a70 con 0x55bafd928ea0 2024-03-16T09:16:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.329+0000 7f42c8d77740 1 -- 172.21.15.112:0/289018426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd928ea0 msgr2=0x55bafd92b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.329+0000 7f42c8d77740 1 --2- 172.21.15.112:0/289018426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd928ea0 0x55bafd92b330 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f42b80c5c20 tx=0x7f42b8004990 comp rx=0 tx=0).stop 2024-03-16T09:16:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.329+0000 7f42c8d77740 1 -- 172.21.15.112:0/289018426 shutdown_connections 2024-03-16T09:16:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.329+0000 7f42c8d77740 1 --2- 172.21.15.112:0/289018426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd928ea0 0x55bafd92b330 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.329+0000 7f42c8d77740 1 --2- 172.21.15.112:0/289018426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923d30 0x55bafd928830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.329+0000 7f42c8d77740 1 --2- 172.21.15.112:0/289018426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd923340 0x55bafd923760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.329+0000 7f42c8d77740 1 -- 172.21.15.112:0/289018426 >> 172.21.15.112:0/289018426 conn(0x55bafd91e8b0 msgr2=0x55bafd91ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.329+0000 7f42c8d77740 1 -- 172.21.15.112:0/289018426 shutdown_connections 2024-03-16T09:16:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.329+0000 7f42c8d77740 1 -- 172.21.15.112:0/289018426 wait complete. 2024-03-16T09:16:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.329+0000 7f42c8d77740 1 -- start start 2024-03-16T09:16:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42c8d77740 1 Processor -- start 2024-03-16T09:16:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42c8d77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923340 0x55bafd92d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42c8d77740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923d30 0x55bafd92d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42c8d77740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd928ea0 0x55bafd9348f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42c8d77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55bafd92bc10 con 0x55bafd923340 2024-03-16T09:16:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42c8d77740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bafd844d70 con 0x55bafd923d30 2024-03-16T09:16:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42c8d77740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bafd92bd90 con 0x55bafd928ea0 2024-03-16T09:16:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42c8d77740 10 client.0 _ll_register_callbacks cb 0x55bafd59ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42bf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923340 0x55bafd92d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42bf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923340 0x55bafd92d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56240/0 (socket says 172.21.15.112:56240) 2024-03-16T09:16:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42bf7fe640 1 -- 172.21.15.112:0/782541841 learned_addr learned my addr 172.21.15.112:0/782541841 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42beffd640 1 --2- 172.21.15.112:0/782541841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923d30 0x55bafd92d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42bffff640 1 --2- 172.21.15.112:0/782541841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd928ea0 0x55bafd9348f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42bf7fe640 1 -- 172.21.15.112:0/782541841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd928ea0 msgr2=0x55bafd9348f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42bf7fe640 1 --2- 172.21.15.112:0/782541841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd928ea0 0x55bafd9348f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42bf7fe640 1 -- 172.21.15.112:0/782541841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923d30 msgr2=0x55bafd92d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42bf7fe640 1 --2- 172.21.15.112:0/782541841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923d30 0x55bafd92d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42bf7fe640 1 -- 172.21.15.112:0/782541841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bafd59d210 con 0x55bafd923340 2024-03-16T09:16:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42bffff640 1 --2- 172.21.15.112:0/782541841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd928ea0 0x55bafd9348f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42beffd640 1 --2- 172.21.15.112:0/782541841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923d30 0x55bafd92d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f42bf7fe640 1 --2- 172.21.15.112:0/782541841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923340 0x55bafd92d370 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f42b0012790 tx=0x7f42b0012c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48536d5a99143c72 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.330+0000 7f429effd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:16:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.331+0000 7f429effd640 1 -- 172.21.15.112:0/782541841 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b000f8c0 con 0x55bafd923340 2024-03-16T09:16:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.331+0000 7f429effd640 1 -- 172.21.15.112:0/782541841 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f42b000fa60 con 0x55bafd923340 2024-03-16T09:16:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.331+0000 7f42c8d77740 1 -- 172.21.15.112:0/782541841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42b80be0b0 con 0x55bafd923340 2024-03-16T09:16:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.331+0000 7f429effd640 1 -- 172.21.15.112:0/782541841 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f42b000fc60 con 0x55bafd923340 2024-03-16T09:16:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.331+0000 7f42c8d77740 10 client.15438 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.331+0000 7f42c8d77740 1 -- 172.21.15.112:0/782541841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55bafd669620 con 0x55bafd923340 2024-03-16T09:16:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.331+0000 7f429effd640 1 -- 172.21.15.112:0/782541841 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f42b80be0b0 con 0x55bafd923340 2024-03-16T09:16:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.331+0000 7f429effd640 1 -- 172.21.15.112:0/782541841 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f42b0004c10 con 0x55bafd923340 2024-03-16T09:16:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.331+0000 7f42c8d77740 10 client.15438 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.331+0000 7f42c8d77740 10 client.15438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.331+0000 7f42c8d77740 20 client.15438 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.331+0000 7f42c8d77740 1 --2- 172.21.15.112:0/782541841 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55bafd939250 0x55bafd93b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.331+0000 7f42c8d77740 4 client.15438 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:16:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.331+0000 7f42c8d77740 1 -- 172.21.15.112:0/782541841 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bafd895f80 con 0x55bafd939250 2024-03-16T09:16:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.331+0000 7f42beffd640 1 --2- 172.21.15.112:0/782541841 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55bafd939250 0x55bafd93b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.332+0000 7f429effd640 10 client.15438 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.332+0000 7f42beffd640 1 --2- 172.21.15.112:0/782541841 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55bafd939250 0x55bafd93b6c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69495131f3bfc2b2 server_cookie=ea641063755369d3 in_seq=0 out_seq=0 2024-03-16T09:16:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.332+0000 7f429effd640 1 -- 172.21.15.112:0/782541841 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bafd895f80 con 0x55bafd939250 2024-03-16T09:16:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.332+0000 7f429effd640 10 client.15438 handle_command_reply: tid=0 2024-03-16T09:16:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.332+0000 7f42c8d77740 1 client.15438 shutdown 2024-03-16T09:16:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.333+0000 7f42c8d77740 1 -- 172.21.15.112:0/782541841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923340 msgr2=0x55bafd92d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.333+0000 7f42c8d77740 1 --2- 172.21.15.112:0/782541841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923340 0x55bafd92d370 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f42b0012790 tx=0x7f42b0012c50 comp rx=0 tx=0).stop 2024-03-16T09:16:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.333+0000 7f42c8d77740 1 -- 172.21.15.112:0/782541841 shutdown_connections 2024-03-16T09:16:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.333+0000 7f42c8d77740 1 --2- 172.21.15.112:0/782541841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd928ea0 0x55bafd9348f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.333+0000 7f42c8d77740 1 --2- 172.21.15.112:0/782541841 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55bafd939250 0x55bafd93b6c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.333+0000 7f42c8d77740 1 --2- 172.21.15.112:0/782541841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923d30 0x55bafd92d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.333+0000 7f42c8d77740 1 --2- 172.21.15.112:0/782541841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923340 0x55bafd92d370 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.333+0000 7f429effd640 0 client.15438 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.333+0000 7f42c8d77740 1 -- 172.21.15.112:0/782541841 >> 172.21.15.112:0/782541841 conn(0x55bafd91e8b0 msgr2=0x55bafd91dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.333+0000 7f42c8d77740 1 -- 172.21.15.112:0/782541841 shutdown_connections 2024-03-16T09:16:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.333+0000 7f42c8d77740 1 -- 172.21.15.112:0/782541841 wait complete. 2024-03-16T09:16:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.333+0000 7f42c8d77740 20 client.15438 trim_cache size 0 max 16384 2024-03-16T09:16:08.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:07 smithi112 ceph-mon[32307]: pgmap v365: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-16T09:16:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42c8d77740 1 Processor -- start 2024-03-16T09:16:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42c8d77740 1 -- start start 2024-03-16T09:16:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42c8d77740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923340 0x55bafd93b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42c8d77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923d30 0x55bafd9348b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42c8d77740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd928ea0 0x55bafd92d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42c8d77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f42c01ab540 con 0x55bafd923d30 2024-03-16T09:16:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42c8d77740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f42c01a8e10 con 0x55bafd923340 2024-03-16T09:16:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42c8d77740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f42c01a6c60 con 0x55bafd928ea0 2024-03-16T09:16:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42beffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923d30 0x55bafd9348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42bf7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923340 0x55bafd93b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42beffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923d30 0x55bafd9348b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56254/0 (socket says 172.21.15.112:56254) 2024-03-16T09:16:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42beffd640 1 -- 172.21.15.112:0/2312904911 learned_addr learned my addr 172.21.15.112:0/2312904911 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42bffff640 1 --2- 172.21.15.112:0/2312904911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd928ea0 0x55bafd92d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42beffd640 1 -- 172.21.15.112:0/2312904911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd928ea0 msgr2=0x55bafd92d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42beffd640 1 --2- 172.21.15.112:0/2312904911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd928ea0 0x55bafd92d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42beffd640 1 -- 172.21.15.112:0/2312904911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923340 msgr2=0x55bafd93b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42beffd640 1 --2- 172.21.15.112:0/2312904911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923340 0x55bafd93b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42beffd640 1 -- 172.21.15.112:0/2312904911 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bafd910140 con 0x55bafd923d30 2024-03-16T09:16:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42bf7fe640 1 --2- 172.21.15.112:0/2312904911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923340 0x55bafd93b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.348+0000 7f42beffd640 1 --2- 172.21.15.112:0/2312904911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923d30 0x55bafd9348b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009710 tx=0x7f42b4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb6d3297ae8fe41 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.349+0000 7f42a57fa640 1 -- 172.21.15.112:0/2312904911 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4013e20 con 0x55bafd923d30 2024-03-16T09:16:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.349+0000 7f42a57fa640 1 -- 172.21.15.112:0/2312904911 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f42b40130c0 con 0x55bafd923d30 2024-03-16T09:16:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.349+0000 7f42a57fa640 1 -- 172.21.15.112:0/2312904911 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4010430 con 0x55bafd923d30 2024-03-16T09:16:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.349+0000 7f42c8d77740 1 -- 172.21.15.112:0/2312904911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923d30 msgr2=0x55bafd9348b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.349+0000 7f42c8d77740 1 --2- 172.21.15.112:0/2312904911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923d30 0x55bafd9348b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009710 tx=0x7f42b4005f90 comp rx=0 tx=0).stop 2024-03-16T09:16:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.349+0000 7f42c8d77740 1 -- 172.21.15.112:0/2312904911 shutdown_connections 2024-03-16T09:16:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.349+0000 7f42c8d77740 1 --2- 172.21.15.112:0/2312904911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd928ea0 0x55bafd92d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.349+0000 7f42c8d77740 1 --2- 172.21.15.112:0/2312904911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923d30 0x55bafd9348b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.349+0000 7f42c8d77740 1 --2- 172.21.15.112:0/2312904911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923340 0x55bafd93b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.349+0000 7f42c8d77740 1 -- 172.21.15.112:0/2312904911 >> 172.21.15.112:0/2312904911 conn(0x55bafd91e8b0 msgr2=0x55bafd91d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:08.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.349+0000 7f42c8d77740 1 -- 172.21.15.112:0/2312904911 shutdown_connections 2024-03-16T09:16:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.349+0000 7f42c8d77740 1 -- 172.21.15.112:0/2312904911 wait complete. 2024-03-16T09:16:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.350+0000 7f42c8d77740 1 -- start start 2024-03-16T09:16:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.350+0000 7f42c8d77740 1 Processor -- start 2024-03-16T09:16:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.350+0000 7f42c8d77740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923340 0x55bafd931be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.350+0000 7f42c8d77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923d30 0x55bafd932120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.350+0000 7f42c8d77740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd928ea0 0x55bafd933130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.350+0000 7f42c8d77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f42c01a72d0 con 0x55bafd923d30 2024-03-16T09:16:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.350+0000 7f42c8d77740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f42c01a8740 con 0x55bafd923340 2024-03-16T09:16:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.350+0000 7f42c8d77740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f42c01a79a0 con 0x55bafd928ea0 2024-03-16T09:16:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.350+0000 7f42c8d77740 10 client.0 _ll_register_callbacks cb 0x7f42b0009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.350+0000 7f42beffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923d30 0x55bafd932120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.350+0000 7f42beffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923d30 0x55bafd932120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56262/0 (socket says 172.21.15.112:56262) 2024-03-16T09:16:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.350+0000 7f42beffd640 1 -- 172.21.15.112:0/1989807735 learned_addr learned my addr 172.21.15.112:0/1989807735 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.350+0000 7f42bf7fe640 1 --2- 172.21.15.112:0/1989807735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923340 0x55bafd931be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.350+0000 7f42bffff640 1 --2- 172.21.15.112:0/1989807735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd928ea0 0x55bafd933130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.350+0000 7f42beffd640 1 -- 172.21.15.112:0/1989807735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd928ea0 msgr2=0x55bafd933130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.350+0000 7f42beffd640 1 --2- 172.21.15.112:0/1989807735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd928ea0 0x55bafd933130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.350+0000 7f42beffd640 1 -- 172.21.15.112:0/1989807735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923340 msgr2=0x55bafd931be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.350+0000 7f42beffd640 1 --2- 172.21.15.112:0/1989807735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923340 0x55bafd931be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.350+0000 7f42beffd640 1 -- 172.21.15.112:0/1989807735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bafd933720 con 0x55bafd923d30 2024-03-16T09:16:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.351+0000 7f42beffd640 1 --2- 172.21.15.112:0/1989807735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923d30 0x55bafd932120 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f42b4013a40 tx=0x7f42b4013a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fb6f1a0689b6d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.351+0000 7f429effd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:16:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.351+0000 7f429effd640 1 -- 172.21.15.112:0/1989807735 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4013330 con 0x55bafd923d30 2024-03-16T09:16:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.351+0000 7f429effd640 1 -- 172.21.15.112:0/1989807735 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f42b4003a40 con 0x55bafd923d30 2024-03-16T09:16:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.351+0000 7f42c8d77740 1 -- 172.21.15.112:0/1989807735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42b40130c0 con 0x55bafd923d30 2024-03-16T09:16:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.351+0000 7f429effd640 1 -- 172.21.15.112:0/1989807735 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f42b4003c40 con 0x55bafd923d30 2024-03-16T09:16:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.351+0000 7f42c8d77740 10 client.15444 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.351+0000 7f42c8d77740 1 -- 172.21.15.112:0/1989807735 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55bafd934a60 con 0x55bafd923d30 2024-03-16T09:16:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.351+0000 7f429effd640 1 -- 172.21.15.112:0/1989807735 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f42b40130c0 con 0x55bafd923d30 2024-03-16T09:16:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.351+0000 7f429effd640 1 -- 172.21.15.112:0/1989807735 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f42b402a070 con 0x55bafd923d30 2024-03-16T09:16:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.351+0000 7f42c8d77740 10 client.15444 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.351+0000 7f42c8d77740 10 client.15444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.351+0000 7f42c8d77740 20 client.15444 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.351+0000 7f42c8d77740 1 --2- 172.21.15.112:0/1989807735 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55bafd99fc50 0x55bafd9a2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.351+0000 7f42c8d77740 4 client.15444 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T09:16:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.351+0000 7f42c8d77740 1 -- 172.21.15.112:0/1989807735 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bafd8902d0 con 0x55bafd99fc50 2024-03-16T09:16:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.352+0000 7f42bf7fe640 1 --2- 172.21.15.112:0/1989807735 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55bafd99fc50 0x55bafd9a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.352+0000 7f429effd640 10 client.15444 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.352+0000 7f42bf7fe640 1 --2- 172.21.15.112:0/1989807735 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55bafd99fc50 0x55bafd9a2070 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0a78f53ceca2cad server_cookie=989bf4d36b5f67b8 in_seq=0 out_seq=0 2024-03-16T09:16:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.357+0000 7f429effd640 1 -- 172.21.15.112:0/1989807735 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bafd8902d0 con 0x55bafd99fc50 2024-03-16T09:16:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.357+0000 7f429effd640 10 client.15444 handle_command_reply: tid=0 2024-03-16T09:16:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.357+0000 7f42c8d77740 1 client.15444 shutdown 2024-03-16T09:16:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.357+0000 7f42c8d77740 1 -- 172.21.15.112:0/1989807735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923d30 msgr2=0x55bafd932120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.357+0000 7f42c8d77740 1 --2- 172.21.15.112:0/1989807735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923d30 0x55bafd932120 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f42b4013a40 tx=0x7f42b4013a70 comp rx=0 tx=0).stop 2024-03-16T09:16:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.357+0000 7f42c8d77740 1 -- 172.21.15.112:0/1989807735 shutdown_connections 2024-03-16T09:16:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.357+0000 7f42c8d77740 1 --2- 172.21.15.112:0/1989807735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bafd928ea0 0x55bafd933130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.357+0000 7f42c8d77740 1 --2- 172.21.15.112:0/1989807735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55bafd923d30 0x55bafd932120 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.357+0000 7f42c8d77740 1 --2- 172.21.15.112:0/1989807735 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55bafd99fc50 0x55bafd9a2070 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.357+0000 7f42c8d77740 1 --2- 172.21.15.112:0/1989807735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bafd923340 0x55bafd931be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.357+0000 7f429effd640 0 client.15444 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.357+0000 7f42c8d77740 1 -- 172.21.15.112:0/1989807735 >> 172.21.15.112:0/1989807735 conn(0x55bafd91e8b0 msgr2=0x55bafd91dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.357+0000 7f42c8d77740 1 -- 172.21.15.112:0/1989807735 shutdown_connections 2024-03-16T09:16:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.357+0000 7f42c8d77740 1 -- 172.21.15.112:0/1989807735 wait complete. 2024-03-16T09:16:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.357+0000 7f42c8d77740 20 client.15444 trim_cache size 0 max 16384 2024-03-16T09:16:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-03-16T09:16:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-03-16T09:16:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-03-16T09:16:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-03-16T09:16:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.358+0000 7f42c6190640 1 -- 172.21.15.112:0/1907524650 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f42a80a2940 msgr2=0x7f42a80a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.358+0000 7f42c6190640 1 --2- 172.21.15.112:0/1907524650 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f42a80a2940 0x7f42a80a4e00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009fd0 tx=0x7f42b4009ca0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.358+0000 7f42c6190640 1 -- 172.21.15.112:0/1907524650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42c01033e0 msgr2=0x7f42c019f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.358+0000 7f42c6190640 1 --2- 172.21.15.112:0/1907524650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42c01033e0 0x7f42c019f050 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f42b802c630 tx=0x7f42b80346f0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.358+0000 7f42c6190640 1 -- 172.21.15.112:0/1907524650 shutdown_connections 2024-03-16T09:16:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.358+0000 7f42c6190640 1 --2- 172.21.15.112:0/1907524650 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f42a80a2940 0x7f42a80a4e00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.358+0000 7f42c6190640 1 --2- 172.21.15.112:0/1907524650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f42c01033e0 0x7f42c019f050 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.358+0000 7f42c6190640 1 --2- 172.21.15.112:0/1907524650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42c0102a20 0x7f42c0197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.358+0000 7f42c6190640 1 --2- 172.21.15.112:0/1907524650 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f42c0101830 0x7f42c0197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.358+0000 7f42c6190640 1 -- 172.21.15.112:0/1907524650 >> 172.21.15.112:0/1907524650 conn(0x7f42c00fcdc0 msgr2=0x7f42c0105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.358+0000 7f42c6190640 1 -- 172.21.15.112:0/1907524650 shutdown_connections 2024-03-16T09:16:08.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:08.358+0000 7f42c6190640 1 -- 172.21.15.112:0/1907524650 wait complete. 2024-03-16T09:16:08.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:07 smithi194 ceph-mon[36729]: pgmap v365: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-03-16T09:16:10.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:09 smithi159 ceph-mon[36739]: pgmap v366: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-03-16T09:16:10.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:09 smithi112 ceph-mon[32307]: pgmap v366: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-03-16T09:16:10.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:09 smithi194 ceph-mon[36729]: pgmap v366: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-03-16T09:16:12.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:11 smithi159 ceph-mon[36739]: pgmap v367: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:16:12.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:11 smithi112 ceph-mon[32307]: pgmap v367: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:16:12.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T09:16:12.388 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T09:16:12.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:11 smithi194 ceph-mon[36729]: pgmap v367: 97 pgs: 97 active+clean; 656 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:16:12.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.503+0000 7ff8e9f97640 1 -- 172.21.15.112:0/2494291065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e41018d0 msgr2=0x7ff8e4101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.503+0000 7ff8e9f97640 1 --2- 172.21.15.112:0/2494291065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e41018d0 0x7ff8e4101d50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009900 tx=0x7ff8d802bc30 comp rx=0 tx=0).stop 2024-03-16T09:16:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.503+0000 7ff8e9f97640 1 -- 172.21.15.112:0/2494291065 shutdown_connections 2024-03-16T09:16:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.503+0000 7ff8e9f97640 1 --2- 172.21.15.112:0/2494291065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8e4102290 0x7ff8e4108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.503+0000 7ff8e9f97640 1 --2- 172.21.15.112:0/2494291065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e41018d0 0x7ff8e4101d50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.503+0000 7ff8e9f97640 1 --2- 172.21.15.112:0/2494291065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8e41006e0 0x7ff8e4100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.503+0000 7ff8e9f97640 1 -- 172.21.15.112:0/2494291065 >> 172.21.15.112:0/2494291065 conn(0x7ff8e40fbc70 msgr2=0x7ff8e40fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.503+0000 7ff8e9f97640 1 -- 172.21.15.112:0/2494291065 shutdown_connections 2024-03-16T09:16:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.503+0000 7ff8e9f97640 1 -- 172.21.15.112:0/2494291065 wait complete. 2024-03-16T09:16:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.504+0000 7ff8e9f97640 1 Processor -- start 2024-03-16T09:16:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.504+0000 7ff8e9f97640 1 -- start start 2024-03-16T09:16:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.504+0000 7ff8e9f97640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8e41006e0 0x7ff8e4197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.504+0000 7ff8e9f97640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e41018d0 0x7ff8e41980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.504+0000 7ff8e9f97640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8e4102290 0x7ff8e419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.504+0000 7ff8e9f97640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e410c440 con 0x7ff8e41018d0 2024-03-16T09:16:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.504+0000 7ff8e9f97640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e410c2c0 con 0x7ff8e4102290 2024-03-16T09:16:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.504+0000 7ff8e9f97640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e410c5c0 con 0x7ff8e41006e0 2024-03-16T09:16:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.505+0000 7ff8e2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e41018d0 0x7ff8e41980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.505+0000 7ff8e2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e41018d0 0x7ff8e41980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40300/0 (socket says 172.21.15.112:40300) 2024-03-16T09:16:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.505+0000 7ff8e2ffd640 1 -- 172.21.15.112:0/2708974057 learned_addr learned my addr 172.21.15.112:0/2708974057 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.505+0000 7ff8e37fe640 1 --2- 172.21.15.112:0/2708974057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8e41006e0 0x7ff8e4197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.505+0000 7ff8e3fff640 1 --2- 172.21.15.112:0/2708974057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8e4102290 0x7ff8e419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.505+0000 7ff8e2ffd640 1 -- 172.21.15.112:0/2708974057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8e41006e0 msgr2=0x7ff8e4197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.505+0000 7ff8e2ffd640 1 --2- 172.21.15.112:0/2708974057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8e41006e0 0x7ff8e4197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.505+0000 7ff8e2ffd640 1 -- 172.21.15.112:0/2708974057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8e4102290 msgr2=0x7ff8e419f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.505+0000 7ff8e2ffd640 1 --2- 172.21.15.112:0/2708974057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8e4102290 0x7ff8e419f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.505+0000 7ff8e2ffd640 1 -- 172.21.15.112:0/2708974057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e419f7d0 con 0x7ff8e41018d0 2024-03-16T09:16:12.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.505+0000 7ff8e2ffd640 1 --2- 172.21.15.112:0/2708974057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e41018d0 0x7ff8e41980f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009230 tx=0x7ff8d8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.505+0000 7ff8e0ff9640 1 -- 172.21.15.112:0/2708974057 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8043020 con 0x7ff8e41018d0 2024-03-16T09:16:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.505+0000 7ff8e0ff9640 1 -- 172.21.15.112:0/2708974057 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff8d8004250 con 0x7ff8e41018d0 2024-03-16T09:16:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.505+0000 7ff8e0ff9640 1 -- 172.21.15.112:0/2708974057 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8003b50 con 0x7ff8e41018d0 2024-03-16T09:16:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.505+0000 7ff8e9f97640 1 -- 172.21.15.112:0/2708974057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e419fa60 con 0x7ff8e41018d0 2024-03-16T09:16:12.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.505+0000 7ff8e9f97640 1 -- 172.21.15.112:0/2708974057 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e419ffb0 con 0x7ff8e41018d0 2024-03-16T09:16:12.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.507+0000 7ff8e0ff9640 1 -- 172.21.15.112:0/2708974057 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7ff8d8046020 con 0x7ff8e41018d0 2024-03-16T09:16:12.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8ecb85740 1 Processor -- start 2024-03-16T09:16:12.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8ecb85740 1 -- start start 2024-03-16T09:16:12.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8ecb85740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e6170 0x55683d9e6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8ecb85740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6b60 0x55683d9eb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8e0ff9640 1 --2- 172.21.15.112:0/2708974057 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff8cc0a2890 0x7ff8cc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8ecb85740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9ebcd0 0x55683d9ee160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8ecb85740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55683d94b500 con 0x55683d9e6170 2024-03-16T09:16:12.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8ecb85740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55683d8f3be0 con 0x55683d9ebcd0 2024-03-16T09:16:12.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8ecb85740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55683d9475a0 con 0x55683d9e6b60 2024-03-16T09:16:12.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e6170 0x55683d9e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8e37fe640 1 --2- 172.21.15.112:0/2708974057 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff8cc0a2890 0x7ff8cc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e6170 0x55683d9e6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40314/0 (socket says 172.21.15.112:40314) 2024-03-16T09:16:12.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8e3fff640 1 -- 172.21.15.112:0/4154006736 learned_addr learned my addr 172.21.15.112:0/4154006736 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:12.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8e2ffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6b60 0x55683d9eb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8e0ff9640 1 -- 172.21.15.112:0/2708974057 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff8d80f0be0 con 0x7ff8e41018d0 2024-03-16T09:16:12.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8e3fff640 1 --2- 172.21.15.112:0/4154006736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9ebcd0 0x55683d9ee160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8e3fff640 1 -- 172.21.15.112:0/4154006736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6b60 msgr2=0x55683d9eb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:12.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8e3fff640 1 --2- 172.21.15.112:0/4154006736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6b60 0x55683d9eb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8e3fff640 1 -- 172.21.15.112:0/4154006736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9ebcd0 msgr2=0x55683d9ee160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:12.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8e3fff640 1 --2- 172.21.15.112:0/4154006736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9ebcd0 0x55683d9ee160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.508+0000 7ff8e3fff640 1 -- 172.21.15.112:0/4154006736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55683d9074b0 con 0x55683d9e6170 2024-03-16T09:16:12.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.509+0000 7ff8e37fe640 1 --2- 172.21.15.112:0/2708974057 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff8cc0a2890 0x7ff8cc0a4d50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4101730 tx=0x7ff8d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:12.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.509+0000 7ff8e3fff640 1 --2- 172.21.15.112:0/4154006736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e6170 0x55683d9e6590 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc013960 tx=0x7ff8dc009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72183e07e2c1c397 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:12.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.509+0000 7ff8c9ffb640 1 -- 172.21.15.112:0/4154006736 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc004b20 con 0x55683d9e6170 2024-03-16T09:16:12.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.509+0000 7ff8c9ffb640 1 -- 172.21.15.112:0/4154006736 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff8dc004cc0 con 0x55683d9e6170 2024-03-16T09:16:12.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.509+0000 7ff8c9ffb640 1 -- 172.21.15.112:0/4154006736 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc004290 con 0x55683d9e6170 2024-03-16T09:16:12.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.509+0000 7ff8ecb85740 1 -- 172.21.15.112:0/4154006736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e6170 msgr2=0x55683d9e6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:12.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.509+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/4154006736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e6170 0x55683d9e6590 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc013960 tx=0x7ff8dc009be0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.510+0000 7ff8ecb85740 1 -- 172.21.15.112:0/4154006736 shutdown_connections 2024-03-16T09:16:12.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.510+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/4154006736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9ebcd0 0x55683d9ee160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.510+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/4154006736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6b60 0x55683d9eb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.510+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/4154006736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e6170 0x55683d9e6590 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.510+0000 7ff8ecb85740 1 -- 172.21.15.112:0/4154006736 >> 172.21.15.112:0/4154006736 conn(0x55683d9e16e0 msgr2=0x55683d9e1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:12.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.510+0000 7ff8ecb85740 1 -- 172.21.15.112:0/4154006736 shutdown_connections 2024-03-16T09:16:12.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.510+0000 7ff8ecb85740 1 -- 172.21.15.112:0/4154006736 wait complete. 2024-03-16T09:16:12.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.510+0000 7ff8ecb85740 1 -- start start 2024-03-16T09:16:12.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.510+0000 7ff8ecb85740 1 Processor -- start 2024-03-16T09:16:12.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.510+0000 7ff8ecb85740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6170 0x55683d9f0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.510+0000 7ff8ecb85740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9e6b60 0x55683d9f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.510+0000 7ff8ecb85740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9ebcd0 0x55683d9f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.510+0000 7ff8ecb85740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55683d9eec20 con 0x55683d9ebcd0 2024-03-16T09:16:12.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.510+0000 7ff8ecb85740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55683d907160 con 0x55683d9e6b60 2024-03-16T09:16:12.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.510+0000 7ff8ecb85740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55683d9eeda0 con 0x55683d9e6170 2024-03-16T09:16:12.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8ecb85740 10 client.0 _ll_register_callbacks cb 0x55683d8eb150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:12.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9ebcd0 0x55683d9f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8e37fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6170 0x55683d9f0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8e3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9ebcd0 0x55683d9f7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40328/0 (socket says 172.21.15.112:40328) 2024-03-16T09:16:12.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8e3fff640 1 -- 172.21.15.112:0/2337223427 learned_addr learned my addr 172.21.15.112:0/2337223427 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:12.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8e2ffd640 1 --2- 172.21.15.112:0/2337223427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9e6b60 0x55683d9f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8e37fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6170 0x55683d9f0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:36436/0 (socket says 172.21.15.112:36436) 2024-03-16T09:16:12.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8e3fff640 1 -- 172.21.15.112:0/2337223427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6170 msgr2=0x55683d9f0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:12.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8e3fff640 1 --2- 172.21.15.112:0/2337223427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6170 0x55683d9f0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8e3fff640 1 -- 172.21.15.112:0/2337223427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9e6b60 msgr2=0x55683d9f0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:12.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8e3fff640 1 --2- 172.21.15.112:0/2337223427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9e6b60 0x55683d9f0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8e3fff640 1 -- 172.21.15.112:0/2337223427 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55683d7297d0 con 0x55683d9ebcd0 2024-03-16T09:16:12.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8e2ffd640 1 --2- 172.21.15.112:0/2337223427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9e6b60 0x55683d9f0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:12.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8e3fff640 1 --2- 172.21.15.112:0/2337223427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9ebcd0 0x55683d9f7b90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc006010 tx=0x7ff8dc003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e214deda8815f4b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:12.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:16:12.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8c2ffd640 1 -- 172.21.15.112:0/2337223427 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc00f8c0 con 0x55683d9ebcd0 2024-03-16T09:16:12.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8c2ffd640 1 -- 172.21.15.112:0/2337223427 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff8dc004040 con 0x55683d9ebcd0 2024-03-16T09:16:12.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8ecb85740 1 -- 172.21.15.112:0/2337223427 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8dc004cc0 con 0x55683d9ebcd0 2024-03-16T09:16:12.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8c2ffd640 1 -- 172.21.15.112:0/2337223427 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff8dc004280 con 0x55683d9ebcd0 2024-03-16T09:16:12.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8ecb85740 10 client.15456 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:12.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.511+0000 7ff8ecb85740 1 -- 172.21.15.112:0/2337223427 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55683d65f3f0 con 0x55683d9ebcd0 2024-03-16T09:16:12.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.512+0000 7ff8c2ffd640 1 -- 172.21.15.112:0/2337223427 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff8dc004cc0 con 0x55683d9ebcd0 2024-03-16T09:16:12.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.512+0000 7ff8c2ffd640 1 -- 172.21.15.112:0/2337223427 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ff8dc02aad0 con 0x55683d9ebcd0 2024-03-16T09:16:12.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.512+0000 7ff8ecb85740 10 client.15456 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:12.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.512+0000 7ff8ecb85740 10 client.15456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:12.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.512+0000 7ff8ecb85740 20 client.15456 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:12.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.512+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/2337223427 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55683d9fbf70 0x55683d9fe3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.512+0000 7ff8ecb85740 4 client.15456 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:16:12.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.512+0000 7ff8ecb85740 1 -- 172.21.15.112:0/2337223427 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55683d958a40 con 0x55683d9fbf70 2024-03-16T09:16:12.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.512+0000 7ff8e37fe640 1 --2- 172.21.15.112:0/2337223427 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55683d9fbf70 0x55683d9fe3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.512+0000 7ff8c2ffd640 10 client.15456 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:12.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.512+0000 7ff8e37fe640 1 --2- 172.21.15.112:0/2337223427 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55683d9fbf70 0x55683d9fe3b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be46e24f464ee9d9 server_cookie=fdb3f2e5eb53bedb in_seq=0 out_seq=0 2024-03-16T09:16:12.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.513+0000 7ff8c2ffd640 1 -- 172.21.15.112:0/2337223427 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55683d958a40 con 0x55683d9fbf70 2024-03-16T09:16:12.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.513+0000 7ff8c2ffd640 10 client.15456 handle_command_reply: tid=0 2024-03-16T09:16:12.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.513+0000 7ff8ecb85740 1 client.15456 shutdown 2024-03-16T09:16:12.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.513+0000 7ff8ecb85740 1 -- 172.21.15.112:0/2337223427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9ebcd0 msgr2=0x55683d9f7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:12.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.513+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/2337223427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9ebcd0 0x55683d9f7b90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc006010 tx=0x7ff8dc003ec0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.513+0000 7ff8ecb85740 1 -- 172.21.15.112:0/2337223427 shutdown_connections 2024-03-16T09:16:12.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.513+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/2337223427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9ebcd0 0x55683d9f7b90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.513+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/2337223427 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55683d9fbf70 0x55683d9fe3b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.513+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/2337223427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9e6b60 0x55683d9f0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.513+0000 7ff8c2ffd640 0 client.15456 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:12.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.513+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/2337223427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6170 0x55683d9f0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.513+0000 7ff8ecb85740 1 -- 172.21.15.112:0/2337223427 >> 172.21.15.112:0/2337223427 conn(0x55683d9e16e0 msgr2=0x55683d9e0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:12.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.513+0000 7ff8ecb85740 1 -- 172.21.15.112:0/2337223427 shutdown_connections 2024-03-16T09:16:12.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.513+0000 7ff8ecb85740 1 -- 172.21.15.112:0/2337223427 wait complete. 2024-03-16T09:16:12.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.514+0000 7ff8ecb85740 20 client.15456 trim_cache size 0 max 16384 2024-03-16T09:16:12.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.531+0000 7ff8ecb85740 1 Processor -- start 2024-03-16T09:16:12.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.531+0000 7ff8ecb85740 1 -- start start 2024-03-16T09:16:12.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.531+0000 7ff8ecb85740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e16e0 0x55683d9e1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.531+0000 7ff8ecb85740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6170 0x55683d9f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.531+0000 7ff8ecb85740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9e6b60 0x55683d9e0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.531+0000 7ff8ecb85740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e41ab600 con 0x55683d9e16e0 2024-03-16T09:16:12.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.531+0000 7ff8ecb85740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e41a8ed0 con 0x55683d9e6b60 2024-03-16T09:16:12.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.531+0000 7ff8ecb85740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e41a6d20 con 0x55683d9e6170 2024-03-16T09:16:12.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.531+0000 7ff8e37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e16e0 0x55683d9e1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.531+0000 7ff8e37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e16e0 0x55683d9e1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40338/0 (socket says 172.21.15.112:40338) 2024-03-16T09:16:12.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.531+0000 7ff8e37fe640 1 -- 172.21.15.112:0/619454929 learned_addr learned my addr 172.21.15.112:0/619454929 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.531+0000 7ff8e3fff640 1 --2- 172.21.15.112:0/619454929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9e6b60 0x55683d9e0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.531+0000 7ff8e2ffd640 1 --2- 172.21.15.112:0/619454929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6170 0x55683d9f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.531+0000 7ff8e37fe640 1 -- 172.21.15.112:0/619454929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6170 msgr2=0x55683d9f7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.531+0000 7ff8e37fe640 1 --2- 172.21.15.112:0/619454929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6170 0x55683d9f7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.531+0000 7ff8e37fe640 1 -- 172.21.15.112:0/619454929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9e6b60 msgr2=0x55683d9e0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.531+0000 7ff8e37fe640 1 --2- 172.21.15.112:0/619454929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9e6b60 0x55683d9e0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.531+0000 7ff8e37fe640 1 -- 172.21.15.112:0/619454929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55683d61f0f0 con 0x55683d9e16e0 2024-03-16T09:16:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.532+0000 7ff8e37fe640 1 --2- 172.21.15.112:0/619454929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e16e0 0x55683d9e1ac0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55683d9e2650 tx=0x7ff8d4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddba9e4416f2413b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.532+0000 7ff8c97fa640 1 -- 172.21.15.112:0/619454929 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4004220 con 0x55683d9e16e0 2024-03-16T09:16:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.532+0000 7ff8c97fa640 1 -- 172.21.15.112:0/619454929 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff8d40043c0 con 0x55683d9e16e0 2024-03-16T09:16:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.532+0000 7ff8c97fa640 1 -- 172.21.15.112:0/619454929 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4003bd0 con 0x55683d9e16e0 2024-03-16T09:16:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.532+0000 7ff8ecb85740 1 -- 172.21.15.112:0/619454929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e16e0 msgr2=0x55683d9e1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.532+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/619454929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e16e0 0x55683d9e1ac0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55683d9e2650 tx=0x7ff8d4014000 comp rx=0 tx=0).stop 2024-03-16T09:16:12.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.532+0000 7ff8ecb85740 1 -- 172.21.15.112:0/619454929 shutdown_connections 2024-03-16T09:16:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.532+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/619454929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9e6b60 0x55683d9e0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.532+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/619454929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6170 0x55683d9f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.532+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/619454929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e16e0 0x55683d9e1ac0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.532+0000 7ff8ecb85740 1 -- 172.21.15.112:0/619454929 >> 172.21.15.112:0/619454929 conn(0x55683d9ef820 msgr2=0x55683d9f0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.532+0000 7ff8ecb85740 1 -- 172.21.15.112:0/619454929 shutdown_connections 2024-03-16T09:16:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.532+0000 7ff8ecb85740 1 -- 172.21.15.112:0/619454929 wait complete. 2024-03-16T09:16:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.533+0000 7ff8ecb85740 1 -- start start 2024-03-16T09:16:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.533+0000 7ff8ecb85740 1 Processor -- start 2024-03-16T09:16:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.533+0000 7ff8ecb85740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e16e0 0x55683d9f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.533+0000 7ff8ecb85740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9e6170 0x55683d9f5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.533+0000 7ff8ecb85740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6b60 0x55683d9f6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.533+0000 7ff8ecb85740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e41a7390 con 0x55683d9e16e0 2024-03-16T09:16:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.533+0000 7ff8ecb85740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e41a8800 con 0x55683d9e6170 2024-03-16T09:16:12.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.533+0000 7ff8ecb85740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e41a7a60 con 0x55683d9e6b60 2024-03-16T09:16:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.533+0000 7ff8ecb85740 10 client.0 _ll_register_callbacks cb 0x7ff8d80057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.533+0000 7ff8e37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e16e0 0x55683d9f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.533+0000 7ff8e37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e16e0 0x55683d9f4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40340/0 (socket says 172.21.15.112:40340) 2024-03-16T09:16:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.533+0000 7ff8e37fe640 1 -- 172.21.15.112:0/932694728 learned_addr learned my addr 172.21.15.112:0/932694728 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.533+0000 7ff8e2ffd640 1 --2- 172.21.15.112:0/932694728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9e6170 0x55683d9f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.533+0000 7ff8e3fff640 1 --2- 172.21.15.112:0/932694728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6b60 0x55683d9f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.533+0000 7ff8e37fe640 1 -- 172.21.15.112:0/932694728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6b60 msgr2=0x55683d9f6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.533+0000 7ff8e37fe640 1 --2- 172.21.15.112:0/932694728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6b60 0x55683d9f6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.533+0000 7ff8e37fe640 1 -- 172.21.15.112:0/932694728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9e6170 msgr2=0x55683d9f5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:12.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.533+0000 7ff8e37fe640 1 --2- 172.21.15.112:0/932694728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9e6170 0x55683d9f5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.533+0000 7ff8e37fe640 1 -- 172.21.15.112:0/932694728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55683d9129d0 con 0x55683d9e16e0 2024-03-16T09:16:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.534+0000 7ff8e37fe640 1 --2- 172.21.15.112:0/932694728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e16e0 0x55683d9f4c00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff8d401d680 tx=0x7ff8d40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14dbf2c794d09b37 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.534+0000 7ff8c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:16:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.534+0000 7ff8c2ffd640 1 -- 172.21.15.112:0/932694728 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4029520 con 0x55683d9e16e0 2024-03-16T09:16:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.534+0000 7ff8c2ffd640 1 -- 172.21.15.112:0/932694728 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7ff8d40038d0 con 0x55683d9e16e0 2024-03-16T09:16:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.534+0000 7ff8ecb85740 1 -- 172.21.15.112:0/932694728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8d40043c0 con 0x55683d9e16e0 2024-03-16T09:16:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.534+0000 7ff8c2ffd640 1 -- 172.21.15.112:0/932694728 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff8d4003ad0 con 0x55683d9e16e0 2024-03-16T09:16:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.534+0000 7ff8ecb85740 10 client.15462 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.534+0000 7ff8ecb85740 1 -- 172.21.15.112:0/932694728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55683d9f7880 con 0x55683d9e16e0 2024-03-16T09:16:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.534+0000 7ff8c2ffd640 1 -- 172.21.15.112:0/932694728 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff8d40043c0 con 0x55683d9e16e0 2024-03-16T09:16:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.534+0000 7ff8c2ffd640 1 -- 172.21.15.112:0/932694728 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ff8d4032bd0 con 0x55683d9e16e0 2024-03-16T09:16:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.534+0000 7ff8ecb85740 10 client.15462 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.534+0000 7ff8ecb85740 10 client.15462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:12.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.534+0000 7ff8ecb85740 20 client.15462 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.534+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/932694728 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55683d9ebcd0 0x55683da64380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.534+0000 7ff8ecb85740 4 client.15462 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T09:16:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.534+0000 7ff8ecb85740 1 -- 172.21.15.112:0/932694728 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55683d952d90 con 0x55683d9ebcd0 2024-03-16T09:16:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.534+0000 7ff8e2ffd640 1 --2- 172.21.15.112:0/932694728 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55683d9ebcd0 0x55683da64380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.535+0000 7ff8c2ffd640 10 client.15462 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.535+0000 7ff8e2ffd640 1 --2- 172.21.15.112:0/932694728 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55683d9ebcd0 0x55683da64380 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=846e39122f130896 server_cookie=2acaa7ad9262efd8 in_seq=0 out_seq=0 2024-03-16T09:16:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.535+0000 7ff8c2ffd640 1 -- 172.21.15.112:0/932694728 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55683d952d90 con 0x55683d9ebcd0 2024-03-16T09:16:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.535+0000 7ff8c2ffd640 10 client.15462 handle_command_reply: tid=0 2024-03-16T09:16:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.535+0000 7ff8ecb85740 1 client.15462 shutdown 2024-03-16T09:16:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.535+0000 7ff8ecb85740 1 -- 172.21.15.112:0/932694728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e16e0 msgr2=0x55683d9f4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.535+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/932694728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e16e0 0x55683d9f4c00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff8d401d680 tx=0x7ff8d40056d0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.535+0000 7ff8ecb85740 1 -- 172.21.15.112:0/932694728 shutdown_connections 2024-03-16T09:16:12.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.535+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/932694728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55683d9e6b60 0x55683d9f6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.535+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/932694728 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55683d9ebcd0 0x55683da64380 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.535+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/932694728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55683d9e6170 0x55683d9f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.535+0000 7ff8c2ffd640 0 client.15462 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:12.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.535+0000 7ff8ecb85740 1 --2- 172.21.15.112:0/932694728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55683d9e16e0 0x55683d9f4c00 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.535+0000 7ff8ecb85740 1 -- 172.21.15.112:0/932694728 >> 172.21.15.112:0/932694728 conn(0x55683d9ef820 msgr2=0x55683d9e4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:12.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.535+0000 7ff8ecb85740 1 -- 172.21.15.112:0/932694728 shutdown_connections 2024-03-16T09:16:12.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.535+0000 7ff8ecb85740 1 -- 172.21.15.112:0/932694728 wait complete. 2024-03-16T09:16:12.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.535+0000 7ff8ecb85740 20 client.15462 trim_cache size 0 max 16384 2024-03-16T09:16:12.545 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:16:12.545 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-03-16T09:16:12.546 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-03-16T09:16:12.546 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:16:12.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.536+0000 7ff8e9f97640 1 -- 172.21.15.112:0/2708974057 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff8cc0a2890 msgr2=0x7ff8cc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:12.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.536+0000 7ff8e9f97640 1 --2- 172.21.15.112:0/2708974057 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff8cc0a2890 0x7ff8cc0a4d50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4101730 tx=0x7ff8d4005f90 comp rx=0 tx=0).stop 2024-03-16T09:16:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.536+0000 7ff8e9f97640 1 -- 172.21.15.112:0/2708974057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e41018d0 msgr2=0x7ff8e41980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.536+0000 7ff8e9f97640 1 --2- 172.21.15.112:0/2708974057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e41018d0 0x7ff8e41980f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009230 tx=0x7ff8d8004ab0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.536+0000 7ff8e9f97640 1 -- 172.21.15.112:0/2708974057 shutdown_connections 2024-03-16T09:16:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.536+0000 7ff8e9f97640 1 --2- 172.21.15.112:0/2708974057 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff8cc0a2890 0x7ff8cc0a4d50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.536+0000 7ff8e9f97640 1 --2- 172.21.15.112:0/2708974057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8e4102290 0x7ff8e419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.536+0000 7ff8e9f97640 1 --2- 172.21.15.112:0/2708974057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8e41018d0 0x7ff8e41980f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.536+0000 7ff8e9f97640 1 --2- 172.21.15.112:0/2708974057 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8e41006e0 0x7ff8e4197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.536+0000 7ff8e9f97640 1 -- 172.21.15.112:0/2708974057 >> 172.21.15.112:0/2708974057 conn(0x7ff8e40fbc70 msgr2=0x7ff8e40fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.536+0000 7ff8e9f97640 1 -- 172.21.15.112:0/2708974057 shutdown_connections 2024-03-16T09:16:12.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.536+0000 7ff8e9f97640 1 -- 172.21.15.112:0/2708974057 wait complete. 2024-03-16T09:16:12.549 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T09:16:12.549 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:16:12.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.649+0000 7f8ec8ec0640 1 -- 172.21.15.112:0/1690844090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec41075a0 msgr2=0x7f8ec4109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:12.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.649+0000 7f8ec8ec0640 1 --2- 172.21.15.112:0/1690844090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec41075a0 0x7f8ec4109990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc009ca0 tx=0x7f8ebc02c0f0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.649+0000 7f8ec8ec0640 1 -- 172.21.15.112:0/1690844090 shutdown_connections 2024-03-16T09:16:12.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.649+0000 7f8ec8ec0640 1 --2- 172.21.15.112:0/1690844090 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec41075a0 0x7f8ec4109990 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.649+0000 7f8ec8ec0640 1 --2- 172.21.15.112:0/1690844090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ec4104c70 0x7f8ec4107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.649+0000 7f8ec8ec0640 1 --2- 172.21.15.112:0/1690844090 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ec4102340 0x7f8ec4104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.649+0000 7f8ec8ec0640 1 -- 172.21.15.112:0/1690844090 >> 172.21.15.112:0/1690844090 conn(0x7f8ec40fbc70 msgr2=0x7f8ec40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:12.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.649+0000 7f8ec8ec0640 1 -- 172.21.15.112:0/1690844090 shutdown_connections 2024-03-16T09:16:12.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.649+0000 7f8ec8ec0640 1 -- 172.21.15.112:0/1690844090 wait complete. 2024-03-16T09:16:12.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.649+0000 7f8ec8ec0640 1 Processor -- start 2024-03-16T09:16:12.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec8ec0640 1 -- start start 2024-03-16T09:16:12.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec8ec0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ec4102340 0x7f8ec4197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec8ec0640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ec4104c70 0x7f8ec4198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec8ec0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec41075a0 0x7f8ec419f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec8ec0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec410d5e0 con 0x7f8ec41075a0 2024-03-16T09:16:12.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec8ec0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec410d460 con 0x7f8ec4102340 2024-03-16T09:16:12.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec8ec0640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec410d760 con 0x7f8ec4104c70 2024-03-16T09:16:12.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec41075a0 0x7f8ec419f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec41075a0 0x7f8ec419f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40354/0 (socket says 172.21.15.112:40354) 2024-03-16T09:16:12.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec2d9d640 1 -- 172.21.15.112:0/2301969179 learned_addr learned my addr 172.21.15.112:0/2301969179 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:12.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ec4102340 0x7f8ec4197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec1d9b640 1 --2- 172.21.15.112:0/2301969179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ec4104c70 0x7f8ec4198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec2d9d640 1 -- 172.21.15.112:0/2301969179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ec4104c70 msgr2=0x7f8ec4198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:12.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec2d9d640 1 --2- 172.21.15.112:0/2301969179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ec4104c70 0x7f8ec4198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec2d9d640 1 -- 172.21.15.112:0/2301969179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ec4102340 msgr2=0x7f8ec4197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:12.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec2d9d640 1 --2- 172.21.15.112:0/2301969179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ec4102340 0x7f8ec4197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec2d9d640 1 -- 172.21.15.112:0/2301969179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec419f7f0 con 0x7f8ec41075a0 2024-03-16T09:16:12.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec259c640 1 --2- 172.21.15.112:0/2301969179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ec4102340 0x7f8ec4197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:12.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec2d9d640 1 --2- 172.21.15.112:0/2301969179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec41075a0 0x7f8ec419f180 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc02c700 tx=0x7f8ebc0346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:12.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8eb37fe640 1 -- 172.21.15.112:0/2301969179 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc039040 con 0x7f8ec41075a0 2024-03-16T09:16:12.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8eb37fe640 1 -- 172.21.15.112:0/2301969179 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8ebc044070 con 0x7f8ec41075a0 2024-03-16T09:16:12.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8eb37fe640 1 -- 172.21.15.112:0/2301969179 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc0043c0 con 0x7f8ec41075a0 2024-03-16T09:16:12.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.650+0000 7f8ec8ec0640 1 -- 172.21.15.112:0/2301969179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec419fa80 con 0x7f8ec41075a0 2024-03-16T09:16:12.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.651+0000 7f8ec8ec0640 1 -- 172.21.15.112:0/2301969179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec419ff70 con 0x7f8ec41075a0 2024-03-16T09:16:12.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.651+0000 7f8ec8ec0640 1 -- 172.21.15.112:0/2301969179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec40fd760 con 0x7f8ec41075a0 2024-03-16T09:16:12.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.652+0000 7f8eb37fe640 1 -- 172.21.15.112:0/2301969179 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7f8ebc004580 con 0x7f8ec41075a0 2024-03-16T09:16:12.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.653+0000 7f8eb37fe640 1 --2- 172.21.15.112:0/2301969179 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8eac0a2890 0x7f8eac0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.653+0000 7f8ec259c640 1 --2- 172.21.15.112:0/2301969179 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8eac0a2890 0x7f8eac0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.653+0000 7f8eb37fe640 1 -- 172.21.15.112:0/2301969179 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8ebc0f10d0 con 0x7f8ec41075a0 2024-03-16T09:16:12.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.653+0000 7f8ec259c640 1 --2- 172.21.15.112:0/2301969179 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8eac0a2890 0x7f8eac0a4d50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8009fc0 tx=0x7f8eb8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:12.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.658+0000 7f8eb37fe640 1 -- 172.21.15.112:0/2301969179 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8ebc0be130 con 0x7f8ec41075a0 2024-03-16T09:16:12.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.873+0000 7f8ec8ec0640 1 -- 172.21.15.112:0/2301969179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ec402d360 con 0x7f8ec41075a0 2024-03-16T09:16:12.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.874+0000 7f8eb37fe640 1 -- 172.21.15.112:0/2301969179 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11881 (secure 0 0 0) 0x7f8ebc0bda30 con 0x7f8ec41075a0 2024-03-16T09:16:12.876 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:16:12.876 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3939748854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3939748854},{"type":"v1","addr":"172.21.15.112:6839","nonce":3939748854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3118523258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3118523258},{"type":"v1","addr":"172.21.15.112:6841","nonce":3118523258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T09:13:36.450216+0000","modified":"2024-03-16T09:15:34.982470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/1682076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":1682076},{"type":"v1","addr":"172.21.15.194:6837","nonce":1682076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.194:6839/2528943309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2528943309},{"type":"v1","addr":"172.21.15.194:6839","nonce":2528943309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/272895965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":272895965},{"type":"v1","addr":"172.21.15.159:6839","nonce":272895965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24452,24466]},"id":1}]} 2024-03-16T09:16:12.877 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 22 2024-03-16T09:16:12.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.876+0000 7f8ec8ec0640 1 -- 172.21.15.112:0/2301969179 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8eac0a2890 msgr2=0x7f8eac0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:12.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.876+0000 7f8ec8ec0640 1 --2- 172.21.15.112:0/2301969179 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8eac0a2890 0x7f8eac0a4d50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8009fc0 tx=0x7f8eb8005f90 comp rx=0 tx=0).stop 2024-03-16T09:16:12.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.876+0000 7f8ec8ec0640 1 -- 172.21.15.112:0/2301969179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec41075a0 msgr2=0x7f8ec419f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:12.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.876+0000 7f8ec8ec0640 1 --2- 172.21.15.112:0/2301969179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec41075a0 0x7f8ec419f180 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc02c700 tx=0x7f8ebc0346d0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.876+0000 7f8ec8ec0640 1 -- 172.21.15.112:0/2301969179 shutdown_connections 2024-03-16T09:16:12.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.876+0000 7f8ec8ec0640 1 --2- 172.21.15.112:0/2301969179 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8eac0a2890 0x7f8eac0a4d50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.876+0000 7f8ec8ec0640 1 --2- 172.21.15.112:0/2301969179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ec41075a0 0x7f8ec419f180 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.876+0000 7f8ec8ec0640 1 --2- 172.21.15.112:0/2301969179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ec4104c70 0x7f8ec4198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.876+0000 7f8ec8ec0640 1 --2- 172.21.15.112:0/2301969179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ec4102340 0x7f8ec4197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.876+0000 7f8ec8ec0640 1 -- 172.21.15.112:0/2301969179 >> 172.21.15.112:0/2301969179 conn(0x7f8ec40fbc70 msgr2=0x7f8ec4102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:12.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.876+0000 7f8ec8ec0640 1 -- 172.21.15.112:0/2301969179 shutdown_connections 2024-03-16T09:16:12.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.876+0000 7f8ec8ec0640 1 -- 172.21.15.112:0/2301969179 wait complete. 2024-03-16T09:16:12.890 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T09:16:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.992+0000 7fa6a8af8640 1 -- 172.21.15.112:0/4096514493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a4100240 msgr2=0x7fa6a4109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.992+0000 7fa6a8af8640 1 --2- 172.21.15.112:0/4096514493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a4100240 0x7fa6a4109df0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009ca0 tx=0x7fa69c02c0f0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.992+0000 7fa6a8af8640 1 -- 172.21.15.112:0/4096514493 shutdown_connections 2024-03-16T09:16:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.992+0000 7fa6a8af8640 1 --2- 172.21.15.112:0/4096514493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a4100240 0x7fa6a4109df0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.992+0000 7fa6a8af8640 1 --2- 172.21.15.112:0/4096514493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6a40ff880 0x7fa6a40ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.992+0000 7fa6a8af8640 1 --2- 172.21.15.112:0/4096514493 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6a4102e10 0x7fa6a4103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.992+0000 7fa6a8af8640 1 -- 172.21.15.112:0/4096514493 >> 172.21.15.112:0/4096514493 conn(0x7fa6a40fb3e0 msgr2=0x7fa6a40fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.992+0000 7fa6a8af8640 1 -- 172.21.15.112:0/4096514493 shutdown_connections 2024-03-16T09:16:12.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.992+0000 7fa6a8af8640 1 -- 172.21.15.112:0/4096514493 wait complete. 2024-03-16T09:16:12.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa6a8af8640 1 Processor -- start 2024-03-16T09:16:12.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa6a8af8640 1 -- start start 2024-03-16T09:16:12.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa6a8af8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a40ff880 0x7fa6a4197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa6a8af8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6a4100240 0x7fa6a41980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa6a8af8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6a4102e10 0x7fa6a419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:12.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa6a8af8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a410d610 con 0x7fa6a40ff880 2024-03-16T09:16:12.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa6a8af8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a410d490 con 0x7fa6a4102e10 2024-03-16T09:16:12.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa6a8af8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a410d790 con 0x7fa6a4100240 2024-03-16T09:16:12.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa6a259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a40ff880 0x7fa6a4197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa6a259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a40ff880 0x7fa6a4197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40374/0 (socket says 172.21.15.112:40374) 2024-03-16T09:16:12.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa6a259c640 1 -- 172.21.15.112:0/724704653 learned_addr learned my addr 172.21.15.112:0/724704653 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:12.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa69bfff640 1 --2- 172.21.15.112:0/724704653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6a4100240 0x7fa6a41980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa6a2d9d640 1 --2- 172.21.15.112:0/724704653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6a4102e10 0x7fa6a419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:12.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa6a259c640 1 -- 172.21.15.112:0/724704653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6a4100240 msgr2=0x7fa6a41980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:12.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa6a259c640 1 --2- 172.21.15.112:0/724704653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6a4100240 0x7fa6a41980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa6a259c640 1 -- 172.21.15.112:0/724704653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6a4102e10 msgr2=0x7fa6a419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:12.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa6a259c640 1 --2- 172.21.15.112:0/724704653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6a4102e10 0x7fa6a419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:12.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa6a259c640 1 -- 172.21.15.112:0/724704653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a419f7a0 con 0x7fa6a40ff880 2024-03-16T09:16:12.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa69bfff640 1 --2- 172.21.15.112:0/724704653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6a4100240 0x7fa6a41980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:12.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa6a259c640 1 --2- 172.21.15.112:0/724704653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a40ff880 0x7fa6a4197b80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa69400e9c0 tx=0x7fa69400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:12.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.993+0000 7fa69b7fe640 1 -- 172.21.15.112:0/724704653 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69400cd80 con 0x7fa6a40ff880 2024-03-16T09:16:12.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.994+0000 7fa69b7fe640 1 -- 172.21.15.112:0/724704653 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa6940035f0 con 0x7fa6a40ff880 2024-03-16T09:16:12.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.994+0000 7fa6a8af8640 1 -- 172.21.15.112:0/724704653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a419fa90 con 0x7fa6a40ff880 2024-03-16T09:16:12.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.994+0000 7fa69b7fe640 1 -- 172.21.15.112:0/724704653 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694010670 con 0x7fa6a40ff880 2024-03-16T09:16:12.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.994+0000 7fa6a8af8640 1 -- 172.21.15.112:0/724704653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a419ff90 con 0x7fa6a40ff880 2024-03-16T09:16:13.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.996+0000 7fa69b7fe640 1 -- 172.21.15.112:0/724704653 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7fa6940037a0 con 0x7fa6a40ff880 2024-03-16T09:16:13.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.996+0000 7fa6ab7c3740 1 Processor -- start 2024-03-16T09:16:13.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.996+0000 7fa6ab7c3740 1 -- start start 2024-03-16T09:16:13.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.996+0000 7fa6ab7c3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc7850 0x561c7fbc7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.996+0000 7fa6ab7c3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbc8240 0x561c7fbccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.996+0000 7fa6ab7c3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 0x561c7fbcf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.996+0000 7fa6ab7c3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561c7fb2c490 con 0x561c7fbcd3b0 2024-03-16T09:16:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.996+0000 7fa6ab7c3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561c7fad4c90 con 0x561c7fbc7850 2024-03-16T09:16:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.996+0000 7fa6ab7c3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561c7fb28530 con 0x561c7fbc8240 2024-03-16T09:16:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.996+0000 7fa69b7fe640 1 --2- 172.21.15.112:0/724704653 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa6900a2890 0x7fa6900a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.996+0000 7fa69bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 0x561c7fbcf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.996+0000 7fa69bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 0x561c7fbcf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40380/0 (socket says 172.21.15.112:40380) 2024-03-16T09:16:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.996+0000 7fa69bfff640 1 -- 172.21.15.112:0/2483039964 learned_addr learned my addr 172.21.15.112:0/2483039964 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.996+0000 7fa6a259c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbc8240 0x561c7fbccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.996+0000 7fa6a2d9d640 1 --2- 172.21.15.112:0/724704653 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa6900a2890 0x7fa6900a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.997+0000 7fa6a2d9d640 1 --2- 172.21.15.112:0/2483039964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc7850 0x561c7fbc7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.997+0000 7fa69b7fe640 1 -- 172.21.15.112:0/724704653 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa6940cef30 con 0x7fa6a40ff880 2024-03-16T09:16:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.997+0000 7fa69bfff640 1 -- 172.21.15.112:0/2483039964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbc8240 msgr2=0x561c7fbccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.997+0000 7fa69bfff640 1 --2- 172.21.15.112:0/2483039964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbc8240 0x561c7fbccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.997+0000 7fa69bfff640 1 -- 172.21.15.112:0/2483039964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc7850 msgr2=0x561c7fbc7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.997+0000 7fa69bfff640 1 --2- 172.21.15.112:0/2483039964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc7850 0x561c7fbc7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.997+0000 7fa69bfff640 1 -- 172.21.15.112:0/2483039964 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c7fb8f500 con 0x561c7fbcd3b0 2024-03-16T09:16:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.997+0000 7fa6a2d9d640 1 --2- 172.21.15.112:0/724704653 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa6900a2890 0x7fa6900a4d50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa69c005e30 tx=0x7fa69c032450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.997+0000 7fa69bfff640 1 --2- 172.21.15.112:0/2483039964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 0x561c7fbcf840 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0114f0 tx=0x7fa68c0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c581dbbe8ee8c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.997+0000 7fa698ff9640 1 -- 172.21.15.112:0/2483039964 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c011250 con 0x561c7fbcd3b0 2024-03-16T09:16:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.997+0000 7fa698ff9640 1 -- 172.21.15.112:0/2483039964 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa68c0035f0 con 0x561c7fbcd3b0 2024-03-16T09:16:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.997+0000 7fa698ff9640 1 -- 172.21.15.112:0/2483039964 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c00f7e0 con 0x561c7fbcd3b0 2024-03-16T09:16:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.998+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/2483039964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 msgr2=0x561c7fbcf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.998+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/2483039964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 0x561c7fbcf840 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa68c0114f0 tx=0x7fa68c0119b0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.998+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/2483039964 shutdown_connections 2024-03-16T09:16:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.998+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/2483039964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 0x561c7fbcf840 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.998+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/2483039964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbc8240 0x561c7fbccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.998+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/2483039964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc7850 0x561c7fbc7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.998+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/2483039964 >> 172.21.15.112:0/2483039964 conn(0x561c7fbc2dc0 msgr2=0x561c7fbc31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.998+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/2483039964 shutdown_connections 2024-03-16T09:16:13.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.998+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/2483039964 wait complete. 2024-03-16T09:16:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.998+0000 7fa6ab7c3740 1 -- start start 2024-03-16T09:16:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.998+0000 7fa6ab7c3740 1 Processor -- start 2024-03-16T09:16:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6ab7c3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbc7850 0x561c7fbd1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6ab7c3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc8240 0x561c7fbd2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6ab7c3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 0x561c7fbd9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6ab7c3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561c7fbd0360 con 0x561c7fbcd3b0 2024-03-16T09:16:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6ab7c3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561c7fae8230 con 0x561c7fbc8240 2024-03-16T09:16:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6ab7c3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561c7fbd04e0 con 0x561c7fbc7850 2024-03-16T09:16:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6ab7c3740 10 client.0 _ll_register_callbacks cb 0x561c7fb24010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6a2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 0x561c7fbd9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6a259c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbc7850 0x561c7fbd1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa69bfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc8240 0x561c7fbd2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6a2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 0x561c7fbd9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40390/0 (socket says 172.21.15.112:40390) 2024-03-16T09:16:13.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6a259c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbc7850 0x561c7fbd1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:36524/0 (socket says 172.21.15.112:36524) 2024-03-16T09:16:13.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6a2d9d640 1 -- 172.21.15.112:0/2097063155 learned_addr learned my addr 172.21.15.112:0/2097063155 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:13.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6a2d9d640 1 -- 172.21.15.112:0/2097063155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbc7850 msgr2=0x561c7fbd1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6a2d9d640 1 --2- 172.21.15.112:0/2097063155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbc7850 0x561c7fbd1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6a2d9d640 1 -- 172.21.15.112:0/2097063155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc8240 msgr2=0x561c7fbd2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6a2d9d640 1 --2- 172.21.15.112:0/2097063155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc8240 0x561c7fbd2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6a2d9d640 1 -- 172.21.15.112:0/2097063155 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c7faf7100 con 0x561c7fbcd3b0 2024-03-16T09:16:13.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6a259c640 1 --2- 172.21.15.112:0/2097063155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbc7850 0x561c7fbd1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:13.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa69bfff640 1 --2- 172.21.15.112:0/2097063155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc8240 0x561c7fbd2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:13.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6a2d9d640 1 --2- 172.21.15.112:0/2097063155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 0x561c7fbd9290 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa69c003d90 tx=0x7fa69c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156908adefcde3e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:13.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa689ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:16:13.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa689ffb640 1 -- 172.21.15.112:0/2097063155 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c051060 con 0x561c7fbcd3b0 2024-03-16T09:16:13.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa689ffb640 1 -- 172.21.15.112:0/2097063155 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa69c004b40 con 0x561c7fbcd3b0 2024-03-16T09:16:13.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/2097063155 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa68c0035f0 con 0x561c7fbcd3b0 2024-03-16T09:16:13.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa689ffb640 1 -- 172.21.15.112:0/2097063155 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa69c039040 con 0x561c7fbcd3b0 2024-03-16T09:16:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6ab7c3740 10 client.15480 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:12.999+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/2097063155 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561c7f8403f0 con 0x561c7fbcd3b0 2024-03-16T09:16:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.000+0000 7fa689ffb640 1 -- 172.21.15.112:0/2097063155 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa68c0035f0 con 0x561c7fbcd3b0 2024-03-16T09:16:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.000+0000 7fa689ffb640 1 -- 172.21.15.112:0/2097063155 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa69c03f030 con 0x561c7fbcd3b0 2024-03-16T09:16:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.000+0000 7fa6ab7c3740 10 client.15480 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.000+0000 7fa6ab7c3740 10 client.15480 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T09:16:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.000+0000 7fa6ab7c3740 20 client.15480 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.000+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/2097063155 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x561c7fbddbc0 0x561c7fbe0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.000+0000 7fa6ab7c3740 4 client.15480 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:16:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.000+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/2097063155 --> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c7fb399d0 con 0x561c7fbddbc0 2024-03-16T09:16:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.000+0000 7fa6a259c640 1 --2- 172.21.15.112:0/2097063155 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x561c7fbddbc0 0x561c7fbe0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.000+0000 7fa6a259c640 1 --2- 172.21.15.112:0/2097063155 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x561c7fbddbc0 0x561c7fbe0060 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f9e24f510a0b18d server_cookie=3fb1ae3cd5865944 in_seq=0 out_seq=0 2024-03-16T09:16:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.000+0000 7fa689ffb640 10 client.15480 ms_handle_connect on v2:172.21.15.159:6836/4023942203 2024-03-16T09:16:13.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.001+0000 7fa689ffb640 1 -- 172.21.15.112:0/2097063155 <== mds.2 v2:172.21.15.159:6836/4023942203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561c7fb399d0 con 0x561c7fbddbc0 2024-03-16T09:16:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.001+0000 7fa689ffb640 10 client.15480 handle_command_reply: tid=0 2024-03-16T09:16:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.001+0000 7fa6ab7c3740 1 client.15480 shutdown 2024-03-16T09:16:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.001+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/2097063155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 msgr2=0x561c7fbd9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.001+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/2097063155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 0x561c7fbd9290 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa69c003d90 tx=0x7fa69c032670 comp rx=0 tx=0).stop 2024-03-16T09:16:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.002+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/2097063155 shutdown_connections 2024-03-16T09:16:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.002+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/2097063155 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x561c7fbddbc0 0x561c7fbe0060 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.002+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/2097063155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 0x561c7fbd9290 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.002+0000 7fa689ffb640 0 client.15480 ms_handle_reset on v2:172.21.15.159:6836/4023942203 2024-03-16T09:16:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.002+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/2097063155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc8240 0x561c7fbd2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.002+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/2097063155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbc7850 0x561c7fbd1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.002+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/2097063155 >> 172.21.15.112:0/2097063155 conn(0x561c7fbc2dc0 msgr2=0x561c7fbc1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.002+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/2097063155 shutdown_connections 2024-03-16T09:16:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.002+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/2097063155 wait complete. 2024-03-16T09:16:13.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.002+0000 7fa6ab7c3740 20 client.15480 trim_cache size 0 max 16384 2024-03-16T09:16:13.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa6ab7c3740 1 Processor -- start 2024-03-16T09:16:13.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa6ab7c3740 1 -- start start 2024-03-16T09:16:13.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa6ab7c3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbc7850 0x561c7fbc9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa6ab7c3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc8240 0x561c7fbc9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa6ab7c3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 0x561c7fbc1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa6ab7c3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a41ab600 con 0x561c7fbcd3b0 2024-03-16T09:16:13.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa6ab7c3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a41a8ed0 con 0x561c7fbc8240 2024-03-16T09:16:13.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa6ab7c3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a41a6d20 con 0x561c7fbc7850 2024-03-16T09:16:13.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa6a2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 0x561c7fbc1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa6a259c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbc7850 0x561c7fbc9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa6a2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 0x561c7fbc1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40404/0 (socket says 172.21.15.112:40404) 2024-03-16T09:16:13.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa6a2d9d640 1 -- 172.21.15.112:0/2976191715 learned_addr learned my addr 172.21.15.112:0/2976191715 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:13.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa69bfff640 1 --2- 172.21.15.112:0/2976191715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc8240 0x561c7fbc9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa6a2d9d640 1 -- 172.21.15.112:0/2976191715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbc7850 msgr2=0x561c7fbc9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa6a2d9d640 1 --2- 172.21.15.112:0/2976191715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbc7850 0x561c7fbc9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa6a2d9d640 1 -- 172.21.15.112:0/2976191715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc8240 msgr2=0x561c7fbc9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa6a2d9d640 1 --2- 172.21.15.112:0/2976191715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc8240 0x561c7fbc9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa6a2d9d640 1 -- 172.21.15.112:0/2976191715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c7f90a7d0 con 0x561c7fbcd3b0 2024-03-16T09:16:13.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa6a2d9d640 1 --2- 172.21.15.112:0/2976191715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 0x561c7fbc1750 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa69c02c630 tx=0x7fa69c0095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3a80bd44caee4e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:13.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa68bfff640 1 -- 172.21.15.112:0/2976191715 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c05c030 con 0x561c7fbcd3b0 2024-03-16T09:16:13.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa68bfff640 1 -- 172.21.15.112:0/2976191715 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa69c004410 con 0x561c7fbcd3b0 2024-03-16T09:16:13.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.015+0000 7fa68bfff640 1 -- 172.21.15.112:0/2976191715 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c051070 con 0x561c7fbcd3b0 2024-03-16T09:16:13.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.016+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/2976191715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 msgr2=0x561c7fbc1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.016+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/2976191715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 0x561c7fbc1750 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa69c02c630 tx=0x7fa69c0095e0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.016+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/2976191715 shutdown_connections 2024-03-16T09:16:13.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.016+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/2976191715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbcd3b0 0x561c7fbc1750 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.016+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/2976191715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc8240 0x561c7fbc9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.016+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/2976191715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbc7850 0x561c7fbc9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.016+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/2976191715 >> 172.21.15.112:0/2976191715 conn(0x561c7fbc2dc0 msgr2=0x561c7fbc31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:13.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.016+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/2976191715 shutdown_connections 2024-03-16T09:16:13.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.016+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/2976191715 wait complete. 2024-03-16T09:16:13.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.016+0000 7fa6ab7c3740 1 -- start start 2024-03-16T09:16:13.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.016+0000 7fa6ab7c3740 1 Processor -- start 2024-03-16T09:16:13.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6ab7c3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbc7850 0x561c7fc3ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6ab7c3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc8240 0x561c7fc3f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6ab7c3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbcd3b0 0x561c7fc46160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6ab7c3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a41a7390 con 0x561c7fbc7850 2024-03-16T09:16:13.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6ab7c3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a41a8800 con 0x561c7fbc8240 2024-03-16T09:16:13.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6ab7c3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a41a7a60 con 0x561c7fbcd3b0 2024-03-16T09:16:13.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6ab7c3740 10 client.0 _ll_register_callbacks cb 0x7fa694005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:13.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6a259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbc7850 0x561c7fc3ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6a259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbc7850 0x561c7fc3ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40418/0 (socket says 172.21.15.112:40418) 2024-03-16T09:16:13.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6a259c640 1 -- 172.21.15.112:0/3563748343 learned_addr learned my addr 172.21.15.112:0/3563748343 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:13.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6a2d9d640 1 --2- 172.21.15.112:0/3563748343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbcd3b0 0x561c7fc46160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa69bfff640 1 --2- 172.21.15.112:0/3563748343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc8240 0x561c7fc3f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6a259c640 1 -- 172.21.15.112:0/3563748343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbcd3b0 msgr2=0x561c7fc46160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6a259c640 1 --2- 172.21.15.112:0/3563748343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbcd3b0 0x561c7fc46160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6a259c640 1 -- 172.21.15.112:0/3563748343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc8240 msgr2=0x561c7fc3f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6a259c640 1 --2- 172.21.15.112:0/3563748343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc8240 0x561c7fc3f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6a259c640 1 -- 172.21.15.112:0/3563748343 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c7fc46750 con 0x561c7fbc7850 2024-03-16T09:16:13.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6a259c640 1 --2- 172.21.15.112:0/3563748343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbc7850 0x561c7fc3ebe0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa694098150 tx=0x7fa69406b2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1c21107b0fcd63 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:13.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa689ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:16:13.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa689ffb640 1 -- 172.21.15.112:0/3563748343 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6940860b0 con 0x561c7fbc7850 2024-03-16T09:16:13.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa689ffb640 1 -- 172.21.15.112:0/3563748343 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa694003ba0 con 0x561c7fbc7850 2024-03-16T09:16:13.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/3563748343 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa69c004410 con 0x561c7fbc7850 2024-03-16T09:16:13.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa689ffb640 1 -- 172.21.15.112:0/3563748343 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa69406a210 con 0x561c7fbc7850 2024-03-16T09:16:13.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6ab7c3740 10 client.15486 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:13.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.017+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/3563748343 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x561c7fc47b20 con 0x561c7fbc7850 2024-03-16T09:16:13.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa689ffb640 1 -- 172.21.15.112:0/3563748343 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa69c004410 con 0x561c7fbc7850 2024-03-16T09:16:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa689ffb640 1 -- 172.21.15.112:0/3563748343 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa694097040 con 0x561c7fbc7850 2024-03-16T09:16:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa6ab7c3740 10 client.15486 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa6ab7c3740 10 client.15486 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T09:16:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa6ab7c3740 20 client.15486 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/3563748343 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x561c7fc48450 0x561c7fc4a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa6ab7c3740 4 client.15486 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T09:16:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/3563748343 --> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c7fb33d20 con 0x561c7fc48450 2024-03-16T09:16:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa69bfff640 1 --2- 172.21.15.112:0/3563748343 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x561c7fc48450 0x561c7fc4a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa689ffb640 10 client.15486 ms_handle_connect on v2:172.21.15.159:6836/4023942203 2024-03-16T09:16:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa69bfff640 1 --2- 172.21.15.112:0/3563748343 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x561c7fc48450 0x561c7fc4a870 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20f051b87f49adeb server_cookie=1d210345d03823c4 in_seq=0 out_seq=0 2024-03-16T09:16:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa689ffb640 1 -- 172.21.15.112:0/3563748343 <== mds.2 v2:172.21.15.159:6836/4023942203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c7fb33d20 con 0x561c7fc48450 2024-03-16T09:16:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa689ffb640 10 client.15486 handle_command_reply: tid=0 2024-03-16T09:16:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa6ab7c3740 1 client.15486 shutdown 2024-03-16T09:16:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/3563748343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbc7850 msgr2=0x561c7fc3ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/3563748343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbc7850 0x561c7fc3ebe0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa694098150 tx=0x7fa69406b2a0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/3563748343 shutdown_connections 2024-03-16T09:16:13.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/3563748343 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x561c7fc48450 0x561c7fc4a870 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/3563748343 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561c7fbcd3b0 0x561c7fc46160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa689ffb640 0 client.15486 ms_handle_reset on v2:172.21.15.159:6836/4023942203 2024-03-16T09:16:13.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/3563748343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c7fbc8240 0x561c7fc3f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa6ab7c3740 1 --2- 172.21.15.112:0/3563748343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561c7fbc7850 0x561c7fc3ebe0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.018+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/3563748343 >> 172.21.15.112:0/3563748343 conn(0x561c7fbc2dc0 msgr2=0x561c7fbd1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:13.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.019+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/3563748343 shutdown_connections 2024-03-16T09:16:13.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.019+0000 7fa6ab7c3740 1 -- 172.21.15.112:0/3563748343 wait complete. 2024-03-16T09:16:13.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.019+0000 7fa6ab7c3740 20 client.15486 trim_cache size 0 max 16384 2024-03-16T09:16:13.029 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-03-16T09:16:13.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.019+0000 7fa6a8af8640 1 -- 172.21.15.112:0/724704653 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa6900a2890 msgr2=0x7fa6900a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.019+0000 7fa6a8af8640 1 --2- 172.21.15.112:0/724704653 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa6900a2890 0x7fa6900a4d50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa69c005e30 tx=0x7fa69c032450 comp rx=0 tx=0).stop 2024-03-16T09:16:13.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.019+0000 7fa6a8af8640 1 -- 172.21.15.112:0/724704653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a40ff880 msgr2=0x7fa6a4197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.019+0000 7fa6a8af8640 1 --2- 172.21.15.112:0/724704653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a40ff880 0x7fa6a4197b80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa69400e9c0 tx=0x7fa69400ee80 comp rx=0 tx=0).stop 2024-03-16T09:16:13.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.019+0000 7fa6a8af8640 1 -- 172.21.15.112:0/724704653 shutdown_connections 2024-03-16T09:16:13.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.019+0000 7fa6a8af8640 1 --2- 172.21.15.112:0/724704653 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa6900a2890 0x7fa6900a4d50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.019+0000 7fa6a8af8640 1 --2- 172.21.15.112:0/724704653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6a4102e10 0x7fa6a419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.019+0000 7fa6a8af8640 1 --2- 172.21.15.112:0/724704653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa6a4100240 0x7fa6a41980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.019+0000 7fa6a8af8640 1 --2- 172.21.15.112:0/724704653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6a40ff880 0x7fa6a4197b80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.019+0000 7fa6a8af8640 1 -- 172.21.15.112:0/724704653 >> 172.21.15.112:0/724704653 conn(0x7fa6a40fb3e0 msgr2=0x7fa6a4105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:13.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.019+0000 7fa6a8af8640 1 -- 172.21.15.112:0/724704653 shutdown_connections 2024-03-16T09:16:13.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.019+0000 7fa6a8af8640 1 -- 172.21.15.112:0/724704653 wait complete. 2024-03-16T09:16:13.032 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T09:16:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.134+0000 7f90770c6640 1 -- 172.21.15.112:0/126444984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90701006f0 msgr2=0x7f9070100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.134+0000 7f90770c6640 1 --2- 172.21.15.112:0/126444984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90701006f0 0x7f9070100b10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9064009900 tx=0x7f906402bc30 comp rx=0 tx=0).stop 2024-03-16T09:16:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.134+0000 7f90770c6640 1 -- 172.21.15.112:0/126444984 shutdown_connections 2024-03-16T09:16:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.134+0000 7f90770c6640 1 --2- 172.21.15.112:0/126444984 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90701022a0 0x7f9070108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.134+0000 7f90770c6640 1 --2- 172.21.15.112:0/126444984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90701018e0 0x7f9070101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.134+0000 7f90770c6640 1 --2- 172.21.15.112:0/126444984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90701006f0 0x7f9070100b10 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.134+0000 7f90770c6640 1 -- 172.21.15.112:0/126444984 >> 172.21.15.112:0/126444984 conn(0x7f90700fbc80 msgr2=0x7f90700fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:13.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.134+0000 7f90770c6640 1 -- 172.21.15.112:0/126444984 shutdown_connections 2024-03-16T09:16:13.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.134+0000 7f90770c6640 1 -- 172.21.15.112:0/126444984 wait complete. 2024-03-16T09:16:13.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.135+0000 7f90770c6640 1 Processor -- start 2024-03-16T09:16:13.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.135+0000 7f90770c6640 1 -- start start 2024-03-16T09:16:13.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.135+0000 7f90770c6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90701006f0 0x7f9070197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.135+0000 7f90770c6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90701018e0 0x7f9070198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.135+0000 7f90770c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90701022a0 0x7f907019f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.135+0000 7f90770c6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f907010c450 con 0x7f90701022a0 2024-03-16T09:16:13.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.135+0000 7f90770c6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f907010c2d0 con 0x7f90701006f0 2024-03-16T09:16:13.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.135+0000 7f90770c6640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f907010c5d0 con 0x7f90701018e0 2024-03-16T09:16:13.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.135+0000 7f9075663640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90701022a0 0x7f907019f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.135+0000 7f9074e62640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90701006f0 0x7f9070197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.135+0000 7f9075663640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90701022a0 0x7f907019f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40426/0 (socket says 172.21.15.112:40426) 2024-03-16T09:16:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.135+0000 7f9075663640 1 -- 172.21.15.112:0/314541405 learned_addr learned my addr 172.21.15.112:0/314541405 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.135+0000 7f906ffff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90701018e0 0x7f9070198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.135+0000 7f9075663640 1 -- 172.21.15.112:0/314541405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90701018e0 msgr2=0x7f9070198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.135+0000 7f9075663640 1 --2- 172.21.15.112:0/314541405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90701018e0 0x7f9070198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.135+0000 7f9075663640 1 -- 172.21.15.112:0/314541405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90701006f0 msgr2=0x7f9070197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.135+0000 7f9075663640 1 --2- 172.21.15.112:0/314541405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90701006f0 0x7f9070197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.135+0000 7f9075663640 1 -- 172.21.15.112:0/314541405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907019f850 con 0x7f90701022a0 2024-03-16T09:16:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.135+0000 7f9075663640 1 --2- 172.21.15.112:0/314541405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90701022a0 0x7f907019f1b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f906800eb10 tx=0x7f906800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.136+0000 7f906dffb640 1 -- 172.21.15.112:0/314541405 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068004b20 con 0x7f90701022a0 2024-03-16T09:16:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.136+0000 7f906dffb640 1 -- 172.21.15.112:0/314541405 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9068004cc0 con 0x7f90701022a0 2024-03-16T09:16:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.136+0000 7f90770c6640 1 -- 172.21.15.112:0/314541405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907019fb40 con 0x7f90701022a0 2024-03-16T09:16:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.136+0000 7f906dffb640 1 -- 172.21.15.112:0/314541405 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90680107f0 con 0x7f90701022a0 2024-03-16T09:16:13.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.136+0000 7f90770c6640 1 -- 172.21.15.112:0/314541405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90701a0060 con 0x7f90701022a0 2024-03-16T09:16:13.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.138+0000 7f906dffb640 1 -- 172.21.15.112:0/314541405 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7f9068005020 con 0x7f90701022a0 2024-03-16T09:16:13.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.138+0000 7f9079d8f740 1 Processor -- start 2024-03-16T09:16:13.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.138+0000 7f906dffb640 1 --2- 172.21.15.112:0/314541405 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f90600a2890 0x7f90600a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.138+0000 7f9079d8f740 1 -- start start 2024-03-16T09:16:13.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.138+0000 7f9079d8f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69655850 0x556b69655c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.138+0000 7f9079d8f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b69656240 0x556b6965ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.138+0000 7f9079d8f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b6965b3b0 0x556b6965d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.138+0000 7f906ffff640 1 --2- 172.21.15.112:0/314541405 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f90600a2890 0x7f90600a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.138+0000 7f9079d8f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556b695ba490 con 0x556b6965b3b0 2024-03-16T09:16:13.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.138+0000 7f9079d8f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556b69562c90 con 0x556b69655850 2024-03-16T09:16:13.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.138+0000 7f9079d8f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556b695b6530 con 0x556b69656240 2024-03-16T09:16:13.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.138+0000 7f906dffb640 1 -- 172.21.15.112:0/314541405 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f90680cf1e0 con 0x7f90701022a0 2024-03-16T09:16:13.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.139+0000 7f906ffff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b69656240 0x556b6965ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.139+0000 7f9074e62640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69655850 0x556b69655c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.139+0000 7f9075663640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b6965b3b0 0x556b6965d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.139+0000 7f906ffff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b69656240 0x556b6965ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:36552/0 (socket says 172.21.15.112:36552) 2024-03-16T09:16:13.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.139+0000 7f9074e62640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69655850 0x556b69655c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:41914/0 (socket says 172.21.15.112:41914) 2024-03-16T09:16:13.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.139+0000 7f906ffff640 1 -- 172.21.15.112:0/593474392 learned_addr learned my addr 172.21.15.112:0/593474392 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:13.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.139+0000 7f9075663640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b6965b3b0 0x556b6965d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40432/0 (socket says 172.21.15.112:40432) 2024-03-16T09:16:13.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.139+0000 7f9074e62640 1 -- 172.21.15.112:0/593474392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b69656240 msgr2=0x556b6965ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.139+0000 7f9074e62640 1 --2- 172.21.15.112:0/593474392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b69656240 0x556b6965ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.139+0000 7f9074e62640 1 -- 172.21.15.112:0/593474392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b6965b3b0 msgr2=0x556b6965d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.139+0000 7f9074e62640 1 --2- 172.21.15.112:0/593474392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b6965b3b0 0x556b6965d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.139+0000 7f906ffff640 1 --2- 172.21.15.112:0/314541405 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f90600a2890 0x7f90600a4d50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f905c013570 tx=0x7f905c013970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:13.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.139+0000 7f9074e62640 1 -- 172.21.15.112:0/593474392 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b6961d500 con 0x556b69655850 2024-03-16T09:16:13.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.139+0000 7f9075663640 1 --2- 172.21.15.112:0/593474392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b6965b3b0 0x556b6965d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:13.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.139+0000 7f9074e62640 1 --2- 172.21.15.112:0/593474392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69655850 0x556b69655c70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f906402c170 tx=0x7f9064009670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c006b22c6f1f5a2d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:13.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.139+0000 7f905affd640 1 -- 172.21.15.112:0/593474392 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906402c5d0 con 0x556b69655850 2024-03-16T09:16:13.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.139+0000 7f905affd640 1 -- 172.21.15.112:0/593474392 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9064004040 con 0x556b69655850 2024-03-16T09:16:13.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.139+0000 7f905affd640 1 -- 172.21.15.112:0/593474392 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90640046c0 con 0x556b69655850 2024-03-16T09:16:13.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.140+0000 7f9079d8f740 1 -- 172.21.15.112:0/593474392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69655850 msgr2=0x556b69655c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.140+0000 7f9079d8f740 1 --2- 172.21.15.112:0/593474392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69655850 0x556b69655c70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f906402c170 tx=0x7f9064009670 comp rx=0 tx=0).stop 2024-03-16T09:16:13.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.140+0000 7f9079d8f740 1 -- 172.21.15.112:0/593474392 shutdown_connections 2024-03-16T09:16:13.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.140+0000 7f9079d8f740 1 --2- 172.21.15.112:0/593474392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b6965b3b0 0x556b6965d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.140+0000 7f9079d8f740 1 --2- 172.21.15.112:0/593474392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b69656240 0x556b6965ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.140+0000 7f9079d8f740 1 --2- 172.21.15.112:0/593474392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69655850 0x556b69655c70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.140+0000 7f9079d8f740 1 -- 172.21.15.112:0/593474392 >> 172.21.15.112:0/593474392 conn(0x556b69650dc0 msgr2=0x556b696511c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:13.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.140+0000 7f9079d8f740 1 -- 172.21.15.112:0/593474392 shutdown_connections 2024-03-16T09:16:13.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.140+0000 7f9079d8f740 1 -- 172.21.15.112:0/593474392 wait complete. 2024-03-16T09:16:13.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.140+0000 7f9079d8f740 1 -- start start 2024-03-16T09:16:13.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.140+0000 7f9079d8f740 1 Processor -- start 2024-03-16T09:16:13.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f9079d8f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 0x556b6965fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f9079d8f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69656240 0x556b69660200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f9079d8f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b6965b3b0 0x556b69667230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f9079d8f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556b6965e300 con 0x556b69655850 2024-03-16T09:16:13.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f9079d8f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556b69576230 con 0x556b69656240 2024-03-16T09:16:13.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f9079d8f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556b6965e480 con 0x556b6965b3b0 2024-03-16T09:16:13.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f9079d8f740 10 client.0 _ll_register_callbacks cb 0x556b695b2010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:13.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f9074e62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 0x556b6965fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f9074e62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 0x556b6965fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40448/0 (socket says 172.21.15.112:40448) 2024-03-16T09:16:13.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f9074e62640 1 -- 172.21.15.112:0/1343483445 learned_addr learned my addr 172.21.15.112:0/1343483445 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:13.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f906ffff640 1 --2- 172.21.15.112:0/1343483445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69656240 0x556b69660200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f9075663640 1 --2- 172.21.15.112:0/1343483445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b6965b3b0 0x556b69667230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f9074e62640 1 -- 172.21.15.112:0/1343483445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b6965b3b0 msgr2=0x556b69667230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f9074e62640 1 --2- 172.21.15.112:0/1343483445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b6965b3b0 0x556b69667230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f9074e62640 1 -- 172.21.15.112:0/1343483445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69656240 msgr2=0x556b69660200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f9074e62640 1 --2- 172.21.15.112:0/1343483445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69656240 0x556b69660200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f9074e62640 1 -- 172.21.15.112:0/1343483445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b69585100 con 0x556b69655850 2024-03-16T09:16:13.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f9074e62640 1 --2- 172.21.15.112:0/1343483445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 0x556b6965fcc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9064009900 tx=0x7f906402c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2aafc79c4b9d5d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:13.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f904ffff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:16:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f904ffff640 1 -- 172.21.15.112:0/1343483445 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906402ccd0 con 0x556b69655850 2024-03-16T09:16:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f904ffff640 1 -- 172.21.15.112:0/1343483445 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f90640035f0 con 0x556b69655850 2024-03-16T09:16:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.141+0000 7f9079d8f740 1 -- 172.21.15.112:0/1343483445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9064004040 con 0x556b69655850 2024-03-16T09:16:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.142+0000 7f904ffff640 1 -- 172.21.15.112:0/1343483445 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f90640037f0 con 0x556b69655850 2024-03-16T09:16:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.142+0000 7f9079d8f740 10 client.15498 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.142+0000 7f9079d8f740 1 -- 172.21.15.112:0/1343483445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x556b692ce3f0 con 0x556b69655850 2024-03-16T09:16:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.142+0000 7f904ffff640 1 -- 172.21.15.112:0/1343483445 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9064004040 con 0x556b69655850 2024-03-16T09:16:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.142+0000 7f904ffff640 1 -- 172.21.15.112:0/1343483445 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f9064004cc0 con 0x556b69655850 2024-03-16T09:16:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.142+0000 7f9079d8f740 10 client.15498 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.142+0000 7f9079d8f740 10 client.15498 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T09:16:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.142+0000 7f9079d8f740 20 client.15498 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.142+0000 7f9079d8f740 1 --2- 172.21.15.112:0/1343483445 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x556b6966b720 0x556b6966db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.142+0000 7f9079d8f740 4 client.15498 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:16:13.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.142+0000 7f9079d8f740 1 -- 172.21.15.112:0/1343483445 --> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b695c79d0 con 0x556b6966b720 2024-03-16T09:16:13.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.142+0000 7f906ffff640 1 --2- 172.21.15.112:0/1343483445 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x556b6966b720 0x556b6966db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.142+0000 7f904ffff640 10 client.15498 ms_handle_connect on v2:172.21.15.194:6838/2528943309 2024-03-16T09:16:13.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.142+0000 7f906ffff640 1 --2- 172.21.15.112:0/1343483445 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x556b6966b720 0x556b6966db10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d92fd4ee41e5e7a server_cookie=86eccfaeaebca9e2 in_seq=0 out_seq=0 2024-03-16T09:16:13.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.143+0000 7f904ffff640 1 -- 172.21.15.112:0/1343483445 <== mds.1 v2:172.21.15.194:6838/2528943309 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556b695c79d0 con 0x556b6966b720 2024-03-16T09:16:13.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.143+0000 7f904ffff640 10 client.15498 handle_command_reply: tid=0 2024-03-16T09:16:13.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.143+0000 7f9079d8f740 1 client.15498 shutdown 2024-03-16T09:16:13.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.143+0000 7f9079d8f740 1 -- 172.21.15.112:0/1343483445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 msgr2=0x556b6965fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.143+0000 7f9079d8f740 1 --2- 172.21.15.112:0/1343483445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 0x556b6965fcc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9064009900 tx=0x7f906402c800 comp rx=0 tx=0).stop 2024-03-16T09:16:13.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.143+0000 7f9079d8f740 1 -- 172.21.15.112:0/1343483445 shutdown_connections 2024-03-16T09:16:13.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.143+0000 7f9079d8f740 1 --2- 172.21.15.112:0/1343483445 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x556b6966b720 0x556b6966db10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.143+0000 7f9079d8f740 1 --2- 172.21.15.112:0/1343483445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b6965b3b0 0x556b69667230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.143+0000 7f9079d8f740 1 --2- 172.21.15.112:0/1343483445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69656240 0x556b69660200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.143+0000 7f904ffff640 0 client.15498 ms_handle_reset on v2:172.21.15.194:6838/2528943309 2024-03-16T09:16:13.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.143+0000 7f9079d8f740 1 --2- 172.21.15.112:0/1343483445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 0x556b6965fcc0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.143+0000 7f9079d8f740 1 -- 172.21.15.112:0/1343483445 >> 172.21.15.112:0/1343483445 conn(0x556b69650dc0 msgr2=0x556b6964ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:13.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.143+0000 7f9079d8f740 1 -- 172.21.15.112:0/1343483445 shutdown_connections 2024-03-16T09:16:13.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.143+0000 7f9079d8f740 1 -- 172.21.15.112:0/1343483445 wait complete. 2024-03-16T09:16:13.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.144+0000 7f9079d8f740 20 client.15498 trim_cache size 0 max 16384 2024-03-16T09:16:13.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.156+0000 7f9079d8f740 1 Processor -- start 2024-03-16T09:16:13.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.156+0000 7f9079d8f740 1 -- start start 2024-03-16T09:16:13.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f9079d8f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 0x556b6965f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f9079d8f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69656240 0x556b696671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f9079d8f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b6965b3b0 0x556b6965f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f9079d8f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f90701ab580 con 0x556b69655850 2024-03-16T09:16:13.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f9079d8f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f90701a8e50 con 0x556b69656240 2024-03-16T09:16:13.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f9079d8f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f90701a6ca0 con 0x556b6965b3b0 2024-03-16T09:16:13.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f9074e62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 0x556b6965f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f9074e62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 0x556b6965f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40452/0 (socket says 172.21.15.112:40452) 2024-03-16T09:16:13.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f9074e62640 1 -- 172.21.15.112:0/2802531808 learned_addr learned my addr 172.21.15.112:0/2802531808 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:13.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f906ffff640 1 --2- 172.21.15.112:0/2802531808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69656240 0x556b696671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f9075663640 1 --2- 172.21.15.112:0/2802531808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b6965b3b0 0x556b6965f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f9074e62640 1 -- 172.21.15.112:0/2802531808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b6965b3b0 msgr2=0x556b6965f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f9074e62640 1 --2- 172.21.15.112:0/2802531808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b6965b3b0 0x556b6965f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f9074e62640 1 -- 172.21.15.112:0/2802531808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69656240 msgr2=0x556b696671c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f9074e62640 1 --2- 172.21.15.112:0/2802531808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69656240 0x556b696671c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f9074e62640 1 -- 172.21.15.112:0/2802531808 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b693987d0 con 0x556b69655850 2024-03-16T09:16:13.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f9075663640 1 --2- 172.21.15.112:0/2802531808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b6965b3b0 0x556b6965f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:16:13.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f9074e62640 1 --2- 172.21.15.112:0/2802531808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 0x556b6965f2e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f906402bc70 tx=0x7f90640344e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a8bf52f0b19092 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:13.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f905a7fc640 1 -- 172.21.15.112:0/2802531808 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064060070 con 0x556b69655850 2024-03-16T09:16:13.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f905a7fc640 1 -- 172.21.15.112:0/2802531808 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9064057030 con 0x556b69655850 2024-03-16T09:16:13.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.157+0000 7f905a7fc640 1 -- 172.21.15.112:0/2802531808 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064034cd0 con 0x556b69655850 2024-03-16T09:16:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.158+0000 7f9079d8f740 1 -- 172.21.15.112:0/2802531808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 msgr2=0x556b6965f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.158+0000 7f9079d8f740 1 --2- 172.21.15.112:0/2802531808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 0x556b6965f2e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f906402bc70 tx=0x7f90640344e0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.158+0000 7f9079d8f740 1 -- 172.21.15.112:0/2802531808 shutdown_connections 2024-03-16T09:16:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.158+0000 7f9079d8f740 1 --2- 172.21.15.112:0/2802531808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b6965b3b0 0x556b6965f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.158+0000 7f9079d8f740 1 --2- 172.21.15.112:0/2802531808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69656240 0x556b696671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.158+0000 7f9079d8f740 1 --2- 172.21.15.112:0/2802531808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 0x556b6965f2e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.158+0000 7f9079d8f740 1 -- 172.21.15.112:0/2802531808 >> 172.21.15.112:0/2802531808 conn(0x556b69650dc0 msgr2=0x556b6964ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.158+0000 7f9079d8f740 1 -- 172.21.15.112:0/2802531808 shutdown_connections 2024-03-16T09:16:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.158+0000 7f9079d8f740 1 -- 172.21.15.112:0/2802531808 wait complete. 2024-03-16T09:16:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.158+0000 7f9079d8f740 1 -- start start 2024-03-16T09:16:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.158+0000 7f9079d8f740 1 Processor -- start 2024-03-16T09:16:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f9079d8f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 0x556b696648b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f9079d8f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69656240 0x556b69664df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f9079d8f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b6965b3b0 0x556b69665e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f9079d8f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f90701a7310 con 0x556b69655850 2024-03-16T09:16:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f9079d8f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f90701a8780 con 0x556b69656240 2024-03-16T09:16:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f9079d8f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f90701a79e0 con 0x556b6965b3b0 2024-03-16T09:16:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f9079d8f740 10 client.0 _ll_register_callbacks cb 0x7f9068004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f9074e62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 0x556b696648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f9074e62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 0x556b696648b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40456/0 (socket says 172.21.15.112:40456) 2024-03-16T09:16:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f9074e62640 1 -- 172.21.15.112:0/531399635 learned_addr learned my addr 172.21.15.112:0/531399635 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f906ffff640 1 --2- 172.21.15.112:0/531399635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69656240 0x556b69664df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f9075663640 1 --2- 172.21.15.112:0/531399635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b6965b3b0 0x556b69665e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f9074e62640 1 -- 172.21.15.112:0/531399635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b6965b3b0 msgr2=0x556b69665e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f9074e62640 1 --2- 172.21.15.112:0/531399635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b6965b3b0 0x556b69665e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f9074e62640 1 -- 172.21.15.112:0/531399635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69656240 msgr2=0x556b69664df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f9074e62640 1 --2- 172.21.15.112:0/531399635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69656240 0x556b69664df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f9074e62640 1 -- 172.21.15.112:0/531399635 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b696663f0 con 0x556b69655850 2024-03-16T09:16:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f9074e62640 1 --2- 172.21.15.112:0/531399635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 0x556b696648b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f906402c370 tx=0x7f9064005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170651a5e39be541 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f904ffff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:16:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f904ffff640 1 -- 172.21.15.112:0/531399635 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064060070 con 0x556b69655850 2024-03-16T09:16:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f904ffff640 1 -- 172.21.15.112:0/531399635 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f9064034a50 con 0x556b69655850 2024-03-16T09:16:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f9079d8f740 1 -- 172.21.15.112:0/531399635 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9064057030 con 0x556b69655850 2024-03-16T09:16:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f904ffff640 1 -- 172.21.15.112:0/531399635 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f9064005d50 con 0x556b69655850 2024-03-16T09:16:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.159+0000 7f9079d8f740 10 client.15504 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.160+0000 7f9079d8f740 1 -- 172.21.15.112:0/531399635 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x556b696d19a0 con 0x556b69655850 2024-03-16T09:16:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.160+0000 7f904ffff640 1 -- 172.21.15.112:0/531399635 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9064057030 con 0x556b69655850 2024-03-16T09:16:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.160+0000 7f904ffff640 1 -- 172.21.15.112:0/531399635 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f90640046c0 con 0x556b69655850 2024-03-16T09:16:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.160+0000 7f9079d8f740 10 client.15504 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.160+0000 7f9079d8f740 10 client.15504 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T09:16:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.160+0000 7f9079d8f740 20 client.15504 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:13.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.160+0000 7f9079d8f740 1 --2- 172.21.15.112:0/531399635 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x556b696d1fc0 0x556b696d43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.160+0000 7f9079d8f740 4 client.15504 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T09:16:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.160+0000 7f9079d8f740 1 -- 172.21.15.112:0/531399635 --> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b695c1d20 con 0x556b696d1fc0 2024-03-16T09:16:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.160+0000 7f906ffff640 1 --2- 172.21.15.112:0/531399635 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x556b696d1fc0 0x556b696d43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.160+0000 7f906ffff640 1 --2- 172.21.15.112:0/531399635 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x556b696d1fc0 0x556b696d43e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9c5b2837d8243f8 server_cookie=ddadeccb14760934 in_seq=0 out_seq=0 2024-03-16T09:16:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.160+0000 7f904ffff640 10 client.15504 ms_handle_connect on v2:172.21.15.194:6838/2528943309 2024-03-16T09:16:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f904ffff640 1 -- 172.21.15.112:0/531399635 <== mds.1 v2:172.21.15.194:6838/2528943309 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b695c1d20 con 0x556b696d1fc0 2024-03-16T09:16:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f904ffff640 10 client.15504 handle_command_reply: tid=0 2024-03-16T09:16:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f9079d8f740 1 client.15504 shutdown 2024-03-16T09:16:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f9079d8f740 1 -- 172.21.15.112:0/531399635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 msgr2=0x556b696648b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f9079d8f740 1 --2- 172.21.15.112:0/531399635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 0x556b696648b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f906402c370 tx=0x7f9064005910 comp rx=0 tx=0).stop 2024-03-16T09:16:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f9079d8f740 1 -- 172.21.15.112:0/531399635 shutdown_connections 2024-03-16T09:16:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f9079d8f740 1 --2- 172.21.15.112:0/531399635 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x556b696d1fc0 0x556b696d43e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f9079d8f740 1 --2- 172.21.15.112:0/531399635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556b6965b3b0 0x556b69665e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f9079d8f740 1 --2- 172.21.15.112:0/531399635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b69656240 0x556b69664df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f904ffff640 0 client.15504 ms_handle_reset on v2:172.21.15.194:6838/2528943309 2024-03-16T09:16:13.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f9079d8f740 1 --2- 172.21.15.112:0/531399635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b69655850 0x556b696648b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f9079d8f740 1 -- 172.21.15.112:0/531399635 >> 172.21.15.112:0/531399635 conn(0x556b69650dc0 msgr2=0x556b6964f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:13.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f9079d8f740 1 -- 172.21.15.112:0/531399635 shutdown_connections 2024-03-16T09:16:13.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f9079d8f740 1 -- 172.21.15.112:0/531399635 wait complete. 2024-03-16T09:16:13.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f9079d8f740 20 client.15504 trim_cache size 0 max 16384 2024-03-16T09:16:13.170 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-03-16T09:16:13.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f90770c6640 1 -- 172.21.15.112:0/314541405 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f90600a2890 msgr2=0x7f90600a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f90770c6640 1 --2- 172.21.15.112:0/314541405 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f90600a2890 0x7f90600a4d50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f905c013570 tx=0x7f905c013970 comp rx=0 tx=0).stop 2024-03-16T09:16:13.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f90770c6640 1 -- 172.21.15.112:0/314541405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90701022a0 msgr2=0x7f907019f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f90770c6640 1 --2- 172.21.15.112:0/314541405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90701022a0 0x7f907019f1b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f906800eb10 tx=0x7f906800c2d0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f90770c6640 1 -- 172.21.15.112:0/314541405 shutdown_connections 2024-03-16T09:16:13.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f90770c6640 1 --2- 172.21.15.112:0/314541405 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f90600a2890 0x7f90600a4d50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f90770c6640 1 --2- 172.21.15.112:0/314541405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90701022a0 0x7f907019f1b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f90770c6640 1 --2- 172.21.15.112:0/314541405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90701018e0 0x7f9070198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f90770c6640 1 --2- 172.21.15.112:0/314541405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90701006f0 0x7f9070197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.161+0000 7f90770c6640 1 -- 172.21.15.112:0/314541405 >> 172.21.15.112:0/314541405 conn(0x7f90700fbc80 msgr2=0x7f90700fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:13.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.162+0000 7f90770c6640 1 -- 172.21.15.112:0/314541405 shutdown_connections 2024-03-16T09:16:13.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.162+0000 7f90770c6640 1 -- 172.21.15.112:0/314541405 wait complete. 2024-03-16T09:16:13.173 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:12 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2301969179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:16:13.174 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T09:16:13.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:12 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2301969179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:16:13.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.276+0000 7f2634777640 1 -- 172.21.15.112:0/3149409529 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262c1017f0 msgr2=0x7f262c101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.276+0000 7f2634777640 1 --2- 172.21.15.112:0/3149409529 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262c1017f0 0x7f262c101c10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2624009ca0 tx=0x7f262402c0f0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.276+0000 7f2634777640 1 -- 172.21.15.112:0/3149409529 shutdown_connections 2024-03-16T09:16:13.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.276+0000 7f2634777640 1 --2- 172.21.15.112:0/3149409529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f262c1033a0 0x7f262c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.276+0000 7f2634777640 1 --2- 172.21.15.112:0/3149409529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262c1029e0 0x7f262c102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.276+0000 7f2634777640 1 --2- 172.21.15.112:0/3149409529 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262c1017f0 0x7f262c101c10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.276+0000 7f2634777640 1 -- 172.21.15.112:0/3149409529 >> 172.21.15.112:0/3149409529 conn(0x7f262c0fcd80 msgr2=0x7f262c0ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:13.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.276+0000 7f2634777640 1 -- 172.21.15.112:0/3149409529 shutdown_connections 2024-03-16T09:16:13.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.276+0000 7f2634777640 1 -- 172.21.15.112:0/3149409529 wait complete. 2024-03-16T09:16:13.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.276+0000 7f2634777640 1 Processor -- start 2024-03-16T09:16:13.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.276+0000 7f2634777640 1 -- start start 2024-03-16T09:16:13.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.276+0000 7f2634777640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262c1017f0 0x7f262c197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.276+0000 7f2634777640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f262c1029e0 0x7f262c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.276+0000 7f2634777640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262c1033a0 0x7f262c19f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.276+0000 7f2634777640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f262c10d5a0 con 0x7f262c1017f0 2024-03-16T09:16:13.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.276+0000 7f2634777640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f262c10d420 con 0x7f262c1029e0 2024-03-16T09:16:13.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.276+0000 7f2634777640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f262c10d720 con 0x7f262c1033a0 2024-03-16T09:16:13.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.276+0000 7f2632513640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262c1017f0 0x7f262c197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.277+0000 7f2632513640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262c1017f0 0x7f262c197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40482/0 (socket says 172.21.15.112:40482) 2024-03-16T09:16:13.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.277+0000 7f2632513640 1 -- 172.21.15.112:0/1325720859 learned_addr learned my addr 172.21.15.112:0/1325720859 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:13.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.277+0000 7f2632d14640 1 --2- 172.21.15.112:0/1325720859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262c1033a0 0x7f262c19f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.277+0000 7f2631d12640 1 --2- 172.21.15.112:0/1325720859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f262c1029e0 0x7f262c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.277+0000 7f2632513640 1 -- 172.21.15.112:0/1325720859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262c1033a0 msgr2=0x7f262c19f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.277+0000 7f2632513640 1 --2- 172.21.15.112:0/1325720859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262c1033a0 0x7f262c19f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.277+0000 7f2632513640 1 -- 172.21.15.112:0/1325720859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f262c1029e0 msgr2=0x7f262c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.277+0000 7f2632513640 1 --2- 172.21.15.112:0/1325720859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f262c1029e0 0x7f262c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.277+0000 7f2632513640 1 -- 172.21.15.112:0/1325720859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262c19f870 con 0x7f262c1017f0 2024-03-16T09:16:13.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.277+0000 7f2632d14640 1 --2- 172.21.15.112:0/1325720859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262c1033a0 0x7f262c19f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:13.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.277+0000 7f2632513640 1 --2- 172.21.15.112:0/1325720859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262c1017f0 0x7f262c197c60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2624037040 tx=0x7f26240346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:13.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.277+0000 7f26237fe640 1 -- 172.21.15.112:0/1325720859 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624039040 con 0x7f262c1017f0 2024-03-16T09:16:13.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.277+0000 7f2634777640 1 -- 172.21.15.112:0/1325720859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262c19fb00 con 0x7f262c1017f0 2024-03-16T09:16:13.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.277+0000 7f26237fe640 1 -- 172.21.15.112:0/1325720859 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2624044070 con 0x7f262c1017f0 2024-03-16T09:16:13.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.277+0000 7f26237fe640 1 -- 172.21.15.112:0/1325720859 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26240044d0 con 0x7f262c1017f0 2024-03-16T09:16:13.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.277+0000 7f2634777640 1 -- 172.21.15.112:0/1325720859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f262c1a0040 con 0x7f262c1017f0 2024-03-16T09:16:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.279+0000 7f26237fe640 1 -- 172.21.15.112:0/1325720859 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7f2624004670 con 0x7f262c1017f0 2024-03-16T09:16:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2637363740 1 Processor -- start 2024-03-16T09:16:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2637363740 1 -- start start 2024-03-16T09:16:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f26237fe640 1 --2- 172.21.15.112:0/1325720859 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f26180a2890 0x7f26180a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2637363740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357ada850 0x560357adac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2637363740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357adb240 0x560357adfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2637363740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 0x560357ae2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2637363740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560357a3f490 con 0x560357ae03b0 2024-03-16T09:16:13.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2637363740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5603579e7c90 con 0x560357ada850 2024-03-16T09:16:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2637363740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560357a3b530 con 0x560357adb240 2024-03-16T09:16:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2632d14640 1 --2- 172.21.15.112:0/1325720859 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f26180a2890 0x7f26180a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f26237fe640 1 -- 172.21.15.112:0/1325720859 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f26240f1100 con 0x7f262c1017f0 2024-03-16T09:16:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2632d14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 0x560357ae2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2632513640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357ada850 0x560357adac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2631d12640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357adb240 0x560357adfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2632d14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 0x560357ae2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40490/0 (socket says 172.21.15.112:40490) 2024-03-16T09:16:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2632d14640 1 -- 172.21.15.112:0/2647768271 learned_addr learned my addr 172.21.15.112:0/2647768271 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2631d12640 1 -- 172.21.15.112:0/2647768271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357ada850 msgr2=0x560357adac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2631d12640 1 --2- 172.21.15.112:0/2647768271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357ada850 0x560357adac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2631d12640 1 -- 172.21.15.112:0/2647768271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 msgr2=0x560357ae2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2631d12640 1 --2- 172.21.15.112:0/2647768271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 0x560357ae2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2631d12640 1 -- 172.21.15.112:0/2647768271 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560357aa2500 con 0x560357adb240 2024-03-16T09:16:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.280+0000 7f2632513640 1 --2- 172.21.15.112:0/2647768271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357ada850 0x560357adac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.281+0000 7f2632d14640 1 --2- 172.21.15.112:0/2647768271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 0x560357ae2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.281+0000 7f2632d14640 1 --2- 172.21.15.112:0/1325720859 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f26180a2890 0x7f26180a4d50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2628011d20 tx=0x7f2628011440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.281+0000 7f2631d12640 1 --2- 172.21.15.112:0/2647768271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357adb240 0x560357adfd40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f261c009c20 tx=0x7f261c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dec66ce0ef16110a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.281+0000 7f2620ff9640 1 -- 172.21.15.112:0/2647768271 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c004050 con 0x560357adb240 2024-03-16T09:16:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.281+0000 7f2620ff9640 1 -- 172.21.15.112:0/2647768271 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f261c005520 con 0x560357adb240 2024-03-16T09:16:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.281+0000 7f2620ff9640 1 -- 172.21.15.112:0/2647768271 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c004ce0 con 0x560357adb240 2024-03-16T09:16:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2637363740 1 -- 172.21.15.112:0/2647768271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357adb240 msgr2=0x560357adfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2637363740 1 --2- 172.21.15.112:0/2647768271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357adb240 0x560357adfd40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f261c009c20 tx=0x7f261c011810 comp rx=0 tx=0).stop 2024-03-16T09:16:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2637363740 1 -- 172.21.15.112:0/2647768271 shutdown_connections 2024-03-16T09:16:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2637363740 1 --2- 172.21.15.112:0/2647768271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 0x560357ae2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2637363740 1 --2- 172.21.15.112:0/2647768271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357adb240 0x560357adfd40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2637363740 1 --2- 172.21.15.112:0/2647768271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357ada850 0x560357adac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2637363740 1 -- 172.21.15.112:0/2647768271 >> 172.21.15.112:0/2647768271 conn(0x560357ad5dc0 msgr2=0x560357ad61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:13.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2637363740 1 -- 172.21.15.112:0/2647768271 shutdown_connections 2024-03-16T09:16:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2637363740 1 -- 172.21.15.112:0/2647768271 wait complete. 2024-03-16T09:16:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2637363740 1 -- start start 2024-03-16T09:16:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2637363740 1 Processor -- start 2024-03-16T09:16:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2637363740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357ada850 0x560357ae4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2637363740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357adb240 0x560357ae5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2637363740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 0x560357aec290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2637363740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560357ae3360 con 0x560357ae03b0 2024-03-16T09:16:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2637363740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5603579fb230 con 0x560357ada850 2024-03-16T09:16:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2637363740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560357ae34e0 con 0x560357adb240 2024-03-16T09:16:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2637363740 10 client.0 _ll_register_callbacks cb 0x560357a37010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2632d14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 0x560357aec290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2632d14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 0x560357aec290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40506/0 (socket says 172.21.15.112:40506) 2024-03-16T09:16:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2632d14640 1 -- 172.21.15.112:0/2662718849 learned_addr learned my addr 172.21.15.112:0/2662718849 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:13.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2631d12640 1 --2- 172.21.15.112:0/2662718849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357adb240 0x560357ae5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.282+0000 7f2632513640 1 --2- 172.21.15.112:0/2662718849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357ada850 0x560357ae4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2632d14640 1 -- 172.21.15.112:0/2662718849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357adb240 msgr2=0x560357ae5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2632d14640 1 --2- 172.21.15.112:0/2662718849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357adb240 0x560357ae5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2632d14640 1 -- 172.21.15.112:0/2662718849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357ada850 msgr2=0x560357ae4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2632d14640 1 --2- 172.21.15.112:0/2662718849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357ada850 0x560357ae4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2632d14640 1 -- 172.21.15.112:0/2662718849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560357a0a100 con 0x560357ae03b0 2024-03-16T09:16:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2632d14640 1 --2- 172.21.15.112:0/2662718849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 0x560357aec290 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2628009d00 tx=0x7f2628005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34940acd74530122 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2615ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:16:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2615ffb640 1 -- 172.21.15.112:0/2662718849 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2628024070 con 0x560357ae03b0 2024-03-16T09:16:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2615ffb640 1 -- 172.21.15.112:0/2662718849 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f2628029460 con 0x560357ae03b0 2024-03-16T09:16:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2637363740 1 -- 172.21.15.112:0/2662718849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f261c005520 con 0x560357ae03b0 2024-03-16T09:16:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2615ffb640 1 -- 172.21.15.112:0/2662718849 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2628029660 con 0x560357ae03b0 2024-03-16T09:16:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2637363740 10 client.15516 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:13.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2637363740 1 -- 172.21.15.112:0/2662718849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5603577533f0 con 0x560357ae03b0 2024-03-16T09:16:13.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2615ffb640 1 -- 172.21.15.112:0/2662718849 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f261c005520 con 0x560357ae03b0 2024-03-16T09:16:13.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2615ffb640 1 -- 172.21.15.112:0/2662718849 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f262801f3e0 con 0x560357ae03b0 2024-03-16T09:16:13.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2637363740 10 client.15516 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:13.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2637363740 10 client.15516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:13.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2637363740 20 client.15516 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:13.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2637363740 1 --2- 172.21.15.112:0/2662718849 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560357af0bc0 0x560357af3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2637363740 4 client.15516 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:16:13.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.283+0000 7f2637363740 1 -- 172.21.15.112:0/2662718849 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560357a4c9d0 con 0x560357af0bc0 2024-03-16T09:16:13.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.284+0000 7f2632513640 1 --2- 172.21.15.112:0/2662718849 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560357af0bc0 0x560357af3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.284+0000 7f2632513640 1 --2- 172.21.15.112:0/2662718849 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560357af0bc0 0x560357af3060 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5767eb15715754f8 server_cookie=1c8446316e27fdb6 in_seq=0 out_seq=0 2024-03-16T09:16:13.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.284+0000 7f2615ffb640 10 client.15516 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:13.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.285+0000 7f2615ffb640 1 -- 172.21.15.112:0/2662718849 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560357a4c9d0 con 0x560357af0bc0 2024-03-16T09:16:13.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.285+0000 7f2615ffb640 10 client.15516 handle_command_reply: tid=0 2024-03-16T09:16:13.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.285+0000 7f2637363740 1 client.15516 shutdown 2024-03-16T09:16:13.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.285+0000 7f2637363740 1 -- 172.21.15.112:0/2662718849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 msgr2=0x560357aec290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.285+0000 7f2637363740 1 --2- 172.21.15.112:0/2662718849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 0x560357aec290 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2628009d00 tx=0x7f2628005f90 comp rx=0 tx=0).stop 2024-03-16T09:16:13.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.285+0000 7f2637363740 1 -- 172.21.15.112:0/2662718849 shutdown_connections 2024-03-16T09:16:13.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.285+0000 7f2637363740 1 --2- 172.21.15.112:0/2662718849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 0x560357aec290 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.285+0000 7f2637363740 1 --2- 172.21.15.112:0/2662718849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357adb240 0x560357ae5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.285+0000 7f2637363740 1 --2- 172.21.15.112:0/2662718849 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560357af0bc0 0x560357af3060 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.285+0000 7f2637363740 1 --2- 172.21.15.112:0/2662718849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357ada850 0x560357ae4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.285+0000 7f2615ffb640 0 client.15516 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:13.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.285+0000 7f2637363740 1 -- 172.21.15.112:0/2662718849 >> 172.21.15.112:0/2662718849 conn(0x560357ad5dc0 msgr2=0x560357ad4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:13.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.285+0000 7f2637363740 1 -- 172.21.15.112:0/2662718849 shutdown_connections 2024-03-16T09:16:13.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.285+0000 7f2637363740 1 -- 172.21.15.112:0/2662718849 wait complete. 2024-03-16T09:16:13.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.285+0000 7f2637363740 20 client.15516 trim_cache size 0 max 16384 2024-03-16T09:16:13.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2637363740 1 Processor -- start 2024-03-16T09:16:13.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2637363740 1 -- start start 2024-03-16T09:16:13.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2637363740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357ada850 0x560357ad4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2637363740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357adb240 0x560357aec250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2637363740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 0x560357af7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2637363740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1ab6b0 con 0x560357ae03b0 2024-03-16T09:16:13.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2637363740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1a8f80 con 0x560357adb240 2024-03-16T09:16:13.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2637363740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1a6dd0 con 0x560357ada850 2024-03-16T09:16:13.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2632513640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357ada850 0x560357ad4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2632d14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 0x560357af7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2631d12640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357adb240 0x560357aec250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2632513640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357ada850 0x560357ad4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:36616/0 (socket says 172.21.15.112:36616) 2024-03-16T09:16:13.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2631d12640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357adb240 0x560357aec250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:41978/0 (socket says 172.21.15.112:41978) 2024-03-16T09:16:13.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2632513640 1 -- 172.21.15.112:0/3321334851 learned_addr learned my addr 172.21.15.112:0/3321334851 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:13.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2632d14640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 0x560357af7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40516/0 (socket says 172.21.15.112:40516) 2024-03-16T09:16:13.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2632513640 1 -- 172.21.15.112:0/3321334851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357adb240 msgr2=0x560357aec250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2632513640 1 --2- 172.21.15.112:0/3321334851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357adb240 0x560357aec250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2632513640 1 -- 172.21.15.112:0/3321334851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 msgr2=0x560357af7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2632513640 1 --2- 172.21.15.112:0/3321334851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 0x560357af7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2632513640 1 -- 172.21.15.112:0/3321334851 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56035781d7d0 con 0x560357ada850 2024-03-16T09:16:13.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2631d12640 1 --2- 172.21.15.112:0/3321334851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357adb240 0x560357aec250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:13.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.298+0000 7f2632d14640 1 --2- 172.21.15.112:0/3321334851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 0x560357af7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:13.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.299+0000 7f2632513640 1 --2- 172.21.15.112:0/3321334851 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357ada850 0x560357ad4e60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f26240ba3a0 tx=0x7f2624032bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f12170a5d06c2b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:13.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.299+0000 7f2617fff640 1 -- 172.21.15.112:0/3321334851 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26240bda60 con 0x560357ada850 2024-03-16T09:16:13.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.299+0000 7f2617fff640 1 -- 172.21.15.112:0/3321334851 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f26240c1bb0 con 0x560357ada850 2024-03-16T09:16:13.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.299+0000 7f2617fff640 1 -- 172.21.15.112:0/3321334851 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26240ba040 con 0x560357ada850 2024-03-16T09:16:13.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.299+0000 7f2637363740 1 -- 172.21.15.112:0/3321334851 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357ada850 msgr2=0x560357ad4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.299+0000 7f2637363740 1 --2- 172.21.15.112:0/3321334851 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357ada850 0x560357ad4e60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f26240ba3a0 tx=0x7f2624032bc0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.299+0000 7f2637363740 1 -- 172.21.15.112:0/3321334851 shutdown_connections 2024-03-16T09:16:13.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.299+0000 7f2637363740 1 --2- 172.21.15.112:0/3321334851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357ae03b0 0x560357af7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.299+0000 7f2637363740 1 --2- 172.21.15.112:0/3321334851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357adb240 0x560357aec250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.299+0000 7f2637363740 1 --2- 172.21.15.112:0/3321334851 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357ada850 0x560357ad4e60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.299+0000 7f2637363740 1 -- 172.21.15.112:0/3321334851 >> 172.21.15.112:0/3321334851 conn(0x560357ad5dc0 msgr2=0x560357ad4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:13.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.299+0000 7f2637363740 1 -- 172.21.15.112:0/3321334851 shutdown_connections 2024-03-16T09:16:13.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.299+0000 7f2637363740 1 -- 172.21.15.112:0/3321334851 wait complete. 2024-03-16T09:16:13.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2637363740 1 -- start start 2024-03-16T09:16:13.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2637363740 1 Processor -- start 2024-03-16T09:16:13.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2637363740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357ada850 0x560357ae9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2637363740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357adb240 0x560357aea4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2637363740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357ae03b0 0x560357aeb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2637363740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1a7440 con 0x560357adb240 2024-03-16T09:16:13.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2637363740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1a88b0 con 0x560357ae03b0 2024-03-16T09:16:13.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2637363740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f262c1a7b10 con 0x560357ada850 2024-03-16T09:16:13.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2637363740 10 client.0 _ll_register_callbacks cb 0x7f2628009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:13.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2631d12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357adb240 0x560357aea4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2632513640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357ada850 0x560357ae9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2631d12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357adb240 0x560357aea4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40526/0 (socket says 172.21.15.112:40526) 2024-03-16T09:16:13.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2631d12640 1 -- 172.21.15.112:0/1774445583 learned_addr learned my addr 172.21.15.112:0/1774445583 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:13.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2632d14640 1 --2- 172.21.15.112:0/1774445583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357ae03b0 0x560357aeb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2631d12640 1 -- 172.21.15.112:0/1774445583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357ada850 msgr2=0x560357ae9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2631d12640 1 --2- 172.21.15.112:0/1774445583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357ada850 0x560357ae9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2631d12640 1 -- 172.21.15.112:0/1774445583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357ae03b0 msgr2=0x560357aeb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:13.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2631d12640 1 --2- 172.21.15.112:0/1774445583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357ae03b0 0x560357aeb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2631d12640 1 -- 172.21.15.112:0/1774445583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560357aeb9d0 con 0x560357adb240 2024-03-16T09:16:13.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2632513640 1 --2- 172.21.15.112:0/1774445583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357ada850 0x560357ae9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:13.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2631d12640 1 --2- 172.21.15.112:0/1774445583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357adb240 0x560357aea4b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f261c014040 tx=0x7f261c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788699bc3795a9dd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:13.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2615ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:16:13.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2615ffb640 1 -- 172.21.15.112:0/1774445583 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c005450 con 0x560357adb240 2024-03-16T09:16:13.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.300+0000 7f2615ffb640 1 -- 172.21.15.112:0/1774445583 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f261c004640 con 0x560357adb240 2024-03-16T09:16:13.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.301+0000 7f2637363740 1 -- 172.21.15.112:0/1774445583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26240c1bb0 con 0x560357adb240 2024-03-16T09:16:13.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.301+0000 7f2615ffb640 1 -- 172.21.15.112:0/1774445583 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f261c004840 con 0x560357adb240 2024-03-16T09:16:13.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.301+0000 7f2637363740 10 client.15522 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:13.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.301+0000 7f2637363740 1 -- 172.21.15.112:0/1774445583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560357b574d0 con 0x560357adb240 2024-03-16T09:16:13.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.301+0000 7f2615ffb640 1 -- 172.21.15.112:0/1774445583 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f26240c1bb0 con 0x560357adb240 2024-03-16T09:16:13.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.301+0000 7f2615ffb640 1 -- 172.21.15.112:0/1774445583 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f261c02a070 con 0x560357adb240 2024-03-16T09:16:13.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.301+0000 7f2637363740 10 client.15522 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:13.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.301+0000 7f2637363740 10 client.15522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:13.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.301+0000 7f2637363740 20 client.15522 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:13.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.301+0000 7f2637363740 1 --2- 172.21.15.112:0/1774445583 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560357b57d30 0x560357b5a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:13.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.301+0000 7f2637363740 4 client.15522 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T09:16:13.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.301+0000 7f2637363740 1 -- 172.21.15.112:0/1774445583 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560357a46d20 con 0x560357b57d30 2024-03-16T09:16:13.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.301+0000 7f2632513640 1 --2- 172.21.15.112:0/1774445583 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560357b57d30 0x560357b5a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:13.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2615ffb640 10 client.15522 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:13.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2632513640 1 --2- 172.21.15.112:0/1774445583 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560357b57d30 0x560357b5a150 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc62fbdc9b82b9e8 server_cookie=8e9338a18c8b60f9 in_seq=0 out_seq=0 2024-03-16T09:16:13.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2615ffb640 1 -- 172.21.15.112:0/1774445583 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560357a46d20 con 0x560357b57d30 2024-03-16T09:16:13.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2615ffb640 10 client.15522 handle_command_reply: tid=0 2024-03-16T09:16:13.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2637363740 1 client.15522 shutdown 2024-03-16T09:16:13.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2637363740 1 -- 172.21.15.112:0/1774445583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357adb240 msgr2=0x560357aea4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2637363740 1 --2- 172.21.15.112:0/1774445583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357adb240 0x560357aea4b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f261c014040 tx=0x7f261c0043c0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2637363740 1 -- 172.21.15.112:0/1774445583 shutdown_connections 2024-03-16T09:16:13.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2637363740 1 --2- 172.21.15.112:0/1774445583 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560357b57d30 0x560357b5a150 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2637363740 1 --2- 172.21.15.112:0/1774445583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560357ae03b0 0x560357aeb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2615ffb640 0 client.15522 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:13.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2637363740 1 --2- 172.21.15.112:0/1774445583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560357adb240 0x560357aea4b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2637363740 1 --2- 172.21.15.112:0/1774445583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560357ada850 0x560357ae9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2637363740 1 -- 172.21.15.112:0/1774445583 >> 172.21.15.112:0/1774445583 conn(0x560357ad5dc0 msgr2=0x560357ad4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:13.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2637363740 1 -- 172.21.15.112:0/1774445583 shutdown_connections 2024-03-16T09:16:13.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2637363740 1 -- 172.21.15.112:0/1774445583 wait complete. 2024-03-16T09:16:13.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2637363740 20 client.15522 trim_cache size 0 max 16384 2024-03-16T09:16:13.312 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-03-16T09:16:13.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2634777640 1 -- 172.21.15.112:0/1325720859 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f26180a2890 msgr2=0x7f26180a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2634777640 1 --2- 172.21.15.112:0/1325720859 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f26180a2890 0x7f26180a4d50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2628011d20 tx=0x7f2628011440 comp rx=0 tx=0).stop 2024-03-16T09:16:13.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2634777640 1 -- 172.21.15.112:0/1325720859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262c1017f0 msgr2=0x7f262c197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:13.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2634777640 1 --2- 172.21.15.112:0/1325720859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262c1017f0 0x7f262c197c60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2624037040 tx=0x7f26240346f0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2634777640 1 -- 172.21.15.112:0/1325720859 shutdown_connections 2024-03-16T09:16:13.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2634777640 1 --2- 172.21.15.112:0/1325720859 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f26180a2890 0x7f26180a4d50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2634777640 1 --2- 172.21.15.112:0/1325720859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262c1033a0 0x7f262c19f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2634777640 1 --2- 172.21.15.112:0/1325720859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f262c1029e0 0x7f262c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2634777640 1 --2- 172.21.15.112:0/1325720859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f262c1017f0 0x7f262c197c60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:13.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.302+0000 7f2634777640 1 -- 172.21.15.112:0/1325720859 >> 172.21.15.112:0/1325720859 conn(0x7f262c0fcd80 msgr2=0x7f262c105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:13.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.303+0000 7f2634777640 1 -- 172.21.15.112:0/1325720859 shutdown_connections 2024-03-16T09:16:13.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:13.303+0000 7f2634777640 1 -- 172.21.15.112:0/1325720859 wait complete. 2024-03-16T09:16:13.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:12 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2301969179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:16:14.303 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:13 smithi194 ceph-mon[36729]: pgmap v368: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-16T09:16:14.315 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T09:16:14.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:16:14.315 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T09:16:14.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:13 smithi112 ceph-mon[32307]: pgmap v368: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-16T09:16:14.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.431+0000 7fd458b1b640 1 -- 172.21.15.112:0/349011770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4541033a0 msgr2=0x7fd454109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:14.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.431+0000 7fd458b1b640 1 --2- 172.21.15.112:0/349011770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4541033a0 0x7fd454109c30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009ca0 tx=0x7fd44c02c0f0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.431+0000 7fd458b1b640 1 -- 172.21.15.112:0/349011770 shutdown_connections 2024-03-16T09:16:14.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.431+0000 7fd458b1b640 1 --2- 172.21.15.112:0/349011770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4541033a0 0x7fd454109c30 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.431+0000 7fd458b1b640 1 --2- 172.21.15.112:0/349011770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4541029e0 0x7fd454102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.431+0000 7fd458b1b640 1 --2- 172.21.15.112:0/349011770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4541017f0 0x7fd454101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.431+0000 7fd458b1b640 1 -- 172.21.15.112:0/349011770 >> 172.21.15.112:0/349011770 conn(0x7fd4540fcd80 msgr2=0x7fd4540ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:14.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.431+0000 7fd458b1b640 1 -- 172.21.15.112:0/349011770 shutdown_connections 2024-03-16T09:16:14.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.431+0000 7fd458b1b640 1 -- 172.21.15.112:0/349011770 wait complete. 2024-03-16T09:16:14.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.432+0000 7fd458b1b640 1 Processor -- start 2024-03-16T09:16:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.432+0000 7fd458b1b640 1 -- start start 2024-03-16T09:16:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.432+0000 7fd458b1b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4541017f0 0x7fd454197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.432+0000 7fd458b1b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4541029e0 0x7fd454198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.432+0000 7fd458b1b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4541033a0 0x7fd45419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.432+0000 7fd458b1b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd45410d5a0 con 0x7fd4541029e0 2024-03-16T09:16:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.432+0000 7fd458b1b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd45410d420 con 0x7fd4541017f0 2024-03-16T09:16:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.432+0000 7fd458b1b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd45410d720 con 0x7fd4541033a0 2024-03-16T09:16:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.432+0000 7fd451d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4541029e0 0x7fd454198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.432+0000 7fd45259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4541017f0 0x7fd454197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.433+0000 7fd45259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4541017f0 0x7fd454197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:42008/0 (socket says 172.21.15.112:42008) 2024-03-16T09:16:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.432+0000 7fd451d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4541029e0 0x7fd454198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40534/0 (socket says 172.21.15.112:40534) 2024-03-16T09:16:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.433+0000 7fd45259c640 1 -- 172.21.15.112:0/2799335866 learned_addr learned my addr 172.21.15.112:0/2799335866 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.433+0000 7fd452d9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4541033a0 0x7fd45419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.433+0000 7fd45259c640 1 -- 172.21.15.112:0/2799335866 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4541033a0 msgr2=0x7fd45419f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.433+0000 7fd45259c640 1 --2- 172.21.15.112:0/2799335866 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4541033a0 0x7fd45419f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.433+0000 7fd45259c640 1 -- 172.21.15.112:0/2799335866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4541029e0 msgr2=0x7fd454198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.433+0000 7fd45259c640 1 --2- 172.21.15.112:0/2799335866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4541029e0 0x7fd454198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.433+0000 7fd45259c640 1 -- 172.21.15.112:0/2799335866 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45419f760 con 0x7fd4541017f0 2024-03-16T09:16:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.433+0000 7fd451d9b640 1 --2- 172.21.15.112:0/2799335866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4541029e0 0x7fd454198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.433+0000 7fd452d9d640 1 --2- 172.21.15.112:0/2799335866 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4541033a0 0x7fd45419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.433+0000 7fd45259c640 1 --2- 172.21.15.112:0/2799335866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4541017f0 0x7fd454197b40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd44400e9c0 tx=0x7fd44400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.433+0000 7fd4437fe640 1 -- 172.21.15.112:0/2799335866 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44400cd80 con 0x7fd4541017f0 2024-03-16T09:16:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.433+0000 7fd458b1b640 1 -- 172.21.15.112:0/2799335866 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd45419fa50 con 0x7fd4541017f0 2024-03-16T09:16:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.433+0000 7fd4437fe640 1 -- 172.21.15.112:0/2799335866 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd4440035f0 con 0x7fd4541017f0 2024-03-16T09:16:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.433+0000 7fd458b1b640 1 -- 172.21.15.112:0/2799335866 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd45419fff0 con 0x7fd4541017f0 2024-03-16T09:16:14.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.433+0000 7fd4437fe640 1 -- 172.21.15.112:0/2799335866 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd444010690 con 0x7fd4541017f0 2024-03-16T09:16:14.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.435+0000 7fd4437fe640 1 -- 172.21.15.112:0/2799335866 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7fd4440037a0 con 0x7fd4541017f0 2024-03-16T09:16:14.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd45b7e6740 1 Processor -- start 2024-03-16T09:16:14.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd45b7e6740 1 -- start start 2024-03-16T09:16:14.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd45b7e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf5f990 0x55d95bf5fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd45b7e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf60380 0x55d95bf64e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd45b7e6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf654f0 0x55d95bf67980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd45b7e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d95bec40d0 con 0x55d95bf5f990 2024-03-16T09:16:14.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd45b7e6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d95be6cc70 con 0x55d95bf654f0 2024-03-16T09:16:14.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd45b7e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d95bec0170 con 0x55d95bf60380 2024-03-16T09:16:14.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd452d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf5f990 0x55d95bf5fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd4437fe640 1 --2- 172.21.15.112:0/2799335866 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd43c0a2890 0x7fd43c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd452d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf5f990 0x55d95bf5fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40538/0 (socket says 172.21.15.112:40538) 2024-03-16T09:16:14.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd452d9d640 1 -- 172.21.15.112:0/3176369288 learned_addr learned my addr 172.21.15.112:0/3176369288 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:14.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd451d9b640 1 --2- 172.21.15.112:0/3176369288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf654f0 0x55d95bf67980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd45259c640 1 --2- 172.21.15.112:0/3176369288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf60380 0x55d95bf64e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd452d9d640 1 --2- 172.21.15.112:0/2799335866 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd43c0a2890 0x7fd43c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd4437fe640 1 -- 172.21.15.112:0/2799335866 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd4440cefc0 con 0x7fd4541017f0 2024-03-16T09:16:14.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd452d9d640 1 -- 172.21.15.112:0/3176369288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf60380 msgr2=0x55d95bf64e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:14.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd452d9d640 1 --2- 172.21.15.112:0/3176369288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf60380 0x55d95bf64e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd452d9d640 1 -- 172.21.15.112:0/3176369288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf654f0 msgr2=0x55d95bf67980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:14.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd452d9d640 1 --2- 172.21.15.112:0/3176369288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf654f0 0x55d95bf67980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd452d9d640 1 -- 172.21.15.112:0/3176369288 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d95bca07e0 con 0x55d95bf5f990 2024-03-16T09:16:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.436+0000 7fd451d9b640 1 --2- 172.21.15.112:0/3176369288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf654f0 0x55d95bf67980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.437+0000 7fd452d9d640 1 --2- 172.21.15.112:0/3176369288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf5f990 0x55d95bf5fdb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd44c006010 tx=0x7fd44c032d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3f994595d339fe server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.437+0000 7fd440ff9640 1 -- 172.21.15.112:0/3176369288 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c050070 con 0x55d95bf5f990 2024-03-16T09:16:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.437+0000 7fd440ff9640 1 -- 172.21.15.112:0/3176369288 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd44c032990 con 0x55d95bf5f990 2024-03-16T09:16:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.437+0000 7fd440ff9640 1 -- 172.21.15.112:0/3176369288 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c039040 con 0x55d95bf5f990 2024-03-16T09:16:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd45b7e6740 1 -- 172.21.15.112:0/3176369288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf5f990 msgr2=0x55d95bf5fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd452d9d640 1 --2- 172.21.15.112:0/2799335866 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd43c0a2890 0x7fd43c0a4d50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd44c04d000 tx=0x7fd44c048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/3176369288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf5f990 0x55d95bf5fdb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd44c006010 tx=0x7fd44c032d30 comp rx=0 tx=0).stop 2024-03-16T09:16:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd45b7e6740 1 -- 172.21.15.112:0/3176369288 shutdown_connections 2024-03-16T09:16:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/3176369288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf654f0 0x55d95bf67980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/3176369288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf60380 0x55d95bf64e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/3176369288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf5f990 0x55d95bf5fdb0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd45b7e6740 1 -- 172.21.15.112:0/3176369288 >> 172.21.15.112:0/3176369288 conn(0x55d95bf27c50 msgr2=0x55d95bf5b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:14.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd45b7e6740 1 -- 172.21.15.112:0/3176369288 shutdown_connections 2024-03-16T09:16:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd45b7e6740 1 -- 172.21.15.112:0/3176369288 wait complete. 2024-03-16T09:16:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd45b7e6740 1 -- start start 2024-03-16T09:16:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd45b7e6740 1 Processor -- start 2024-03-16T09:16:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd45b7e6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf5f990 0x55d95bf69e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd45b7e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf60380 0x55d95bf6a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd45b7e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf654f0 0x55d95bf71400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd45b7e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d95bf684a0 con 0x55d95bf60380 2024-03-16T09:16:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd45b7e6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d95be800d0 con 0x55d95bf5f990 2024-03-16T09:16:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd45b7e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d95bf68620 con 0x55d95bf654f0 2024-03-16T09:16:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd45b7e6740 10 client.0 _ll_register_callbacks cb 0x55d95be49760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.438+0000 7fd451d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf60380 0x55d95bf6a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.439+0000 7fd451d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf60380 0x55d95bf6a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40544/0 (socket says 172.21.15.112:40544) 2024-03-16T09:16:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.439+0000 7fd451d9b640 1 -- 172.21.15.112:0/2582100896 learned_addr learned my addr 172.21.15.112:0/2582100896 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:14.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.439+0000 7fd452d9d640 1 --2- 172.21.15.112:0/2582100896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf654f0 0x55d95bf71400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.439+0000 7fd45259c640 1 --2- 172.21.15.112:0/2582100896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf5f990 0x55d95bf69e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.439+0000 7fd451d9b640 1 -- 172.21.15.112:0/2582100896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf654f0 msgr2=0x55d95bf71400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.439+0000 7fd451d9b640 1 --2- 172.21.15.112:0/2582100896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf654f0 0x55d95bf71400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.439+0000 7fd451d9b640 1 -- 172.21.15.112:0/2582100896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf5f990 msgr2=0x55d95bf69e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.439+0000 7fd451d9b640 1 --2- 172.21.15.112:0/2582100896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf5f990 0x55d95bf69e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.439+0000 7fd451d9b640 1 -- 172.21.15.112:0/2582100896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d95bd00f70 con 0x55d95bf60380 2024-03-16T09:16:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.439+0000 7fd452d9d640 1 --2- 172.21.15.112:0/2582100896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf654f0 0x55d95bf71400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.439+0000 7fd451d9b640 1 --2- 172.21.15.112:0/2582100896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf60380 0x55d95bf6a3c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd448012b10 tx=0x7fd44800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae6fc8b5a03c371 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.439+0000 7fd439ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:16:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.439+0000 7fd439ffb640 1 -- 172.21.15.112:0/2582100896 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44800fae0 con 0x55d95bf60380 2024-03-16T09:16:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.439+0000 7fd439ffb640 1 -- 172.21.15.112:0/2582100896 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd44800fc80 con 0x55d95bf60380 2024-03-16T09:16:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.439+0000 7fd45b7e6740 1 -- 172.21.15.112:0/2582100896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd44c032990 con 0x55d95bf60380 2024-03-16T09:16:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.439+0000 7fd439ffb640 1 -- 172.21.15.112:0/2582100896 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd4480047a0 con 0x55d95bf60380 2024-03-16T09:16:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.439+0000 7fd45b7e6740 10 client.15534 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.439+0000 7fd45b7e6740 1 -- 172.21.15.112:0/2582100896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d95bb98170 con 0x55d95bf60380 2024-03-16T09:16:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.439+0000 7fd439ffb640 1 -- 172.21.15.112:0/2582100896 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd44c032990 con 0x55d95bf60380 2024-03-16T09:16:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.440+0000 7fd439ffb640 1 -- 172.21.15.112:0/2582100896 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd448010660 con 0x55d95bf60380 2024-03-16T09:16:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.440+0000 7fd45b7e6740 10 client.15534 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.440+0000 7fd45b7e6740 10 client.15534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.440+0000 7fd45b7e6740 20 client.15534 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.440+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/2582100896 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55d95bf75a50 0x55d95bf77e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.440+0000 7fd45b7e6740 4 client.15534 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:16:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.440+0000 7fd45b7e6740 1 -- 172.21.15.112:0/2582100896 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d95bed1610 con 0x55d95bf75a50 2024-03-16T09:16:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.440+0000 7fd45259c640 1 --2- 172.21.15.112:0/2582100896 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55d95bf75a50 0x55d95bf77e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.440+0000 7fd439ffb640 10 client.15534 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.440+0000 7fd45259c640 1 --2- 172.21.15.112:0/2582100896 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55d95bf75a50 0x55d95bf77e90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d64a4887f49c8c2 server_cookie=4e87e159bb994785 in_seq=0 out_seq=0 2024-03-16T09:16:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.441+0000 7fd439ffb640 1 -- 172.21.15.112:0/2582100896 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d95bed1610 con 0x55d95bf75a50 2024-03-16T09:16:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.441+0000 7fd439ffb640 10 client.15534 handle_command_reply: tid=0 2024-03-16T09:16:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.441+0000 7fd45b7e6740 1 client.15534 shutdown 2024-03-16T09:16:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.441+0000 7fd45b7e6740 1 -- 172.21.15.112:0/2582100896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf60380 msgr2=0x55d95bf6a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.441+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/2582100896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf60380 0x55d95bf6a3c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd448012b10 tx=0x7fd44800f3c0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.441+0000 7fd45b7e6740 1 -- 172.21.15.112:0/2582100896 shutdown_connections 2024-03-16T09:16:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.441+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/2582100896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf654f0 0x55d95bf71400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.441+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/2582100896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf60380 0x55d95bf6a3c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.441+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/2582100896 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55d95bf75a50 0x55d95bf77e90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.441+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/2582100896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf5f990 0x55d95bf69e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.441+0000 7fd439ffb640 0 client.15534 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.441+0000 7fd45b7e6740 1 -- 172.21.15.112:0/2582100896 >> 172.21.15.112:0/2582100896 conn(0x55d95bf27c50 msgr2=0x55d95bf5a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.442+0000 7fd45b7e6740 1 -- 172.21.15.112:0/2582100896 shutdown_connections 2024-03-16T09:16:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.442+0000 7fd45b7e6740 1 -- 172.21.15.112:0/2582100896 wait complete. 2024-03-16T09:16:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.442+0000 7fd45b7e6740 20 client.15534 trim_cache size 0 max 16384 2024-03-16T09:16:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.459+0000 7fd45b7e6740 1 Processor -- start 2024-03-16T09:16:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.459+0000 7fd45b7e6740 1 -- start start 2024-03-16T09:16:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.459+0000 7fd45b7e6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf5f990 0x55d95bf61880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.459+0000 7fd45b7e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf60380 0x55d95bf61dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.459+0000 7fd45b7e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf654f0 0x55d95bf5e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.459+0000 7fd45b7e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4541ab620 con 0x55d95bf654f0 2024-03-16T09:16:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.459+0000 7fd45b7e6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd4541a8ef0 con 0x55d95bf5f990 2024-03-16T09:16:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.459+0000 7fd45b7e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd4541a6d40 con 0x55d95bf60380 2024-03-16T09:16:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.459+0000 7fd452d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf654f0 0x55d95bf5e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.459+0000 7fd452d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf654f0 0x55d95bf5e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40554/0 (socket says 172.21.15.112:40554) 2024-03-16T09:16:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.459+0000 7fd452d9d640 1 -- 172.21.15.112:0/2470691617 learned_addr learned my addr 172.21.15.112:0/2470691617 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.460+0000 7fd451d9b640 1 --2- 172.21.15.112:0/2470691617 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf60380 0x55d95bf61dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.460+0000 7fd45259c640 1 --2- 172.21.15.112:0/2470691617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf5f990 0x55d95bf61880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.460+0000 7fd452d9d640 1 -- 172.21.15.112:0/2470691617 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf60380 msgr2=0x55d95bf61dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.460+0000 7fd452d9d640 1 --2- 172.21.15.112:0/2470691617 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf60380 0x55d95bf61dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.460+0000 7fd452d9d640 1 -- 172.21.15.112:0/2470691617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf5f990 msgr2=0x55d95bf61880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.460+0000 7fd452d9d640 1 --2- 172.21.15.112:0/2470691617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf5f990 0x55d95bf61880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.460+0000 7fd452d9d640 1 -- 172.21.15.112:0/2470691617 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d95bf5f5d0 con 0x55d95bf654f0 2024-03-16T09:16:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.460+0000 7fd452d9d640 1 --2- 172.21.15.112:0/2470691617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf654f0 0x55d95bf5e540 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55d95bf60be0 tx=0x7fd44c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d576239729ff69 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.460+0000 7fd43bfff640 1 -- 172.21.15.112:0/2470691617 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c050070 con 0x55d95bf654f0 2024-03-16T09:16:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.460+0000 7fd43bfff640 1 -- 172.21.15.112:0/2470691617 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd44c059070 con 0x55d95bf654f0 2024-03-16T09:16:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.460+0000 7fd43bfff640 1 -- 172.21.15.112:0/2470691617 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c039040 con 0x55d95bf654f0 2024-03-16T09:16:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd45b7e6740 1 -- 172.21.15.112:0/2470691617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf654f0 msgr2=0x55d95bf5e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/2470691617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf654f0 0x55d95bf5e540 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55d95bf60be0 tx=0x7fd44c032670 comp rx=0 tx=0).stop 2024-03-16T09:16:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd45b7e6740 1 -- 172.21.15.112:0/2470691617 shutdown_connections 2024-03-16T09:16:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/2470691617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf654f0 0x55d95bf5e540 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/2470691617 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf60380 0x55d95bf61dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/2470691617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf5f990 0x55d95bf61880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd45b7e6740 1 -- 172.21.15.112:0/2470691617 >> 172.21.15.112:0/2470691617 conn(0x55d95bf27c50 msgr2=0x55d95bf69e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd45b7e6740 1 -- 172.21.15.112:0/2470691617 shutdown_connections 2024-03-16T09:16:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd45b7e6740 1 -- 172.21.15.112:0/2470691617 wait complete. 2024-03-16T09:16:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd45b7e6740 1 -- start start 2024-03-16T09:16:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd45b7e6740 1 Processor -- start 2024-03-16T09:16:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd45b7e6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf5f990 0x55d95bfd75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd45b7e6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf60380 0x55d95bfd7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd45b7e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf654f0 0x55d95bfdeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd45b7e6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4541a73b0 con 0x55d95bf60380 2024-03-16T09:16:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd45b7e6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd4541a8820 con 0x55d95bf5f990 2024-03-16T09:16:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd45b7e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd4541a7a80 con 0x55d95bf654f0 2024-03-16T09:16:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd45b7e6740 10 client.0 _ll_register_callbacks cb 0x7fd448009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:14.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd451d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf60380 0x55d95bfd7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd451d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf60380 0x55d95bfd7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40566/0 (socket says 172.21.15.112:40566) 2024-03-16T09:16:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd451d9b640 1 -- 172.21.15.112:0/1710775654 learned_addr learned my addr 172.21.15.112:0/1710775654 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd45259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf5f990 0x55d95bfd75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.461+0000 7fd452d9d640 1 --2- 172.21.15.112:0/1710775654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf654f0 0x55d95bfdeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd451d9b640 1 -- 172.21.15.112:0/1710775654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf654f0 msgr2=0x55d95bfdeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd451d9b640 1 --2- 172.21.15.112:0/1710775654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf654f0 0x55d95bfdeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd451d9b640 1 -- 172.21.15.112:0/1710775654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf5f990 msgr2=0x55d95bfd75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd451d9b640 1 --2- 172.21.15.112:0/1710775654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf5f990 0x55d95bfd75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd451d9b640 1 -- 172.21.15.112:0/1710775654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d95bf5b8a0 con 0x55d95bf60380 2024-03-16T09:16:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd45259c640 1 --2- 172.21.15.112:0/1710775654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf5f990 0x55d95bfd75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd451d9b640 1 --2- 172.21.15.112:0/1710775654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf60380 0x55d95bfd7b00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd448012ae0 tx=0x7fd448010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d0d06b5f06ac5c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd439ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:16:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd439ffb640 1 -- 172.21.15.112:0/1710775654 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448004040 con 0x55d95bf60380 2024-03-16T09:16:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd439ffb640 1 -- 172.21.15.112:0/1710775654 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd4480041e0 con 0x55d95bf60380 2024-03-16T09:16:14.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd45b7e6740 1 -- 172.21.15.112:0/1710775654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd44c059070 con 0x55d95bf60380 2024-03-16T09:16:14.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd439ffb640 1 -- 172.21.15.112:0/1710775654 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd448037050 con 0x55d95bf60380 2024-03-16T09:16:14.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd45b7e6740 10 client.15540 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:14.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd45b7e6740 1 -- 172.21.15.112:0/1710775654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55d95bfe0310 con 0x55d95bf60380 2024-03-16T09:16:14.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd439ffb640 1 -- 172.21.15.112:0/1710775654 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd44c059070 con 0x55d95bf60380 2024-03-16T09:16:14.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd439ffb640 1 -- 172.21.15.112:0/1710775654 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd44801c470 con 0x55d95bf60380 2024-03-16T09:16:14.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd45b7e6740 10 client.15540 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:14.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd45b7e6740 10 client.15540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:14.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd45b7e6740 20 client.15540 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:14.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/1710775654 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55d95bfe0c50 0x55d95bfe3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd45b7e6740 4 client.15540 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T09:16:14.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.462+0000 7fd45b7e6740 1 -- 172.21.15.112:0/1710775654 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d95bf69c70 con 0x55d95bfe0c50 2024-03-16T09:16:14.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.464+0000 7fd45259c640 1 --2- 172.21.15.112:0/1710775654 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55d95bfe0c50 0x55d95bfe3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.466+0000 7fd45259c640 1 --2- 172.21.15.112:0/1710775654 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55d95bfe0c50 0x55d95bfe3070 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d7ec0a6633b50b9 server_cookie=805699e069c869ef in_seq=0 out_seq=0 2024-03-16T09:16:14.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.466+0000 7fd439ffb640 10 client.15540 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:14.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.467+0000 7fd439ffb640 1 -- 172.21.15.112:0/1710775654 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d95bf69c70 con 0x55d95bfe0c50 2024-03-16T09:16:14.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.467+0000 7fd439ffb640 10 client.15540 handle_command_reply: tid=0 2024-03-16T09:16:14.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.467+0000 7fd45b7e6740 1 client.15540 shutdown 2024-03-16T09:16:14.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.467+0000 7fd45b7e6740 1 -- 172.21.15.112:0/1710775654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf60380 msgr2=0x55d95bfd7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:14.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.467+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/1710775654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf60380 0x55d95bfd7b00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd448012ae0 tx=0x7fd448010660 comp rx=0 tx=0).stop 2024-03-16T09:16:14.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.467+0000 7fd45b7e6740 1 -- 172.21.15.112:0/1710775654 shutdown_connections 2024-03-16T09:16:14.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.467+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/1710775654 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d95bf654f0 0x55d95bfdeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.467+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/1710775654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d95bf60380 0x55d95bfd7b00 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.467+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/1710775654 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55d95bfe0c50 0x55d95bfe3070 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.467+0000 7fd45b7e6740 1 --2- 172.21.15.112:0/1710775654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d95bf5f990 0x55d95bfd75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.467+0000 7fd439ffb640 0 client.15540 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:14.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.467+0000 7fd45b7e6740 1 -- 172.21.15.112:0/1710775654 >> 172.21.15.112:0/1710775654 conn(0x55d95bf27c50 msgr2=0x55d95bf69e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:14.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.467+0000 7fd45b7e6740 1 -- 172.21.15.112:0/1710775654 shutdown_connections 2024-03-16T09:16:14.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.467+0000 7fd45b7e6740 1 -- 172.21.15.112:0/1710775654 wait complete. 2024-03-16T09:16:14.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.467+0000 7fd45b7e6740 20 client.15540 trim_cache size 0 max 16384 2024-03-16T09:16:14.475 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:16:14.475 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-03-16T09:16:14.475 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "1ec61872-e2ee-4f82-a6b6-19d35a8410dd", 2024-03-16T09:16:14.475 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-03-16T09:16:14.475 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:16:14.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.468+0000 7fd458b1b640 1 -- 172.21.15.112:0/2799335866 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd43c0a2890 msgr2=0x7fd43c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:14.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.468+0000 7fd458b1b640 1 --2- 172.21.15.112:0/2799335866 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd43c0a2890 0x7fd43c0a4d50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd44c04d000 tx=0x7fd44c048000 comp rx=0 tx=0).stop 2024-03-16T09:16:14.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.468+0000 7fd458b1b640 1 -- 172.21.15.112:0/2799335866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4541017f0 msgr2=0x7fd454197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:14.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.468+0000 7fd458b1b640 1 --2- 172.21.15.112:0/2799335866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4541017f0 0x7fd454197b40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd44400e9c0 tx=0x7fd44400ee80 comp rx=0 tx=0).stop 2024-03-16T09:16:14.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.468+0000 7fd458b1b640 1 -- 172.21.15.112:0/2799335866 shutdown_connections 2024-03-16T09:16:14.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.468+0000 7fd458b1b640 1 --2- 172.21.15.112:0/2799335866 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd43c0a2890 0x7fd43c0a4d50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.468+0000 7fd458b1b640 1 --2- 172.21.15.112:0/2799335866 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4541033a0 0x7fd45419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.468+0000 7fd458b1b640 1 --2- 172.21.15.112:0/2799335866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4541029e0 0x7fd454198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.468+0000 7fd458b1b640 1 --2- 172.21.15.112:0/2799335866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4541017f0 0x7fd454197b40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.468+0000 7fd458b1b640 1 -- 172.21.15.112:0/2799335866 >> 172.21.15.112:0/2799335866 conn(0x7fd4540fcd80 msgr2=0x7fd454105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:14.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.468+0000 7fd458b1b640 1 -- 172.21.15.112:0/2799335866 shutdown_connections 2024-03-16T09:16:14.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.468+0000 7fd458b1b640 1 -- 172.21.15.112:0/2799335866 wait complete. 2024-03-16T09:16:14.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T09:16:14.481 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T09:16:14.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:13 smithi159 ceph-mon[36739]: pgmap v368: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-16T09:16:14.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.622+0000 7f1e23516640 1 -- 172.21.15.112:0/344038991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e1c104c30 msgr2=0x7f1e1c107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:14.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.623+0000 7f1e23516640 1 --2- 172.21.15.112:0/344038991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e1c104c30 0x7f1e1c107020 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009900 tx=0x7f1e1002bc30 comp rx=0 tx=0).stop 2024-03-16T09:16:14.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.623+0000 7f1e23516640 1 -- 172.21.15.112:0/344038991 shutdown_connections 2024-03-16T09:16:14.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.623+0000 7f1e23516640 1 --2- 172.21.15.112:0/344038991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e1c107560 0x7f1e1c109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.623+0000 7f1e23516640 1 --2- 172.21.15.112:0/344038991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e1c104c30 0x7f1e1c107020 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.623+0000 7f1e23516640 1 --2- 172.21.15.112:0/344038991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e1c102300 0x7f1e1c1046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.623+0000 7f1e23516640 1 -- 172.21.15.112:0/344038991 >> 172.21.15.112:0/344038991 conn(0x7f1e1c0fbc70 msgr2=0x7f1e1c0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:14.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.623+0000 7f1e23516640 1 -- 172.21.15.112:0/344038991 shutdown_connections 2024-03-16T09:16:14.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.623+0000 7f1e23516640 1 -- 172.21.15.112:0/344038991 wait complete. 2024-03-16T09:16:14.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.623+0000 7f1e23516640 1 Processor -- start 2024-03-16T09:16:14.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.623+0000 7f1e23516640 1 -- start start 2024-03-16T09:16:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.623+0000 7f1e23516640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e1c102300 0x7f1e1c197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.623+0000 7f1e23516640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e1c104c30 0x7f1e1c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.623+0000 7f1e23516640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e1c107560 0x7f1e1c19f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.623+0000 7f1e23516640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c10d560 con 0x7f1e1c102300 2024-03-16T09:16:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.623+0000 7f1e23516640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c10d3e0 con 0x7f1e1c104c30 2024-03-16T09:16:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.623+0000 7f1e23516640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c10d6e0 con 0x7f1e1c107560 2024-03-16T09:16:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.623+0000 7f1e212b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e1c102300 0x7f1e1c197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.624+0000 7f1e212b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e1c102300 0x7f1e1c197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40586/0 (socket says 172.21.15.112:40586) 2024-03-16T09:16:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.624+0000 7f1e212b2640 1 -- 172.21.15.112:0/3955152145 learned_addr learned my addr 172.21.15.112:0/3955152145 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.624+0000 7f1e20ab1640 1 --2- 172.21.15.112:0/3955152145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e1c104c30 0x7f1e1c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.624+0000 7f1e21ab3640 1 --2- 172.21.15.112:0/3955152145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e1c107560 0x7f1e1c19f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.624+0000 7f1e212b2640 1 -- 172.21.15.112:0/3955152145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e1c107560 msgr2=0x7f1e1c19f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.624+0000 7f1e212b2640 1 --2- 172.21.15.112:0/3955152145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e1c107560 0x7f1e1c19f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.624+0000 7f1e212b2640 1 -- 172.21.15.112:0/3955152145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e1c104c30 msgr2=0x7f1e1c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:14.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.624+0000 7f1e212b2640 1 --2- 172.21.15.112:0/3955152145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e1c104c30 0x7f1e1c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.624+0000 7f1e212b2640 1 -- 172.21.15.112:0/3955152145 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1c19f870 con 0x7f1e1c102300 2024-03-16T09:16:14.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.624+0000 7f1e212b2640 1 --2- 172.21.15.112:0/3955152145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e1c102300 0x7f1e1c197c60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400ebc0 tx=0x7f1e1400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:14.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.624+0000 7f1e1a7fc640 1 -- 172.21.15.112:0/3955152145 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14010630 con 0x7f1e1c102300 2024-03-16T09:16:14.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.624+0000 7f1e1a7fc640 1 -- 172.21.15.112:0/3955152145 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1e14010cd0 con 0x7f1e1c102300 2024-03-16T09:16:14.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.624+0000 7f1e23516640 1 -- 172.21.15.112:0/3955152145 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e1c19fb60 con 0x7f1e1c102300 2024-03-16T09:16:14.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.624+0000 7f1e1a7fc640 1 -- 172.21.15.112:0/3955152145 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14018cf0 con 0x7f1e1c102300 2024-03-16T09:16:14.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.624+0000 7f1e23516640 1 -- 172.21.15.112:0/3955152145 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e1c1a00b0 con 0x7f1e1c102300 2024-03-16T09:16:14.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.627+0000 7f1e261d7740 1 Processor -- start 2024-03-16T09:16:14.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.627+0000 7f1e261d7740 1 -- start start 2024-03-16T09:16:14.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.627+0000 7f1e261d7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f216170 0x55a11f216590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.627+0000 7f1e261d7740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216b60 0x55a11f21b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.627+0000 7f1e261d7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f21bcd0 0x55a11f21e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.627+0000 7f1e261d7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a11f17b500 con 0x55a11f21bcd0 2024-03-16T09:16:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.627+0000 7f1e261d7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a11f123be0 con 0x55a11f216170 2024-03-16T09:16:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.627+0000 7f1e261d7740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a11f1775a0 con 0x55a11f216b60 2024-03-16T09:16:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.627+0000 7f1e20ab1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f21bcd0 0x55a11f21e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.627+0000 7f1e20ab1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f21bcd0 0x55a11f21e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40600/0 (socket says 172.21.15.112:40600) 2024-03-16T09:16:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.627+0000 7f1e20ab1640 1 -- 172.21.15.112:0/2740597209 learned_addr learned my addr 172.21.15.112:0/2740597209 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.627+0000 7f1e21ab3640 1 --2- 172.21.15.112:0/2740597209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f216170 0x55a11f216590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.627+0000 7f1e1a7fc640 1 -- 172.21.15.112:0/3955152145 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7f1e140107d0 con 0x7f1e1c102300 2024-03-16T09:16:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.627+0000 7f1e20ab1640 1 -- 172.21.15.112:0/2740597209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216b60 msgr2=0x55a11f21b660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-16T09:16:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.627+0000 7f1e20ab1640 1 --2- 172.21.15.112:0/2740597209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216b60 0x55a11f21b660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.627+0000 7f1e20ab1640 1 -- 172.21.15.112:0/2740597209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f216170 msgr2=0x55a11f216590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:14.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.627+0000 7f1e20ab1640 1 --2- 172.21.15.112:0/2740597209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f216170 0x55a11f216590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.627+0000 7f1e20ab1640 1 -- 172.21.15.112:0/2740597209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a11f1374b0 con 0x55a11f21bcd0 2024-03-16T09:16:14.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.627+0000 7f1e21ab3640 1 --2- 172.21.15.112:0/2740597209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f216170 0x55a11f216590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:14.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.628+0000 7f1e20ab1640 1 --2- 172.21.15.112:0/2740597209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f21bcd0 0x55a11f21e160 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c069400 tx=0x7f1e10009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd217790cbe122a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:14.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.628+0000 7f1e037fe640 1 -- 172.21.15.112:0/2740597209 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1002c5d0 con 0x55a11f21bcd0 2024-03-16T09:16:14.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.628+0000 7f1e037fe640 1 -- 172.21.15.112:0/2740597209 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1e10004040 con 0x55a11f21bcd0 2024-03-16T09:16:14.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.628+0000 7f1e1a7fc640 1 --2- 172.21.15.112:0/3955152145 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1e040a2890 0x7f1e040a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.628+0000 7f1e037fe640 1 -- 172.21.15.112:0/2740597209 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e100046c0 con 0x55a11f21bcd0 2024-03-16T09:16:14.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.628+0000 7f1e21ab3640 1 --2- 172.21.15.112:0/3955152145 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1e040a2890 0x7f1e040a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.628+0000 7f1e1a7fc640 1 -- 172.21.15.112:0/3955152145 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f1e140cf2f0 con 0x7f1e1c102300 2024-03-16T09:16:14.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.629+0000 7f1e261d7740 1 -- 172.21.15.112:0/2740597209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f21bcd0 msgr2=0x55a11f21e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:14.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.629+0000 7f1e21ab3640 1 --2- 172.21.15.112:0/3955152145 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1e040a2890 0x7f1e040a4d50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55a11f2173c0 tx=0x7f1e0c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:14.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.629+0000 7f1e261d7740 1 --2- 172.21.15.112:0/2740597209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f21bcd0 0x55a11f21e160 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c069400 tx=0x7f1e10009670 comp rx=0 tx=0).stop 2024-03-16T09:16:14.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.629+0000 7f1e261d7740 1 -- 172.21.15.112:0/2740597209 shutdown_connections 2024-03-16T09:16:14.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.629+0000 7f1e261d7740 1 --2- 172.21.15.112:0/2740597209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f21bcd0 0x55a11f21e160 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.629+0000 7f1e261d7740 1 --2- 172.21.15.112:0/2740597209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216b60 0x55a11f21b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.629+0000 7f1e261d7740 1 --2- 172.21.15.112:0/2740597209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f216170 0x55a11f216590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.629+0000 7f1e261d7740 1 -- 172.21.15.112:0/2740597209 >> 172.21.15.112:0/2740597209 conn(0x55a11f2116e0 msgr2=0x55a11f211ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:14.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.629+0000 7f1e261d7740 1 -- 172.21.15.112:0/2740597209 shutdown_connections 2024-03-16T09:16:14.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.629+0000 7f1e261d7740 1 -- 172.21.15.112:0/2740597209 wait complete. 2024-03-16T09:16:14.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.629+0000 7f1e261d7740 1 -- start start 2024-03-16T09:16:14.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.629+0000 7f1e261d7740 1 Processor -- start 2024-03-16T09:16:14.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.629+0000 7f1e261d7740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216170 0x55a11f220650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.629+0000 7f1e261d7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216b60 0x55a11f220b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.629+0000 7f1e261d7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f21bcd0 0x55a11f227bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.629+0000 7f1e261d7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a11f21ec20 con 0x55a11f216b60 2024-03-16T09:16:14.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.629+0000 7f1e261d7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a11f137160 con 0x55a11f21bcd0 2024-03-16T09:16:14.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.629+0000 7f1e261d7740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a11f21eda0 con 0x55a11f216170 2024-03-16T09:16:14.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.629+0000 7f1e261d7740 10 client.0 _ll_register_callbacks cb 0x55a11f11b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:14.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.630+0000 7f1e20ab1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216b60 0x55a11f220b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.630+0000 7f1e20ab1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216b60 0x55a11f220b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40612/0 (socket says 172.21.15.112:40612) 2024-03-16T09:16:14.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.630+0000 7f1e20ab1640 1 -- 172.21.15.112:0/3965195101 learned_addr learned my addr 172.21.15.112:0/3965195101 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:14.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.630+0000 7f1e212b2640 1 --2- 172.21.15.112:0/3965195101 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216170 0x55a11f220650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.630+0000 7f1e21ab3640 1 --2- 172.21.15.112:0/3965195101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f21bcd0 0x55a11f227bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.630+0000 7f1e20ab1640 1 -- 172.21.15.112:0/3965195101 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216170 msgr2=0x55a11f220650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:14.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.630+0000 7f1e20ab1640 1 --2- 172.21.15.112:0/3965195101 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216170 0x55a11f220650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.630+0000 7f1e20ab1640 1 -- 172.21.15.112:0/3965195101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f21bcd0 msgr2=0x55a11f227bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:14.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.630+0000 7f1e20ab1640 1 --2- 172.21.15.112:0/3965195101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f21bcd0 0x55a11f227bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.630+0000 7f1e20ab1640 1 -- 172.21.15.112:0/3965195101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a11ef597d0 con 0x55a11f216b60 2024-03-16T09:16:14.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.630+0000 7f1e20ab1640 1 --2- 172.21.15.112:0/3965195101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216b60 0x55a11f220b90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1e1002c170 tx=0x7f1e100041f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811040709fae81f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:14.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.630+0000 7f1e00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:16:14.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.630+0000 7f1e00ff9640 1 -- 172.21.15.112:0/3965195101 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1004f070 con 0x55a11f216b60 2024-03-16T09:16:14.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.630+0000 7f1e00ff9640 1 -- 172.21.15.112:0/3965195101 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f1e1002ccd0 con 0x55a11f216b60 2024-03-16T09:16:14.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.630+0000 7f1e261d7740 1 -- 172.21.15.112:0/3965195101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e10004040 con 0x55a11f216b60 2024-03-16T09:16:14.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.630+0000 7f1e00ff9640 1 -- 172.21.15.112:0/3965195101 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1e1005b050 con 0x55a11f216b60 2024-03-16T09:16:14.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.630+0000 7f1e261d7740 10 client.15552 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:14.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.630+0000 7f1e261d7740 1 -- 172.21.15.112:0/3965195101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a11ee8f3f0 con 0x55a11f216b60 2024-03-16T09:16:14.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.630+0000 7f1e00ff9640 1 -- 172.21.15.112:0/3965195101 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1e10004040 con 0x55a11f216b60 2024-03-16T09:16:14.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.631+0000 7f1e00ff9640 1 -- 172.21.15.112:0/3965195101 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1e10034840 con 0x55a11f216b60 2024-03-16T09:16:14.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.631+0000 7f1e261d7740 10 client.15552 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:14.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.631+0000 7f1e261d7740 10 client.15552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:14.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.631+0000 7f1e261d7740 20 client.15552 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:14.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.631+0000 7f1e261d7740 1 --2- 172.21.15.112:0/3965195101 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55a11f22bd30 0x55a11f22e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.631+0000 7f1e261d7740 4 client.15552 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:16:14.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.631+0000 7f1e261d7740 1 -- 172.21.15.112:0/3965195101 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a11f188a40 con 0x55a11f22bd30 2024-03-16T09:16:14.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.631+0000 7f1e212b2640 1 --2- 172.21.15.112:0/3965195101 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55a11f22bd30 0x55a11f22e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.631+0000 7f1e00ff9640 10 client.15552 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:14.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.631+0000 7f1e212b2640 1 --2- 172.21.15.112:0/3965195101 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55a11f22bd30 0x55a11f22e120 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=662917f58f253e80 server_cookie=ab66e85bb4a8b9f4 in_seq=0 out_seq=0 2024-03-16T09:16:14.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.632+0000 7f1e00ff9640 1 -- 172.21.15.112:0/3965195101 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a11f188a40 con 0x55a11f22bd30 2024-03-16T09:16:14.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.632+0000 7f1e00ff9640 10 client.15552 handle_command_reply: tid=0 2024-03-16T09:16:14.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.632+0000 7f1e261d7740 1 client.15552 shutdown 2024-03-16T09:16:14.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.632+0000 7f1e261d7740 1 -- 172.21.15.112:0/3965195101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216b60 msgr2=0x55a11f220b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:14.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.632+0000 7f1e261d7740 1 --2- 172.21.15.112:0/3965195101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216b60 0x55a11f220b90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1e1002c170 tx=0x7f1e100041f0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.632+0000 7f1e261d7740 1 -- 172.21.15.112:0/3965195101 shutdown_connections 2024-03-16T09:16:14.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.632+0000 7f1e261d7740 1 --2- 172.21.15.112:0/3965195101 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55a11f22bd30 0x55a11f22e120 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.632+0000 7f1e261d7740 1 --2- 172.21.15.112:0/3965195101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f21bcd0 0x55a11f227bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.632+0000 7f1e00ff9640 0 client.15552 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:14.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.632+0000 7f1e261d7740 1 --2- 172.21.15.112:0/3965195101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216b60 0x55a11f220b90 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.632+0000 7f1e261d7740 1 --2- 172.21.15.112:0/3965195101 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216170 0x55a11f220650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.632+0000 7f1e261d7740 1 -- 172.21.15.112:0/3965195101 >> 172.21.15.112:0/3965195101 conn(0x55a11f2116e0 msgr2=0x55a11f210900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:14.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.632+0000 7f1e261d7740 1 -- 172.21.15.112:0/3965195101 shutdown_connections 2024-03-16T09:16:14.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.632+0000 7f1e261d7740 1 -- 172.21.15.112:0/3965195101 wait complete. 2024-03-16T09:16:14.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.632+0000 7f1e261d7740 20 client.15552 trim_cache size 0 max 16384 2024-03-16T09:16:14.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.649+0000 7f1e261d7740 1 Processor -- start 2024-03-16T09:16:14.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.649+0000 7f1e261d7740 1 -- start start 2024-03-16T09:16:14.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.649+0000 7f1e261d7740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216170 0x55a11f218060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.649+0000 7f1e261d7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216b60 0x55a11f2185a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.649+0000 7f1e261d7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f21bcd0 0x55a11f227b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.649+0000 7f1e261d7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1ab600 con 0x55a11f216b60 2024-03-16T09:16:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.649+0000 7f1e261d7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1a8ed0 con 0x55a11f21bcd0 2024-03-16T09:16:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.649+0000 7f1e261d7740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1a6d20 con 0x55a11f216170 2024-03-16T09:16:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.649+0000 7f1e20ab1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216b60 0x55a11f2185a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.649+0000 7f1e20ab1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216b60 0x55a11f2185a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40622/0 (socket says 172.21.15.112:40622) 2024-03-16T09:16:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.649+0000 7f1e20ab1640 1 -- 172.21.15.112:0/598713736 learned_addr learned my addr 172.21.15.112:0/598713736 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.649+0000 7f1e212b2640 1 --2- 172.21.15.112:0/598713736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216170 0x55a11f218060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.649+0000 7f1e21ab3640 1 --2- 172.21.15.112:0/598713736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f21bcd0 0x55a11f227b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.650+0000 7f1e20ab1640 1 -- 172.21.15.112:0/598713736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216170 msgr2=0x55a11f218060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.650+0000 7f1e20ab1640 1 --2- 172.21.15.112:0/598713736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216170 0x55a11f218060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.650+0000 7f1e20ab1640 1 -- 172.21.15.112:0/598713736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f21bcd0 msgr2=0x55a11f227b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.650+0000 7f1e20ab1640 1 --2- 172.21.15.112:0/598713736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f21bcd0 0x55a11f227b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.650+0000 7f1e20ab1640 1 -- 172.21.15.112:0/598713736 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a11ee4f0f0 con 0x55a11f216b60 2024-03-16T09:16:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.650+0000 7f1e20ab1640 1 --2- 172.21.15.112:0/598713736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216b60 0x55a11f2185a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009900 tx=0x7f1e10004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8f9a12695bc6cf server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:14.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.650+0000 7f1e02ffd640 1 -- 172.21.15.112:0/598713736 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1004f070 con 0x55a11f216b60 2024-03-16T09:16:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.650+0000 7f1e02ffd640 1 -- 172.21.15.112:0/598713736 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1e100349e0 con 0x55a11f216b60 2024-03-16T09:16:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.650+0000 7f1e02ffd640 1 -- 172.21.15.112:0/598713736 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e100039e0 con 0x55a11f216b60 2024-03-16T09:16:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.650+0000 7f1e261d7740 1 -- 172.21.15.112:0/598713736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216b60 msgr2=0x55a11f2185a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e261d7740 1 --2- 172.21.15.112:0/598713736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216b60 0x55a11f2185a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009900 tx=0x7f1e10004260 comp rx=0 tx=0).stop 2024-03-16T09:16:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e261d7740 1 -- 172.21.15.112:0/598713736 shutdown_connections 2024-03-16T09:16:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e261d7740 1 --2- 172.21.15.112:0/598713736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f21bcd0 0x55a11f227b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e261d7740 1 --2- 172.21.15.112:0/598713736 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216b60 0x55a11f2185a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e261d7740 1 --2- 172.21.15.112:0/598713736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216170 0x55a11f218060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e261d7740 1 -- 172.21.15.112:0/598713736 >> 172.21.15.112:0/598713736 conn(0x55a11f2116e0 msgr2=0x55a11f21f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e261d7740 1 -- 172.21.15.112:0/598713736 shutdown_connections 2024-03-16T09:16:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e261d7740 1 -- 172.21.15.112:0/598713736 wait complete. 2024-03-16T09:16:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e261d7740 1 -- start start 2024-03-16T09:16:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e261d7740 1 Processor -- start 2024-03-16T09:16:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e261d7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216170 0x55a11f225560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e261d7740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216b60 0x55a11f225aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e261d7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f21bcd0 0x55a11f226ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e261d7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1a7390 con 0x55a11f216170 2024-03-16T09:16:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e261d7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1a8800 con 0x55a11f21bcd0 2024-03-16T09:16:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e261d7740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1a7a60 con 0x55a11f216b60 2024-03-16T09:16:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e261d7740 10 client.0 _ll_register_callbacks cb 0x7f1e0c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e212b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216170 0x55a11f225560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e212b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216170 0x55a11f225560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40630/0 (socket says 172.21.15.112:40630) 2024-03-16T09:16:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e212b2640 1 -- 172.21.15.112:0/1893353496 learned_addr learned my addr 172.21.15.112:0/1893353496 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e20ab1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216b60 0x55a11f225aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.651+0000 7f1e21ab3640 1 --2- 172.21.15.112:0/1893353496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f21bcd0 0x55a11f226ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e212b2640 1 -- 172.21.15.112:0/1893353496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216b60 msgr2=0x55a11f225aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:14.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e212b2640 1 --2- 172.21.15.112:0/1893353496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216b60 0x55a11f225aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e212b2640 1 -- 172.21.15.112:0/1893353496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f21bcd0 msgr2=0x55a11f226ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:14.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e212b2640 1 --2- 172.21.15.112:0/1893353496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f21bcd0 0x55a11f226ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e212b2640 1 -- 172.21.15.112:0/1893353496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a11f1429d0 con 0x55a11f216170 2024-03-16T09:16:14.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e20ab1640 1 --2- 172.21.15.112:0/1893353496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216b60 0x55a11f225aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:14.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e212b2640 1 --2- 172.21.15.112:0/1893353496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216170 0x55a11f225560 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1e140ccce0 tx=0x7f1e1400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df581bc4a5c5cf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:14.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:16:14.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e00ff9640 1 -- 172.21.15.112:0/1893353496 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e140b9070 con 0x55a11f216170 2024-03-16T09:16:14.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e00ff9640 1 -- 172.21.15.112:0/1893353496 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f1e140981d0 con 0x55a11f216170 2024-03-16T09:16:14.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e261d7740 1 -- 172.21.15.112:0/1893353496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e100349e0 con 0x55a11f216170 2024-03-16T09:16:14.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e00ff9640 1 -- 172.21.15.112:0/1893353496 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1e1409bbf0 con 0x55a11f216170 2024-03-16T09:16:14.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e261d7740 10 client.15558 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:14.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e261d7740 1 -- 172.21.15.112:0/1893353496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55a11f292140 con 0x55a11f216170 2024-03-16T09:16:14.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e00ff9640 1 -- 172.21.15.112:0/1893353496 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1e100349e0 con 0x55a11f216170 2024-03-16T09:16:14.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e00ff9640 1 -- 172.21.15.112:0/1893353496 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1e1409b040 con 0x55a11f216170 2024-03-16T09:16:14.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e261d7740 10 client.15558 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:14.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e261d7740 10 client.15558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:14.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e261d7740 20 client.15558 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:14.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e261d7740 1 --2- 172.21.15.112:0/1893353496 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55a11f22bd30 0x55a11f294aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:14.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e261d7740 4 client.15558 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T09:16:14.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.652+0000 7f1e261d7740 1 -- 172.21.15.112:0/1893353496 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a11f220440 con 0x55a11f22bd30 2024-03-16T09:16:14.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.653+0000 7f1e20ab1640 1 --2- 172.21.15.112:0/1893353496 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55a11f22bd30 0x55a11f294aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:14.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.653+0000 7f1e00ff9640 10 client.15558 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:14.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.653+0000 7f1e20ab1640 1 --2- 172.21.15.112:0/1893353496 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55a11f22bd30 0x55a11f294aa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ce267d4b4956872 server_cookie=cdf7038bf03b0e3a in_seq=0 out_seq=0 2024-03-16T09:16:14.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.653+0000 7f1e00ff9640 1 -- 172.21.15.112:0/1893353496 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a11f220440 con 0x55a11f22bd30 2024-03-16T09:16:14.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.653+0000 7f1e00ff9640 10 client.15558 handle_command_reply: tid=0 2024-03-16T09:16:14.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.653+0000 7f1e261d7740 1 client.15558 shutdown 2024-03-16T09:16:14.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.653+0000 7f1e261d7740 1 -- 172.21.15.112:0/1893353496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216170 msgr2=0x55a11f225560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:14.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.653+0000 7f1e261d7740 1 --2- 172.21.15.112:0/1893353496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216170 0x55a11f225560 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1e140ccce0 tx=0x7f1e1400ef50 comp rx=0 tx=0).stop 2024-03-16T09:16:14.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.653+0000 7f1e261d7740 1 -- 172.21.15.112:0/1893353496 shutdown_connections 2024-03-16T09:16:14.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.653+0000 7f1e261d7740 1 --2- 172.21.15.112:0/1893353496 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55a11f22bd30 0x55a11f294aa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.653+0000 7f1e261d7740 1 --2- 172.21.15.112:0/1893353496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a11f21bcd0 0x55a11f226ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.653+0000 7f1e261d7740 1 --2- 172.21.15.112:0/1893353496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a11f216b60 0x55a11f225aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.653+0000 7f1e261d7740 1 --2- 172.21.15.112:0/1893353496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a11f216170 0x55a11f225560 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.653+0000 7f1e00ff9640 0 client.15558 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:14.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.653+0000 7f1e261d7740 1 -- 172.21.15.112:0/1893353496 >> 172.21.15.112:0/1893353496 conn(0x55a11f2116e0 msgr2=0x55a11f21f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:14.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.653+0000 7f1e261d7740 1 -- 172.21.15.112:0/1893353496 shutdown_connections 2024-03-16T09:16:14.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.653+0000 7f1e261d7740 1 -- 172.21.15.112:0/1893353496 wait complete. 2024-03-16T09:16:14.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.653+0000 7f1e261d7740 20 client.15558 trim_cache size 0 max 16384 2024-03-16T09:16:14.668 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:16:14.668 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T09:16:14.668 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-03-16T09:16:14.668 INFO:teuthology.orchestra.run.smithi112.stdout: "1ec61872-e2ee-4f82-a6b6-19d35a8410dd": { 2024-03-16T09:16:14.669 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-03-16T09:16:14.669 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "1ec61872-e2ee-4f82-a6b6-19d35a8410dd", 2024-03-16T09:16:14.669 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-03-16T09:16:14.669 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:16:14.669 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:16:14.669 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:16:14.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.654+0000 7f1e23516640 1 -- 172.21.15.112:0/3955152145 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1e040a2890 msgr2=0x7f1e040a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:14.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.654+0000 7f1e23516640 1 --2- 172.21.15.112:0/3955152145 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1e040a2890 0x7f1e040a4d50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55a11f2173c0 tx=0x7f1e0c005f90 comp rx=0 tx=0).stop 2024-03-16T09:16:14.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.654+0000 7f1e23516640 1 -- 172.21.15.112:0/3955152145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e1c102300 msgr2=0x7f1e1c197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:14.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.654+0000 7f1e23516640 1 --2- 172.21.15.112:0/3955152145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e1c102300 0x7f1e1c197c60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400ebc0 tx=0x7f1e1400c6a0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.654+0000 7f1e23516640 1 -- 172.21.15.112:0/3955152145 shutdown_connections 2024-03-16T09:16:14.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.654+0000 7f1e23516640 1 --2- 172.21.15.112:0/3955152145 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f1e040a2890 0x7f1e040a4d50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.654+0000 7f1e23516640 1 --2- 172.21.15.112:0/3955152145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e1c107560 0x7f1e1c19f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.654+0000 7f1e23516640 1 --2- 172.21.15.112:0/3955152145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e1c104c30 0x7f1e1c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.654+0000 7f1e23516640 1 --2- 172.21.15.112:0/3955152145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e1c102300 0x7f1e1c197c60 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:14.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.654+0000 7f1e23516640 1 -- 172.21.15.112:0/3955152145 >> 172.21.15.112:0/3955152145 conn(0x7f1e1c0fbc70 msgr2=0x7f1e1c102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:14.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.654+0000 7f1e23516640 1 -- 172.21.15.112:0/3955152145 shutdown_connections 2024-03-16T09:16:14.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:14.654+0000 7f1e23516640 1 -- 172.21.15.112:0/3955152145 wait complete. 2024-03-16T09:16:14.672 INFO:tasks.cephfs.filesystem:scrub status for tag:1ec61872-e2ee-4f82-a6b6-19d35a8410dd - {'path': '/', 'tag': '1ec61872-e2ee-4f82-a6b6-19d35a8410dd', 'options': 'recursive,force'} 2024-03-16T09:16:16.288 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:15 smithi159 ceph-mon[36739]: scrub queued for path: / 2024-03-16T09:16:16.289 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:15 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:16:16.289 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:15 smithi159 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-16T09:16:16.289 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:15 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:16:16.289 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:15 smithi159 ceph-mon[36739]: pgmap v369: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T09:16:16.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:15 smithi112 ceph-mon[32307]: scrub queued for path: / 2024-03-16T09:16:16.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:15 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:16:16.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:15 smithi112 ceph-mon[32307]: scrub summary: active paths [/] 2024-03-16T09:16:16.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:15 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:16:16.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:15 smithi112 ceph-mon[32307]: pgmap v369: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T09:16:16.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:15 smithi194 ceph-mon[36729]: scrub queued for path: / 2024-03-16T09:16:16.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:15 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:16:16.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:15 smithi194 ceph-mon[36729]: scrub summary: active paths [/] 2024-03-16T09:16:16.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:15 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:16:16.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:15 smithi194 ceph-mon[36729]: pgmap v369: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T09:16:17.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:16 smithi112 ceph-mon[32307]: scrub summary: idle 2024-03-16T09:16:17.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:17 smithi194 ceph-mon[36729]: scrub summary: idle 2024-03-16T09:16:17.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:17 smithi159 ceph-mon[36739]: scrub summary: idle 2024-03-16T09:16:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:18 smithi112 ceph-mon[32307]: pgmap v370: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-16T09:16:18.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:18 smithi194 ceph-mon[36729]: pgmap v370: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-16T09:16:18.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:18 smithi159 ceph-mon[36739]: pgmap v370: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-16T09:16:20.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:20 smithi112 ceph-mon[32307]: pgmap v371: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-16T09:16:20.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:20 smithi194 ceph-mon[36729]: pgmap v371: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-16T09:16:20.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:20 smithi159 ceph-mon[36739]: pgmap v371: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-16T09:16:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.449+0000 7f8b85989640 1 -- 172.21.15.112:0/3321544041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b80100700 msgr2=0x7f8b80100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.449+0000 7f8b85989640 1 --2- 172.21.15.112:0/3321544041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b80100700 0x7f8b80100b20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009860 tx=0x7f8b7002bb70 comp rx=0 tx=0).stop 2024-03-16T09:16:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.449+0000 7f8b85989640 1 -- 172.21.15.112:0/3321544041 shutdown_connections 2024-03-16T09:16:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.449+0000 7f8b85989640 1 --2- 172.21.15.112:0/3321544041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b801022b0 0x7f8b80108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.449+0000 7f8b85989640 1 --2- 172.21.15.112:0/3321544041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b801018f0 0x7f8b80101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.449+0000 7f8b85989640 1 --2- 172.21.15.112:0/3321544041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b80100700 0x7f8b80100b20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.449+0000 7f8b85989640 1 -- 172.21.15.112:0/3321544041 >> 172.21.15.112:0/3321544041 conn(0x7f8b800fbc70 msgr2=0x7f8b800fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.449+0000 7f8b85989640 1 -- 172.21.15.112:0/3321544041 shutdown_connections 2024-03-16T09:16:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.449+0000 7f8b85989640 1 -- 172.21.15.112:0/3321544041 wait complete. 2024-03-16T09:16:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.449+0000 7f8b85989640 1 Processor -- start 2024-03-16T09:16:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b85989640 1 -- start start 2024-03-16T09:16:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b85989640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b80100700 0x7f8b80197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b85989640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b801018f0 0x7f8b801980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b85989640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b801022b0 0x7f8b8019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b85989640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8010c4b0 con 0x7f8b80100700 2024-03-16T09:16:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b85989640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8010c330 con 0x7f8b801018f0 2024-03-16T09:16:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b85989640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8010c630 con 0x7f8b801022b0 2024-03-16T09:16:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b7effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b80100700 0x7f8b80197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b7effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b80100700 0x7f8b80197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34722/0 (socket says 172.21.15.112:34722) 2024-03-16T09:16:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b7effd640 1 -- 172.21.15.112:0/1730836014 learned_addr learned my addr 172.21.15.112:0/1730836014 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b7f7fe640 1 --2- 172.21.15.112:0/1730836014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b801022b0 0x7f8b8019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b77fff640 1 --2- 172.21.15.112:0/1730836014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b801018f0 0x7f8b801980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b7effd640 1 -- 172.21.15.112:0/1730836014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b801022b0 msgr2=0x7f8b8019f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b7effd640 1 --2- 172.21.15.112:0/1730836014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b801022b0 0x7f8b8019f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b7effd640 1 -- 172.21.15.112:0/1730836014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b801018f0 msgr2=0x7f8b801980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b7effd640 1 --2- 172.21.15.112:0/1730836014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b801018f0 0x7f8b801980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b7effd640 1 -- 172.21.15.112:0/1730836014 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8019f7d0 con 0x7f8b80100700 2024-03-16T09:16:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b7f7fe640 1 --2- 172.21.15.112:0/1730836014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b801022b0 0x7f8b8019f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b77fff640 1 --2- 172.21.15.112:0/1730836014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b801018f0 0x7f8b801980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.450+0000 7f8b7effd640 1 --2- 172.21.15.112:0/1730836014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b80100700 0x7f8b80197bb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8b70034a80 tx=0x7f8b70035f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.451+0000 7f8b84987640 1 -- 172.21.15.112:0/1730836014 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70041020 con 0x7f8b80100700 2024-03-16T09:16:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.451+0000 7f8b85989640 1 -- 172.21.15.112:0/1730836014 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b8019fa60 con 0x7f8b80100700 2024-03-16T09:16:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.451+0000 7f8b84987640 1 -- 172.21.15.112:0/1730836014 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b7002ca10 con 0x7f8b80100700 2024-03-16T09:16:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.451+0000 7f8b84987640 1 -- 172.21.15.112:0/1730836014 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70039040 con 0x7f8b80100700 2024-03-16T09:16:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.451+0000 7f8b85989640 1 -- 172.21.15.112:0/1730836014 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b8019ff50 con 0x7f8b80100700 2024-03-16T09:16:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.453+0000 7f8b84987640 1 -- 172.21.15.112:0/1730836014 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7f8b70044020 con 0x7f8b80100700 2024-03-16T09:16:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.453+0000 7f8b84987640 1 --2- 172.21.15.112:0/1730836014 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8b600a2940 0x7f8b600a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.454+0000 7f8b77fff640 1 --2- 172.21.15.112:0/1730836014 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8b600a2940 0x7f8b600a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.454+0000 7f8b84987640 1 -- 172.21.15.112:0/1730836014 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8b700f0350 con 0x7f8b80100700 2024-03-16T09:16:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.454+0000 7f8b8857b740 1 Processor -- start 2024-03-16T09:16:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.454+0000 7f8b8857b740 1 -- start start 2024-03-16T09:16:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.454+0000 7f8b77fff640 1 --2- 172.21.15.112:0/1730836014 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8b600a2940 0x7f8b600a4e00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8b801990c0 tx=0x7f8b68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.454+0000 7f8b8857b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb0340 0x560e26bb0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.454+0000 7f8b8857b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bb0d30 0x560e26bb5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.454+0000 7f8b8857b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb5ea0 0x560e26bb8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.454+0000 7f8b8857b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560e26b15a40 con 0x560e26bb5ea0 2024-03-16T09:16:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.454+0000 7f8b8857b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560e26abe920 con 0x560e26bb0d30 2024-03-16T09:16:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.454+0000 7f8b8857b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560e26b11ae0 con 0x560e26bb0340 2024-03-16T09:16:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.454+0000 7f8b7f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb5ea0 0x560e26bb8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.454+0000 7f8b7f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb5ea0 0x560e26bb8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34736/0 (socket says 172.21.15.112:34736) 2024-03-16T09:16:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.454+0000 7f8b77fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bb0d30 0x560e26bb5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.454+0000 7f8b7f7fe640 1 -- 172.21.15.112:0/2983520766 learned_addr learned my addr 172.21.15.112:0/2983520766 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.454+0000 7f8b7effd640 1 --2- 172.21.15.112:0/2983520766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb0340 0x560e26bb0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.455+0000 7f8b7f7fe640 1 -- 172.21.15.112:0/2983520766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb0340 msgr2=0x560e26bb0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.455+0000 7f8b7f7fe640 1 --2- 172.21.15.112:0/2983520766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb0340 0x560e26bb0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.455+0000 7f8b7f7fe640 1 -- 172.21.15.112:0/2983520766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bb0d30 msgr2=0x560e26bb5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.455+0000 7f8b7f7fe640 1 --2- 172.21.15.112:0/2983520766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bb0d30 0x560e26bb5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.455+0000 7f8b7f7fe640 1 -- 172.21.15.112:0/2983520766 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e26ad6460 con 0x560e26bb5ea0 2024-03-16T09:16:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.455+0000 7f8b77fff640 1 --2- 172.21.15.112:0/2983520766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bb0d30 0x560e26bb5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.455+0000 7f8b7f7fe640 1 --2- 172.21.15.112:0/2983520766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb5ea0 0x560e26bb8330 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009fd0 tx=0x7f8b7800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92eacc9c9ebc979 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.455+0000 7f8b75ffb640 1 -- 172.21.15.112:0/2983520766 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7801d070 con 0x560e26bb5ea0 2024-03-16T09:16:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.455+0000 7f8b75ffb640 1 -- 172.21.15.112:0/2983520766 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b7800cce0 con 0x560e26bb5ea0 2024-03-16T09:16:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.455+0000 7f8b75ffb640 1 -- 172.21.15.112:0/2983520766 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b780183f0 con 0x560e26bb5ea0 2024-03-16T09:16:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.456+0000 7f8b8857b740 1 -- 172.21.15.112:0/2983520766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb5ea0 msgr2=0x560e26bb8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.456+0000 7f8b8857b740 1 --2- 172.21.15.112:0/2983520766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb5ea0 0x560e26bb8330 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009fd0 tx=0x7f8b7800ec80 comp rx=0 tx=0).stop 2024-03-16T09:16:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.456+0000 7f8b8857b740 1 -- 172.21.15.112:0/2983520766 shutdown_connections 2024-03-16T09:16:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.456+0000 7f8b8857b740 1 --2- 172.21.15.112:0/2983520766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb5ea0 0x560e26bb8330 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.456+0000 7f8b8857b740 1 --2- 172.21.15.112:0/2983520766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bb0d30 0x560e26bb5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.456+0000 7f8b8857b740 1 --2- 172.21.15.112:0/2983520766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb0340 0x560e26bb0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.456+0000 7f8b8857b740 1 -- 172.21.15.112:0/2983520766 >> 172.21.15.112:0/2983520766 conn(0x560e26bab8b0 msgr2=0x560e26babcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.456+0000 7f8b8857b740 1 -- 172.21.15.112:0/2983520766 shutdown_connections 2024-03-16T09:16:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.456+0000 7f8b8857b740 1 -- 172.21.15.112:0/2983520766 wait complete. 2024-03-16T09:16:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.457+0000 7f8b8857b740 1 -- start start 2024-03-16T09:16:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.457+0000 7f8b8857b740 1 Processor -- start 2024-03-16T09:16:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.457+0000 7f8b8857b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb0340 0x560e26bba630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.457+0000 7f8b8857b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bb0d30 0x560e26bbab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.457+0000 7f8b8857b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb5ea0 0x560e26bc1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.457+0000 7f8b8857b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560e26bb8e50 con 0x560e26bb0340 2024-03-16T09:16:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.457+0000 7f8b8857b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560e26ad1d70 con 0x560e26bb0d30 2024-03-16T09:16:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.457+0000 7f8b8857b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560e26bb8fd0 con 0x560e26bb5ea0 2024-03-16T09:16:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.457+0000 7f8b7effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb0340 0x560e26bba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.457+0000 7f8b7effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb0340 0x560e26bba630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34748/0 (socket says 172.21.15.112:34748) 2024-03-16T09:16:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.457+0000 7f8b7effd640 1 -- 172.21.15.112:0/290705728 learned_addr learned my addr 172.21.15.112:0/290705728 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.457+0000 7f8b8857b740 10 client.0 _ll_register_callbacks cb 0x560e26829a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.457+0000 7f8b7f7fe640 1 --2- 172.21.15.112:0/290705728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb5ea0 0x560e26bc1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.457+0000 7f8b77fff640 1 --2- 172.21.15.112:0/290705728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bb0d30 0x560e26bbab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.458+0000 7f8b7effd640 1 -- 172.21.15.112:0/290705728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb5ea0 msgr2=0x560e26bc1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.458+0000 7f8b7effd640 1 --2- 172.21.15.112:0/290705728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb5ea0 0x560e26bc1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.458+0000 7f8b7effd640 1 -- 172.21.15.112:0/290705728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bb0d30 msgr2=0x560e26bbab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.458+0000 7f8b7effd640 1 --2- 172.21.15.112:0/290705728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bb0d30 0x560e26bbab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.458+0000 7f8b7effd640 1 -- 172.21.15.112:0/290705728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e2682a210 con 0x560e26bb0340 2024-03-16T09:16:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.458+0000 7f8b77fff640 1 --2- 172.21.15.112:0/290705728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bb0d30 0x560e26bbab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.458+0000 7f8b7effd640 1 --2- 172.21.15.112:0/290705728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb0340 0x560e26bba630 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8b700bc9b0 tx=0x7f8b700bc6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dabf61ed2a7c485 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.458+0000 7f8b6effd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:16:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.458+0000 7f8b6effd640 1 -- 172.21.15.112:0/290705728 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b700da060 con 0x560e26bb0340 2024-03-16T09:16:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.458+0000 7f8b6effd640 1 -- 172.21.15.112:0/290705728 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8b70032640 con 0x560e26bb0340 2024-03-16T09:16:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.458+0000 7f8b8857b740 1 -- 172.21.15.112:0/290705728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b7800cce0 con 0x560e26bb0340 2024-03-16T09:16:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.458+0000 7f8b6effd640 1 -- 172.21.15.112:0/290705728 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f8b700c0e00 con 0x560e26bb0340 2024-03-16T09:16:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.458+0000 7f8b8857b740 10 client.15570 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.458+0000 7f8b8857b740 1 -- 172.21.15.112:0/290705728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560e268f6620 con 0x560e26bb0340 2024-03-16T09:16:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.458+0000 7f8b6effd640 1 -- 172.21.15.112:0/290705728 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b7800cce0 con 0x560e26bb0340 2024-03-16T09:16:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.458+0000 7f8b6effd640 1 -- 172.21.15.112:0/290705728 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f8b7003c030 con 0x560e26bb0340 2024-03-16T09:16:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.459+0000 7f8b8857b740 10 client.15570 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.459+0000 7f8b8857b740 10 client.15570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.459+0000 7f8b8857b740 20 client.15570 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.459+0000 7f8b8857b740 1 --2- 172.21.15.112:0/290705728 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560e26bc66e0 0x560e26bc8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.459+0000 7f8b8857b740 4 client.15570 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:16:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.459+0000 7f8b8857b740 1 -- 172.21.15.112:0/290705728 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e26b22f80 con 0x560e26bc66e0 2024-03-16T09:16:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.459+0000 7f8b77fff640 1 --2- 172.21.15.112:0/290705728 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560e26bc66e0 0x560e26bc8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.459+0000 7f8b77fff640 1 --2- 172.21.15.112:0/290705728 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560e26bc66e0 0x560e26bc8b80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9ec97466fd27c6 server_cookie=150f41de651e3d12 in_seq=0 out_seq=0 2024-03-16T09:16:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.459+0000 7f8b6effd640 10 client.15570 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.460+0000 7f8b6effd640 1 -- 172.21.15.112:0/290705728 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560e26b22f80 con 0x560e26bc66e0 2024-03-16T09:16:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.460+0000 7f8b6effd640 10 client.15570 handle_command_reply: tid=0 2024-03-16T09:16:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.460+0000 7f8b8857b740 1 client.15570 shutdown 2024-03-16T09:16:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.460+0000 7f8b8857b740 1 -- 172.21.15.112:0/290705728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb0340 msgr2=0x560e26bba630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.460+0000 7f8b8857b740 1 --2- 172.21.15.112:0/290705728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb0340 0x560e26bba630 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8b700bc9b0 tx=0x7f8b700bc6d0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.460+0000 7f8b8857b740 1 -- 172.21.15.112:0/290705728 shutdown_connections 2024-03-16T09:16:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.460+0000 7f8b8857b740 1 --2- 172.21.15.112:0/290705728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb5ea0 0x560e26bc1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.460+0000 7f8b8857b740 1 --2- 172.21.15.112:0/290705728 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560e26bc66e0 0x560e26bc8b80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.460+0000 7f8b8857b740 1 --2- 172.21.15.112:0/290705728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bb0d30 0x560e26bbab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.460+0000 7f8b6effd640 0 client.15570 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.460+0000 7f8b8857b740 1 --2- 172.21.15.112:0/290705728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb0340 0x560e26bba630 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.460+0000 7f8b8857b740 1 -- 172.21.15.112:0/290705728 >> 172.21.15.112:0/290705728 conn(0x560e26bab8b0 msgr2=0x560e26baabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.460+0000 7f8b8857b740 1 -- 172.21.15.112:0/290705728 shutdown_connections 2024-03-16T09:16:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.460+0000 7f8b8857b740 1 -- 172.21.15.112:0/290705728 wait complete. 2024-03-16T09:16:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.460+0000 7f8b8857b740 20 client.15570 trim_cache size 0 max 16384 2024-03-16T09:16:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.475+0000 7f8b8857b740 1 Processor -- start 2024-03-16T09:16:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.475+0000 7f8b8857b740 1 -- start start 2024-03-16T09:16:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.475+0000 7f8b8857b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb0d30 0x560e26ba6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.475+0000 7f8b8857b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb5ea0 0x560e26bc1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.475+0000 7f8b8857b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bc66e0 0x560e26bba240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.475+0000 7f8b8857b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8b801a93b0 con 0x560e26bb5ea0 2024-03-16T09:16:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.475+0000 7f8b8857b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8b801a6c80 con 0x560e26bc66e0 2024-03-16T09:16:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.475+0000 7f8b8857b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8b801a4ad0 con 0x560e26bb0d30 2024-03-16T09:16:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.475+0000 7f8b77fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb5ea0 0x560e26bc1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.476+0000 7f8b77fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb5ea0 0x560e26bc1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34764/0 (socket says 172.21.15.112:34764) 2024-03-16T09:16:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.476+0000 7f8b77fff640 1 -- 172.21.15.112:0/2152993508 learned_addr learned my addr 172.21.15.112:0/2152993508 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.476+0000 7f8b7effd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb0d30 0x560e26ba6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.476+0000 7f8b7f7fe640 1 --2- 172.21.15.112:0/2152993508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bc66e0 0x560e26bba240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.476+0000 7f8b7effd640 1 -- 172.21.15.112:0/2152993508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bc66e0 msgr2=0x560e26bba240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.476+0000 7f8b7effd640 1 --2- 172.21.15.112:0/2152993508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bc66e0 0x560e26bba240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.476+0000 7f8b7effd640 1 -- 172.21.15.112:0/2152993508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb5ea0 msgr2=0x560e26bc1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.476+0000 7f8b7effd640 1 --2- 172.21.15.112:0/2152993508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb5ea0 0x560e26bc1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.476+0000 7f8b7effd640 1 -- 172.21.15.112:0/2152993508 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e26bb1b40 con 0x560e26bb0d30 2024-03-16T09:16:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.476+0000 7f8b77fff640 1 --2- 172.21.15.112:0/2152993508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb5ea0 0x560e26bc1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.476+0000 7f8b7f7fe640 1 --2- 172.21.15.112:0/2152993508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bc66e0 0x560e26bba240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:16:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.476+0000 7f8b7effd640 1 --2- 172.21.15.112:0/2152993508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb0d30 0x560e26ba6a40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b700b9630 tx=0x7f8b700bc740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b66cd7a26e013a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.476+0000 7f8b757fa640 1 -- 172.21.15.112:0/2152993508 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b700b8f50 con 0x560e26bb0d30 2024-03-16T09:16:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.476+0000 7f8b757fa640 1 -- 172.21.15.112:0/2152993508 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b700c5450 con 0x560e26bb0d30 2024-03-16T09:16:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.476+0000 7f8b757fa640 1 -- 172.21.15.112:0/2152993508 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b700c5160 con 0x560e26bb0d30 2024-03-16T09:16:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.477+0000 7f8b8857b740 1 -- 172.21.15.112:0/2152993508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb0d30 msgr2=0x560e26ba6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.477+0000 7f8b8857b740 1 --2- 172.21.15.112:0/2152993508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb0d30 0x560e26ba6a40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8b700b9630 tx=0x7f8b700bc740 comp rx=0 tx=0).stop 2024-03-16T09:16:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.477+0000 7f8b8857b740 1 -- 172.21.15.112:0/2152993508 shutdown_connections 2024-03-16T09:16:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.477+0000 7f8b8857b740 1 --2- 172.21.15.112:0/2152993508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bc66e0 0x560e26bba240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.477+0000 7f8b8857b740 1 --2- 172.21.15.112:0/2152993508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb5ea0 0x560e26bc1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.477+0000 7f8b8857b740 1 --2- 172.21.15.112:0/2152993508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb0d30 0x560e26ba6a40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.477+0000 7f8b8857b740 1 -- 172.21.15.112:0/2152993508 >> 172.21.15.112:0/2152993508 conn(0x560e26bab8b0 msgr2=0x560e26babc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.477+0000 7f8b8857b740 1 -- 172.21.15.112:0/2152993508 shutdown_connections 2024-03-16T09:16:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.477+0000 7f8b8857b740 1 -- 172.21.15.112:0/2152993508 wait complete. 2024-03-16T09:16:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.477+0000 7f8b8857b740 1 -- start start 2024-03-16T09:16:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.478+0000 7f8b8857b740 1 Processor -- start 2024-03-16T09:16:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.478+0000 7f8b8857b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb0d30 0x560e26bbf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.478+0000 7f8b8857b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb5ea0 0x560e26bbfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.478+0000 7f8b8857b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bc66e0 0x560e26bc0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.478+0000 7f8b8857b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8b801a5140 con 0x560e26bb0d30 2024-03-16T09:16:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.478+0000 7f8b8857b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8b801a65b0 con 0x560e26bc66e0 2024-03-16T09:16:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.478+0000 7f8b8857b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8b801a5810 con 0x560e26bb5ea0 2024-03-16T09:16:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.478+0000 7f8b8857b740 10 client.0 _ll_register_callbacks cb 0x7f8b700097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.478+0000 7f8b77fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb5ea0 0x560e26bbfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.478+0000 7f8b7effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb0d30 0x560e26bbf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.478+0000 7f8b77fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb5ea0 0x560e26bbfd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:45088/0 (socket says 172.21.15.112:45088) 2024-03-16T09:16:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.478+0000 7f8b7effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb0d30 0x560e26bbf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34768/0 (socket says 172.21.15.112:34768) 2024-03-16T09:16:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.478+0000 7f8b77fff640 1 -- 172.21.15.112:0/2162339066 learned_addr learned my addr 172.21.15.112:0/2162339066 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.478+0000 7f8b7f7fe640 1 --2- 172.21.15.112:0/2162339066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bc66e0 0x560e26bc0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.478+0000 7f8b77fff640 1 -- 172.21.15.112:0/2162339066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bc66e0 msgr2=0x560e26bc0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.478+0000 7f8b77fff640 1 --2- 172.21.15.112:0/2162339066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bc66e0 0x560e26bc0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.478+0000 7f8b77fff640 1 -- 172.21.15.112:0/2162339066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb0d30 msgr2=0x560e26bbf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.478+0000 7f8b77fff640 1 --2- 172.21.15.112:0/2162339066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb0d30 0x560e26bbf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.478+0000 7f8b77fff640 1 -- 172.21.15.112:0/2162339066 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e26bc12c0 con 0x560e26bb5ea0 2024-03-16T09:16:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.479+0000 7f8b77fff640 1 --2- 172.21.15.112:0/2162339066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb5ea0 0x560e26bbfd10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8b6801db90 tx=0x7f8b68004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c52cde0e749cf66f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.479+0000 7f8b6effd640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T09:16:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.479+0000 7f8b6effd640 1 -- 172.21.15.112:0/2162339066 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68003db0 con 0x560e26bb5ea0 2024-03-16T09:16:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.479+0000 7f8b6effd640 1 -- 172.21.15.112:0/2162339066 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8b6802f1d0 con 0x560e26bb5ea0 2024-03-16T09:16:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.479+0000 7f8b8857b740 1 -- 172.21.15.112:0/2162339066 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b700c5450 con 0x560e26bb5ea0 2024-03-16T09:16:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.479+0000 7f8b6effd640 1 -- 172.21.15.112:0/2162339066 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f8b6802f3d0 con 0x560e26bb5ea0 2024-03-16T09:16:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.479+0000 7f8b8857b740 10 client.25438 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.479+0000 7f8b8857b740 1 -- 172.21.15.112:0/2162339066 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560e26c2d0e0 con 0x560e26bb5ea0 2024-03-16T09:16:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.479+0000 7f8b6effd640 1 -- 172.21.15.112:0/2162339066 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8b700c5450 con 0x560e26bb5ea0 2024-03-16T09:16:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.479+0000 7f8b6effd640 1 -- 172.21.15.112:0/2162339066 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f8b6800fcb0 con 0x560e26bb5ea0 2024-03-16T09:16:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.479+0000 7f8b8857b740 10 client.25438 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.479+0000 7f8b8857b740 10 client.25438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.479+0000 7f8b8857b740 20 client.25438 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.479+0000 7f8b8857b740 1 --2- 172.21.15.112:0/2162339066 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560e26c2db40 0x560e26c2ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.479+0000 7f8b8857b740 4 client.25438 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T09:16:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.479+0000 7f8b8857b740 1 -- 172.21.15.112:0/2162339066 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e26b1d2d0 con 0x560e26c2db40 2024-03-16T09:16:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.480+0000 7f8b7effd640 1 --2- 172.21.15.112:0/2162339066 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560e26c2db40 0x560e26c2ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.480+0000 7f8b6effd640 10 client.25438 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.480+0000 7f8b7effd640 1 --2- 172.21.15.112:0/2162339066 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560e26c2db40 0x560e26c2ff60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e301ff85a7b1ee server_cookie=208353ad99c89d4d in_seq=0 out_seq=0 2024-03-16T09:16:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.486+0000 7f8b6effd640 1 -- 172.21.15.112:0/2162339066 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560e26b1d2d0 con 0x560e26c2db40 2024-03-16T09:16:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.486+0000 7f8b6effd640 10 client.25438 handle_command_reply: tid=0 2024-03-16T09:16:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.486+0000 7f8b8857b740 1 client.25438 shutdown 2024-03-16T09:16:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.486+0000 7f8b8857b740 1 -- 172.21.15.112:0/2162339066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb5ea0 msgr2=0x560e26bbfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.486+0000 7f8b8857b740 1 --2- 172.21.15.112:0/2162339066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb5ea0 0x560e26bbfd10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8b6801db90 tx=0x7f8b68004730 comp rx=0 tx=0).stop 2024-03-16T09:16:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.486+0000 7f8b8857b740 1 -- 172.21.15.112:0/2162339066 shutdown_connections 2024-03-16T09:16:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.486+0000 7f8b8857b740 1 --2- 172.21.15.112:0/2162339066 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560e26c2db40 0x560e26c2ff60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.486+0000 7f8b8857b740 1 --2- 172.21.15.112:0/2162339066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e26bc66e0 0x560e26bc0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.486+0000 7f8b8857b740 1 --2- 172.21.15.112:0/2162339066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560e26bb5ea0 0x560e26bbfd10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.486+0000 7f8b6effd640 0 client.25438 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.486+0000 7f8b8857b740 1 --2- 172.21.15.112:0/2162339066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560e26bb0d30 0x560e26bbf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.486+0000 7f8b8857b740 1 -- 172.21.15.112:0/2162339066 >> 172.21.15.112:0/2162339066 conn(0x560e26bab8b0 msgr2=0x560e26bb5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.486+0000 7f8b8857b740 1 -- 172.21.15.112:0/2162339066 shutdown_connections 2024-03-16T09:16:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.486+0000 7f8b8857b740 1 -- 172.21.15.112:0/2162339066 wait complete. 2024-03-16T09:16:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.487+0000 7f8b8857b740 20 client.25438 trim_cache size 0 max 16384 2024-03-16T09:16:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-03-16T09:16:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-03-16T09:16:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-03-16T09:16:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-03-16T09:16:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.487+0000 7f8b85989640 1 -- 172.21.15.112:0/1730836014 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8b600a2940 msgr2=0x7f8b600a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.487+0000 7f8b85989640 1 --2- 172.21.15.112:0/1730836014 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8b600a2940 0x7f8b600a4e00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8b801990c0 tx=0x7f8b68005f90 comp rx=0 tx=0).stop 2024-03-16T09:16:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.487+0000 7f8b85989640 1 -- 172.21.15.112:0/1730836014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b80100700 msgr2=0x7f8b80197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.487+0000 7f8b85989640 1 --2- 172.21.15.112:0/1730836014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b80100700 0x7f8b80197bb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8b70034a80 tx=0x7f8b70035f10 comp rx=0 tx=0).stop 2024-03-16T09:16:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.487+0000 7f8b85989640 1 -- 172.21.15.112:0/1730836014 shutdown_connections 2024-03-16T09:16:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.487+0000 7f8b85989640 1 --2- 172.21.15.112:0/1730836014 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f8b600a2940 0x7f8b600a4e00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.487+0000 7f8b85989640 1 --2- 172.21.15.112:0/1730836014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b801022b0 0x7f8b8019f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.487+0000 7f8b85989640 1 --2- 172.21.15.112:0/1730836014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b801018f0 0x7f8b801980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.487+0000 7f8b85989640 1 --2- 172.21.15.112:0/1730836014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b80100700 0x7f8b80197bb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.487+0000 7f8b85989640 1 -- 172.21.15.112:0/1730836014 >> 172.21.15.112:0/1730836014 conn(0x7f8b800fbc70 msgr2=0x7f8b800fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.487+0000 7f8b85989640 1 -- 172.21.15.112:0/1730836014 shutdown_connections 2024-03-16T09:16:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:21.487+0000 7f8b85989640 1 -- 172.21.15.112:0/1730836014 wait complete. 2024-03-16T09:16:22.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:22 smithi112 ceph-mon[32307]: pgmap v372: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T09:16:22.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:22 smithi194 ceph-mon[36729]: pgmap v372: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T09:16:22.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:22 smithi159 ceph-mon[36739]: pgmap v372: 97 pgs: 97 active+clean; 657 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T09:16:24.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:24 smithi112 ceph-mon[32307]: pgmap v373: 97 pgs: 97 active+clean; 666 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-03-16T09:16:24.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:24 smithi194 ceph-mon[36729]: pgmap v373: 97 pgs: 97 active+clean; 666 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-03-16T09:16:24.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:24 smithi159 ceph-mon[36739]: pgmap v373: 97 pgs: 97 active+clean; 666 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-03-16T09:16:26.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:26 smithi112 ceph-mon[32307]: pgmap v374: 97 pgs: 97 active+clean; 666 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-16T09:16:26.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:26 smithi194 ceph-mon[36729]: pgmap v374: 97 pgs: 97 active+clean; 666 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-16T09:16:26.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:26 smithi159 ceph-mon[36739]: pgmap v374: 97 pgs: 97 active+clean; 666 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-16T09:16:28.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:28 smithi112 ceph-mon[32307]: pgmap v375: 97 pgs: 97 active+clean; 667 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-16T09:16:28.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:28 smithi194 ceph-mon[36729]: pgmap v375: 97 pgs: 97 active+clean; 667 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-16T09:16:28.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:28 smithi159 ceph-mon[36739]: pgmap v375: 97 pgs: 97 active+clean; 667 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-16T09:16:30.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:30 smithi112 ceph-mon[32307]: pgmap v376: 97 pgs: 97 active+clean; 667 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-16T09:16:30.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:30 smithi194 ceph-mon[36729]: pgmap v376: 97 pgs: 97 active+clean; 667 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-16T09:16:30.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:30 smithi159 ceph-mon[36739]: pgmap v376: 97 pgs: 97 active+clean; 667 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-16T09:16:32.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:32 smithi112 ceph-mon[32307]: pgmap v377: 97 pgs: 97 active+clean; 667 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-16T09:16:32.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:32 smithi194 ceph-mon[36729]: pgmap v377: 97 pgs: 97 active+clean; 667 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-16T09:16:32.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:32 smithi159 ceph-mon[36739]: pgmap v377: 97 pgs: 97 active+clean; 667 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-16T09:16:34.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:34 smithi112 ceph-mon[32307]: pgmap v378: 97 pgs: 97 active+clean; 667 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-16T09:16:34.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:34 smithi194 ceph-mon[36729]: pgmap v378: 97 pgs: 97 active+clean; 667 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-16T09:16:34.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:34 smithi159 ceph-mon[36739]: pgmap v378: 97 pgs: 97 active+clean; 667 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-16T09:16:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.574+0000 7f87ae10c640 1 -- 172.21.15.112:0/2556799521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a8101830 msgr2=0x7f87a8101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.574+0000 7f87ae10c640 1 --2- 172.21.15.112:0/2556799521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a8101830 0x7f87a8101c50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8798009900 tx=0x7f879802bc30 comp rx=0 tx=0).stop 2024-03-16T09:16:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.574+0000 7f87ae10c640 1 -- 172.21.15.112:0/2556799521 shutdown_connections 2024-03-16T09:16:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.574+0000 7f87ae10c640 1 --2- 172.21.15.112:0/2556799521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87a81033e0 0x7f87a8109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.574+0000 7f87ae10c640 1 --2- 172.21.15.112:0/2556799521 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87a8102a20 0x7f87a8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.574+0000 7f87ae10c640 1 --2- 172.21.15.112:0/2556799521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a8101830 0x7f87a8101c50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.574+0000 7f87ae10c640 1 -- 172.21.15.112:0/2556799521 >> 172.21.15.112:0/2556799521 conn(0x7f87a80fcda0 msgr2=0x7f87a80ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.574+0000 7f87ae10c640 1 -- 172.21.15.112:0/2556799521 shutdown_connections 2024-03-16T09:16:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.574+0000 7f87ae10c640 1 -- 172.21.15.112:0/2556799521 wait complete. 2024-03-16T09:16:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.574+0000 7f87ae10c640 1 Processor -- start 2024-03-16T09:16:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.575+0000 7f87ae10c640 1 -- start start 2024-03-16T09:16:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.575+0000 7f87ae10c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87a8101830 0x7f87a8197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.575+0000 7f87ae10c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87a8102a20 0x7f87a81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.575+0000 7f87ae10c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a81033e0 0x7f87a819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.575+0000 7f87ae10c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f87a810d590 con 0x7f87a81033e0 2024-03-16T09:16:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.575+0000 7f87ae10c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f87a810d410 con 0x7f87a8102a20 2024-03-16T09:16:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.575+0000 7f87ae10c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f87a810d710 con 0x7f87a8101830 2024-03-16T09:16:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.575+0000 7f87a77fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87a8101830 0x7f87a8197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.575+0000 7f87a6ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87a8102a20 0x7f87a81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.575+0000 7f87a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a81033e0 0x7f87a819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.575+0000 7f87a77fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87a8101830 0x7f87a8197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:47688/0 (socket says 172.21.15.112:47688) 2024-03-16T09:16:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.575+0000 7f87a77fe640 1 -- 172.21.15.112:0/2279968597 learned_addr learned my addr 172.21.15.112:0/2279968597 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.575+0000 7f87a6ffd640 1 -- 172.21.15.112:0/2279968597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87a8101830 msgr2=0x7f87a8197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.575+0000 7f87a6ffd640 1 --2- 172.21.15.112:0/2279968597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87a8101830 0x7f87a8197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.575+0000 7f87a6ffd640 1 -- 172.21.15.112:0/2279968597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a81033e0 msgr2=0x7f87a819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.575+0000 7f87a6ffd640 1 --2- 172.21.15.112:0/2279968597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a81033e0 0x7f87a819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.575+0000 7f87a6ffd640 1 -- 172.21.15.112:0/2279968597 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a819f7c0 con 0x7f87a8102a20 2024-03-16T09:16:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.575+0000 7f87a7fff640 1 --2- 172.21.15.112:0/2279968597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a81033e0 0x7f87a819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.576+0000 7f87a6ffd640 1 --2- 172.21.15.112:0/2279968597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87a8102a20 0x7f87a81980e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f879c009b50 tx=0x7f879c00b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.576+0000 7f87a4ff9640 1 -- 172.21.15.112:0/2279968597 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c009e50 con 0x7f87a8102a20 2024-03-16T09:16:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.576+0000 7f87ae10c640 1 -- 172.21.15.112:0/2279968597 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a8073a70 con 0x7f87a8102a20 2024-03-16T09:16:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.576+0000 7f87a4ff9640 1 -- 172.21.15.112:0/2279968597 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f879c005520 con 0x7f87a8102a20 2024-03-16T09:16:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.576+0000 7f87a4ff9640 1 -- 172.21.15.112:0/2279968597 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c019670 con 0x7f87a8102a20 2024-03-16T09:16:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.576+0000 7f87ae10c640 1 -- 172.21.15.112:0/2279968597 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a8073fc0 con 0x7f87a8102a20 2024-03-16T09:16:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.578+0000 7f87b0dcc740 1 Processor -- start 2024-03-16T09:16:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.578+0000 7f87b0dcc740 1 -- start start 2024-03-16T09:16:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.579+0000 7f87b0dcc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f58340 0x5654c1f58760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.579+0000 7f87b0dcc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58d30 0x5654c1f5d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.579+0000 7f87b0dcc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f5dea0 0x5654c1f60330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.579+0000 7f87b0dcc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5654c1ebda40 con 0x5654c1f58d30 2024-03-16T09:16:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.579+0000 7f87b0dcc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5654c1e66920 con 0x5654c1f5dea0 2024-03-16T09:16:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.579+0000 7f87b0dcc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5654c1eb9ae0 con 0x5654c1f58340 2024-03-16T09:16:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.579+0000 7f87a7fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f58340 0x5654c1f58760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.579+0000 7f87a77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58d30 0x5654c1f5d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.579+0000 7f87a7fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f58340 0x5654c1f58760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:47694/0 (socket says 172.21.15.112:47694) 2024-03-16T09:16:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.579+0000 7f87a7fff640 1 -- 172.21.15.112:0/3356291291 learned_addr learned my addr 172.21.15.112:0/3356291291 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.579+0000 7f87a77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58d30 0x5654c1f5d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60946/0 (socket says 172.21.15.112:60946) 2024-03-16T09:16:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.579+0000 7f87a4ff9640 1 -- 172.21.15.112:0/2279968597 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7f879c020020 con 0x7f87a8102a20 2024-03-16T09:16:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.579+0000 7f87a77fe640 1 -- 172.21.15.112:0/3356291291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f58340 msgr2=0x5654c1f58760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.579+0000 7f87a77fe640 1 --2- 172.21.15.112:0/3356291291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f58340 0x5654c1f58760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.579+0000 7f87a77fe640 1 -- 172.21.15.112:0/3356291291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f5dea0 msgr2=0x5654c1f60330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-16T09:16:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.579+0000 7f87a77fe640 1 --2- 172.21.15.112:0/3356291291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f5dea0 0x5654c1f60330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.579+0000 7f87a77fe640 1 -- 172.21.15.112:0/3356291291 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654c1e7e460 con 0x5654c1f58d30 2024-03-16T09:16:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.580+0000 7f87a4ff9640 1 --2- 172.21.15.112:0/2279968597 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f87900a2b60 0x7f87900a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.580+0000 7f87a7fff640 1 --2- 172.21.15.112:0/2279968597 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f87900a2b60 0x7f87900a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.580+0000 7f87a77fe640 1 --2- 172.21.15.112:0/3356291291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58d30 0x5654c1f5d830 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f879802c900 tx=0x7f87980095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b463ba63ad37917c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.580+0000 7f87a4ff9640 1 -- 172.21.15.112:0/2279968597 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f879c0d2520 con 0x7f87a8102a20 2024-03-16T09:16:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.580+0000 7f878dffb640 1 -- 172.21.15.112:0/3356291291 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879802c650 con 0x5654c1f58d30 2024-03-16T09:16:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.580+0000 7f878dffb640 1 -- 172.21.15.112:0/3356291291 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8798004040 con 0x5654c1f58d30 2024-03-16T09:16:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.580+0000 7f878dffb640 1 -- 172.21.15.112:0/3356291291 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879802c650 con 0x5654c1f58d30 2024-03-16T09:16:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.581+0000 7f87b0dcc740 1 -- 172.21.15.112:0/3356291291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58d30 msgr2=0x5654c1f5d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.581+0000 7f87a7fff640 1 --2- 172.21.15.112:0/2279968597 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f87900a2b60 0x7f87900a5020 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5654c1f59590 tx=0x7f87a0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.581+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/3356291291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58d30 0x5654c1f5d830 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f879802c900 tx=0x7f87980095d0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.581+0000 7f87b0dcc740 1 -- 172.21.15.112:0/3356291291 shutdown_connections 2024-03-16T09:16:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.581+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/3356291291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f5dea0 0x5654c1f60330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.581+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/3356291291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58d30 0x5654c1f5d830 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.581+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/3356291291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f58340 0x5654c1f58760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.581+0000 7f87b0dcc740 1 -- 172.21.15.112:0/3356291291 >> 172.21.15.112:0/3356291291 conn(0x5654c1f538b0 msgr2=0x5654c1f53cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.581+0000 7f87b0dcc740 1 -- 172.21.15.112:0/3356291291 shutdown_connections 2024-03-16T09:16:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.581+0000 7f87b0dcc740 1 -- 172.21.15.112:0/3356291291 wait complete. 2024-03-16T09:16:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.582+0000 7f87b0dcc740 1 -- start start 2024-03-16T09:16:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.582+0000 7f87b0dcc740 1 Processor -- start 2024-03-16T09:16:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.582+0000 7f87b0dcc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58340 0x5654c1f62670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.582+0000 7f87b0dcc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f58d30 0x5654c1f62bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.582+0000 7f87b0dcc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f5dea0 0x5654c1f69bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.582+0000 7f87b0dcc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5654c1f60df0 con 0x5654c1f58340 2024-03-16T09:16:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.582+0000 7f87b0dcc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5654c1e79d70 con 0x5654c1f58d30 2024-03-16T09:16:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.582+0000 7f87b0dcc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5654c1f60f70 con 0x5654c1f5dea0 2024-03-16T09:16:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.582+0000 7f87b0dcc740 10 client.0 _ll_register_callbacks cb 0x5654c1bd1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.582+0000 7f87a77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58340 0x5654c1f62670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.582+0000 7f87a77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58340 0x5654c1f62670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60956/0 (socket says 172.21.15.112:60956) 2024-03-16T09:16:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.582+0000 7f87a77fe640 1 -- 172.21.15.112:0/1143142342 learned_addr learned my addr 172.21.15.112:0/1143142342 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.582+0000 7f87a6ffd640 1 --2- 172.21.15.112:0/1143142342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f58d30 0x5654c1f62bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.582+0000 7f87a7fff640 1 --2- 172.21.15.112:0/1143142342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f5dea0 0x5654c1f69bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.583+0000 7f87a77fe640 1 -- 172.21.15.112:0/1143142342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f5dea0 msgr2=0x5654c1f69bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.583+0000 7f87a77fe640 1 --2- 172.21.15.112:0/1143142342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f5dea0 0x5654c1f69bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.583+0000 7f87a77fe640 1 -- 172.21.15.112:0/1143142342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f58d30 msgr2=0x5654c1f62bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.583+0000 7f87a77fe640 1 --2- 172.21.15.112:0/1143142342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f58d30 0x5654c1f62bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.583+0000 7f87a77fe640 1 -- 172.21.15.112:0/1143142342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654c1bd2210 con 0x5654c1f58340 2024-03-16T09:16:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.583+0000 7f87a77fe640 1 --2- 172.21.15.112:0/1143142342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58340 0x5654c1f62670 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8798005ab0 tx=0x7f87980054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2923d9bf3d06e0c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.583+0000 7f8786ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:16:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.583+0000 7f8786ffd640 1 -- 172.21.15.112:0/1143142342 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87980035f0 con 0x5654c1f58340 2024-03-16T09:16:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.583+0000 7f8786ffd640 1 -- 172.21.15.112:0/1143142342 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f8798003790 con 0x5654c1f58340 2024-03-16T09:16:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.583+0000 7f87b0dcc740 1 -- 172.21.15.112:0/1143142342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8798004040 con 0x5654c1f58340 2024-03-16T09:16:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.583+0000 7f8786ffd640 1 -- 172.21.15.112:0/1143142342 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f879805a050 con 0x5654c1f58340 2024-03-16T09:16:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.583+0000 7f87b0dcc740 10 client.15588 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.583+0000 7f87b0dcc740 1 -- 172.21.15.112:0/1143142342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5654c1c9e620 con 0x5654c1f58340 2024-03-16T09:16:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.584+0000 7f8786ffd640 1 -- 172.21.15.112:0/1143142342 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8798004040 con 0x5654c1f58340 2024-03-16T09:16:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.584+0000 7f8786ffd640 1 -- 172.21.15.112:0/1143142342 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f8798034cb0 con 0x5654c1f58340 2024-03-16T09:16:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.584+0000 7f87b0dcc740 10 client.15588 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.584+0000 7f87b0dcc740 10 client.15588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.584+0000 7f87b0dcc740 20 client.15588 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.584+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/1143142342 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5654c1f6e0e0 0x5654c1f704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.584+0000 7f87b0dcc740 4 client.15588 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:16:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.584+0000 7f87b0dcc740 1 -- 172.21.15.112:0/1143142342 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654c1ecaf80 con 0x5654c1f6e0e0 2024-03-16T09:16:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.584+0000 7f87a6ffd640 1 --2- 172.21.15.112:0/1143142342 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5654c1f6e0e0 0x5654c1f704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.585+0000 7f8786ffd640 10 client.15588 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.585+0000 7f87a6ffd640 1 --2- 172.21.15.112:0/1143142342 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5654c1f6e0e0 0x5654c1f704d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a9ac3c4a2ec2bf4 server_cookie=4229778fd82c4871 in_seq=0 out_seq=0 2024-03-16T09:16:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.585+0000 7f8786ffd640 1 -- 172.21.15.112:0/1143142342 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5654c1ecaf80 con 0x5654c1f6e0e0 2024-03-16T09:16:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.585+0000 7f8786ffd640 10 client.15588 handle_command_reply: tid=0 2024-03-16T09:16:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.585+0000 7f87b0dcc740 1 client.15588 shutdown 2024-03-16T09:16:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.586+0000 7f87b0dcc740 1 -- 172.21.15.112:0/1143142342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58340 msgr2=0x5654c1f62670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.586+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/1143142342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58340 0x5654c1f62670 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8798005ab0 tx=0x7f87980054e0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.586+0000 7f87b0dcc740 1 -- 172.21.15.112:0/1143142342 shutdown_connections 2024-03-16T09:16:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.586+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/1143142342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f5dea0 0x5654c1f69bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.586+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/1143142342 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5654c1f6e0e0 0x5654c1f704d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.586+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/1143142342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f58d30 0x5654c1f62bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.586+0000 7f8786ffd640 0 client.15588 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.586+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/1143142342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58340 0x5654c1f62670 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.586+0000 7f87b0dcc740 1 -- 172.21.15.112:0/1143142342 >> 172.21.15.112:0/1143142342 conn(0x5654c1f538b0 msgr2=0x5654c1f52bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.586+0000 7f87b0dcc740 1 -- 172.21.15.112:0/1143142342 shutdown_connections 2024-03-16T09:16:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.586+0000 7f87b0dcc740 1 -- 172.21.15.112:0/1143142342 wait complete. 2024-03-16T09:16:34.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.586+0000 7f87b0dcc740 20 client.15588 trim_cache size 0 max 16384 2024-03-16T09:16:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.600+0000 7f87b0dcc740 1 Processor -- start 2024-03-16T09:16:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.600+0000 7f87b0dcc740 1 -- start start 2024-03-16T09:16:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.600+0000 7f87b0dcc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58340 0x5654c1f61c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.600+0000 7f87b0dcc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f58d30 0x5654c1f69bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.601+0000 7f87b0dcc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f5dea0 0x5654c1f621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.601+0000 7f87a77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58340 0x5654c1f61c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.601+0000 7f87a77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58340 0x5654c1f61c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60958/0 (socket says 172.21.15.112:60958) 2024-03-16T09:16:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.601+0000 7f87b0dcc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f87a81a92a0 con 0x5654c1f58340 2024-03-16T09:16:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.601+0000 7f87b0dcc740 1 -- 172.21.15.112:0/2427696389 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f87a81a6b70 con 0x5654c1f58d30 2024-03-16T09:16:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.601+0000 7f87a77fe640 1 -- 172.21.15.112:0/2427696389 learned_addr learned my addr 172.21.15.112:0/2427696389 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.601+0000 7f87b0dcc740 1 -- 172.21.15.112:0/2427696389 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f87a81a49c0 con 0x5654c1f5dea0 2024-03-16T09:16:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.601+0000 7f87a6ffd640 1 --2- 172.21.15.112:0/2427696389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f58d30 0x5654c1f69bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.601+0000 7f87a7fff640 1 --2- 172.21.15.112:0/2427696389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f5dea0 0x5654c1f621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.601+0000 7f87a77fe640 1 -- 172.21.15.112:0/2427696389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f5dea0 msgr2=0x5654c1f621d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.601+0000 7f87a77fe640 1 --2- 172.21.15.112:0/2427696389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f5dea0 0x5654c1f621d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.601+0000 7f87a77fe640 1 -- 172.21.15.112:0/2427696389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f58d30 msgr2=0x5654c1f69bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.601+0000 7f87a77fe640 1 --2- 172.21.15.112:0/2427696389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f58d30 0x5654c1f69bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.601+0000 7f87a77fe640 1 -- 172.21.15.112:0/2427696389 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654c1f70ab0 con 0x5654c1f58340 2024-03-16T09:16:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.601+0000 7f87a6ffd640 1 --2- 172.21.15.112:0/2427696389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f58d30 0x5654c1f69bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.601+0000 7f87a77fe640 1 --2- 172.21.15.112:0/2427696389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58340 0x5654c1f61c90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f879802c860 tx=0x7f8798004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5867a1b615b641e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.601+0000 7f878d7fa640 1 -- 172.21.15.112:0/2427696389 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879803f840 con 0x5654c1f58340 2024-03-16T09:16:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.601+0000 7f878d7fa640 1 -- 172.21.15.112:0/2427696389 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f87980344e0 con 0x5654c1f58340 2024-03-16T09:16:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.601+0000 7f878d7fa640 1 -- 172.21.15.112:0/2427696389 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798054690 con 0x5654c1f58340 2024-03-16T09:16:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.602+0000 7f87b0dcc740 1 -- 172.21.15.112:0/2427696389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58340 msgr2=0x5654c1f61c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.602+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/2427696389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58340 0x5654c1f61c90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f879802c860 tx=0x7f8798004500 comp rx=0 tx=0).stop 2024-03-16T09:16:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.602+0000 7f87b0dcc740 1 -- 172.21.15.112:0/2427696389 shutdown_connections 2024-03-16T09:16:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.602+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/2427696389 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f5dea0 0x5654c1f621d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.610 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.602+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/2427696389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f58d30 0x5654c1f69bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.602+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/2427696389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f58340 0x5654c1f61c90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.602+0000 7f87b0dcc740 1 -- 172.21.15.112:0/2427696389 >> 172.21.15.112:0/2427696389 conn(0x5654c1f538b0 msgr2=0x5654c1f573a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.602+0000 7f87b0dcc740 1 -- 172.21.15.112:0/2427696389 shutdown_connections 2024-03-16T09:16:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.602+0000 7f87b0dcc740 1 -- 172.21.15.112:0/2427696389 wait complete. 2024-03-16T09:16:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.603+0000 7f87b0dcc740 1 -- start start 2024-03-16T09:16:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.603+0000 7f87b0dcc740 1 Processor -- start 2024-03-16T09:16:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.603+0000 7f87b0dcc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f58340 0x5654c1f673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.603+0000 7f87b0dcc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f58d30 0x5654c1f678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.603+0000 7f87b0dcc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f5dea0 0x5654c1f688f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.603+0000 7f87b0dcc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f87a81a5030 con 0x5654c1f5dea0 2024-03-16T09:16:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.603+0000 7f87b0dcc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f87a81a64a0 con 0x5654c1f58d30 2024-03-16T09:16:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.603+0000 7f87b0dcc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f87a81a5700 con 0x5654c1f58340 2024-03-16T09:16:34.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.603+0000 7f87a77fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f58340 0x5654c1f673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.603+0000 7f87a77fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f58340 0x5654c1f673a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:47732/0 (socket says 172.21.15.112:47732) 2024-03-16T09:16:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.603+0000 7f87a77fe640 1 -- 172.21.15.112:0/2113741531 learned_addr learned my addr 172.21.15.112:0/2113741531 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.603+0000 7f87a6ffd640 1 --2- 172.21.15.112:0/2113741531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f58d30 0x5654c1f678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.604+0000 7f87a7fff640 1 --2- 172.21.15.112:0/2113741531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f5dea0 0x5654c1f688f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.604+0000 7f87b0dcc740 10 client.0 _ll_register_callbacks cb 0x7f87a0009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.604+0000 7f87a77fe640 1 -- 172.21.15.112:0/2113741531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f58d30 msgr2=0x5654c1f678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.604+0000 7f87a77fe640 1 --2- 172.21.15.112:0/2113741531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f58d30 0x5654c1f678e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.604+0000 7f87a77fe640 1 -- 172.21.15.112:0/2113741531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f5dea0 msgr2=0x5654c1f688f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.604+0000 7f87a77fe640 1 --2- 172.21.15.112:0/2113741531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f5dea0 0x5654c1f688f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.604+0000 7f87a77fe640 1 -- 172.21.15.112:0/2113741531 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654c1f68ee0 con 0x5654c1f58340 2024-03-16T09:16:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.604+0000 7f87a77fe640 1 --2- 172.21.15.112:0/2113741531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f58340 0x5654c1f673a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f879802c900 tx=0x7f8798034c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=511dbd83aa946c50 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.604+0000 7f8786ffd640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T09:16:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.604+0000 7f8786ffd640 1 -- 172.21.15.112:0/2113741531 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879803f9d0 con 0x5654c1f58340 2024-03-16T09:16:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.604+0000 7f87b0dcc740 1 -- 172.21.15.112:0/2113741531 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87980344e0 con 0x5654c1f58340 2024-03-16T09:16:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.604+0000 7f8786ffd640 1 -- 172.21.15.112:0/2113741531 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f879803fb70 con 0x5654c1f58340 2024-03-16T09:16:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.604+0000 7f8786ffd640 1 -- 172.21.15.112:0/2113741531 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f879803fd70 con 0x5654c1f58340 2024-03-16T09:16:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.604+0000 7f87b0dcc740 10 client.25456 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.604+0000 7f87b0dcc740 1 -- 172.21.15.112:0/2113741531 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5654c1fd4360 con 0x5654c1f58340 2024-03-16T09:16:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.605+0000 7f8786ffd640 1 -- 172.21.15.112:0/2113741531 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f87980344e0 con 0x5654c1f58340 2024-03-16T09:16:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.605+0000 7f8786ffd640 1 -- 172.21.15.112:0/2113741531 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f879802cce0 con 0x5654c1f58340 2024-03-16T09:16:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.605+0000 7f87b0dcc740 10 client.25456 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.605+0000 7f87b0dcc740 10 client.25456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.605+0000 7f87b0dcc740 20 client.25456 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:34.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.605+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/2113741531 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5654c1f6e0e0 0x5654c1fd6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.605+0000 7f87b0dcc740 4 client.25456 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T09:16:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.605+0000 7f87b0dcc740 1 -- 172.21.15.112:0/2113741531 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5654c1ec52d0 con 0x5654c1f6e0e0 2024-03-16T09:16:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.605+0000 7f87a6ffd640 1 --2- 172.21.15.112:0/2113741531 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5654c1f6e0e0 0x5654c1fd6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.605+0000 7f87a6ffd640 1 --2- 172.21.15.112:0/2113741531 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5654c1f6e0e0 0x5654c1fd6c30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f1a4852909cd98e server_cookie=2c7ca6a2a54e0367 in_seq=0 out_seq=0 2024-03-16T09:16:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.605+0000 7f8786ffd640 10 client.25456 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.608+0000 7f8786ffd640 1 -- 172.21.15.112:0/2113741531 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5654c1ec52d0 con 0x5654c1f6e0e0 2024-03-16T09:16:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.608+0000 7f8786ffd640 10 client.25456 handle_command_reply: tid=0 2024-03-16T09:16:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.608+0000 7f87b0dcc740 1 client.25456 shutdown 2024-03-16T09:16:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.608+0000 7f87b0dcc740 1 -- 172.21.15.112:0/2113741531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f58340 msgr2=0x5654c1f673a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.609+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/2113741531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f58340 0x5654c1f673a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f879802c900 tx=0x7f8798034c00 comp rx=0 tx=0).stop 2024-03-16T09:16:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.609+0000 7f87b0dcc740 1 -- 172.21.15.112:0/2113741531 shutdown_connections 2024-03-16T09:16:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.609+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/2113741531 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5654c1f5dea0 0x5654c1f688f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.609+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/2113741531 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5654c1f6e0e0 0x5654c1fd6c30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.609+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/2113741531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654c1f58d30 0x5654c1f678e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.609+0000 7f87b0dcc740 1 --2- 172.21.15.112:0/2113741531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5654c1f58340 0x5654c1f673a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.609+0000 7f8786ffd640 0 client.25456 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.609+0000 7f87b0dcc740 1 -- 172.21.15.112:0/2113741531 >> 172.21.15.112:0/2113741531 conn(0x5654c1f538b0 msgr2=0x5654c1f52320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.609+0000 7f87b0dcc740 1 -- 172.21.15.112:0/2113741531 shutdown_connections 2024-03-16T09:16:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.609+0000 7f87b0dcc740 1 -- 172.21.15.112:0/2113741531 wait complete. 2024-03-16T09:16:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.609+0000 7f87b0dcc740 20 client.25456 trim_cache size 0 max 16384 2024-03-16T09:16:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-03-16T09:16:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-03-16T09:16:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-03-16T09:16:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-03-16T09:16:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.609+0000 7f87ae10c640 1 -- 172.21.15.112:0/2279968597 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f87900a2b60 msgr2=0x7f87900a5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.609+0000 7f87ae10c640 1 --2- 172.21.15.112:0/2279968597 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f87900a2b60 0x7f87900a5020 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5654c1f59590 tx=0x7f87a0009500 comp rx=0 tx=0).stop 2024-03-16T09:16:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.609+0000 7f87ae10c640 1 -- 172.21.15.112:0/2279968597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87a8102a20 msgr2=0x7f87a81980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.609+0000 7f87ae10c640 1 --2- 172.21.15.112:0/2279968597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87a8102a20 0x7f87a81980e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f879c009b50 tx=0x7f879c00b690 comp rx=0 tx=0).stop 2024-03-16T09:16:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.610+0000 7f87ae10c640 1 -- 172.21.15.112:0/2279968597 shutdown_connections 2024-03-16T09:16:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.610+0000 7f87ae10c640 1 --2- 172.21.15.112:0/2279968597 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f87900a2b60 0x7f87900a5020 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.610+0000 7f87ae10c640 1 --2- 172.21.15.112:0/2279968597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87a81033e0 0x7f87a819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.610+0000 7f87ae10c640 1 --2- 172.21.15.112:0/2279968597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f87a8102a20 0x7f87a81980e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.610+0000 7f87ae10c640 1 --2- 172.21.15.112:0/2279968597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87a8101830 0x7f87a8197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.610+0000 7f87ae10c640 1 -- 172.21.15.112:0/2279968597 >> 172.21.15.112:0/2279968597 conn(0x7f87a80fcda0 msgr2=0x7f87a8105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.610+0000 7f87ae10c640 1 -- 172.21.15.112:0/2279968597 shutdown_connections 2024-03-16T09:16:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:34.610+0000 7f87ae10c640 1 -- 172.21.15.112:0/2279968597 wait complete. 2024-03-16T09:16:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:36 smithi112 ceph-mon[32307]: pgmap v379: 97 pgs: 97 active+clean; 667 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-16T09:16:36.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:36 smithi194 ceph-mon[36729]: pgmap v379: 97 pgs: 97 active+clean; 667 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-16T09:16:36.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:36 smithi159 ceph-mon[36739]: pgmap v379: 97 pgs: 97 active+clean; 667 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-16T09:16:38.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:38 smithi112 ceph-mon[32307]: pgmap v380: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2024-03-16T09:16:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:38 smithi194 ceph-mon[36729]: pgmap v380: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2024-03-16T09:16:38.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:38 smithi159 ceph-mon[36739]: pgmap v380: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 7 op/s 2024-03-16T09:16:40.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:40 smithi112 ceph-mon[32307]: pgmap v381: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:16:40.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:40 smithi194 ceph-mon[36729]: pgmap v381: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:16:40.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:40 smithi159 ceph-mon[36739]: pgmap v381: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:16:42.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:42 smithi112 ceph-mon[32307]: pgmap v382: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T09:16:42.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:42 smithi194 ceph-mon[36729]: pgmap v382: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T09:16:42.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:42 smithi159 ceph-mon[36739]: pgmap v382: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T09:16:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:44 smithi112 ceph-mon[32307]: pgmap v383: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T09:16:44.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:44 smithi194 ceph-mon[36729]: pgmap v383: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T09:16:44.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:44 smithi159 ceph-mon[36739]: pgmap v383: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T09:16:44.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T09:16:44.674 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T09:16:44.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.781+0000 7f6ac275b640 1 -- 172.21.15.112:0/732673224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6abc107520 msgr2=0x7f6abc109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:44.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.781+0000 7f6ac275b640 1 --2- 172.21.15.112:0/732673224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6abc107520 0x7f6abc109910 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009ca0 tx=0x7f6ab402c0f0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.781+0000 7f6ac275b640 1 -- 172.21.15.112:0/732673224 shutdown_connections 2024-03-16T09:16:44.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.781+0000 7f6ac275b640 1 --2- 172.21.15.112:0/732673224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6abc107520 0x7f6abc109910 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.781+0000 7f6ac275b640 1 --2- 172.21.15.112:0/732673224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6abc104bf0 0x7f6abc106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.781+0000 7f6ac275b640 1 --2- 172.21.15.112:0/732673224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6abc1022c0 0x7f6abc1046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.781+0000 7f6ac275b640 1 -- 172.21.15.112:0/732673224 >> 172.21.15.112:0/732673224 conn(0x7f6abc0fbc70 msgr2=0x7f6abc0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:44.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.782+0000 7f6ac275b640 1 -- 172.21.15.112:0/732673224 shutdown_connections 2024-03-16T09:16:44.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.782+0000 7f6ac275b640 1 -- 172.21.15.112:0/732673224 wait complete. 2024-03-16T09:16:44.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.782+0000 7f6ac275b640 1 Processor -- start 2024-03-16T09:16:44.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.782+0000 7f6ac275b640 1 -- start start 2024-03-16T09:16:44.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.783+0000 7f6ac275b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6abc1022c0 0x7f6abc112440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.783+0000 7f6ac275b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6abc104bf0 0x7f6abc112980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.783+0000 7f6ac275b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6abc107520 0x7f6abc110a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.783+0000 7f6ac275b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc10b390 con 0x7f6abc104bf0 2024-03-16T09:16:44.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.783+0000 7f6ac275b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc10b210 con 0x7f6abc107520 2024-03-16T09:16:44.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.783+0000 7f6ac275b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc10b510 con 0x7f6abc1022c0 2024-03-16T09:16:44.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.783+0000 7f6abb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6abc104bf0 0x7f6abc112980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.783+0000 7f6abbfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6abc1022c0 0x7f6abc112440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.783+0000 7f6abb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6abc104bf0 0x7f6abc112980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40662/0 (socket says 172.21.15.112:40662) 2024-03-16T09:16:44.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.783+0000 7f6abb7fe640 1 -- 172.21.15.112:0/466815244 learned_addr learned my addr 172.21.15.112:0/466815244 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:44.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.783+0000 7f6ac0cf8640 1 --2- 172.21.15.112:0/466815244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6abc107520 0x7f6abc110a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.783+0000 7f6abbfff640 1 -- 172.21.15.112:0/466815244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6abc107520 msgr2=0x7f6abc110a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:44.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.783+0000 7f6abbfff640 1 --2- 172.21.15.112:0/466815244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6abc107520 0x7f6abc110a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.783+0000 7f6abbfff640 1 -- 172.21.15.112:0/466815244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6abc104bf0 msgr2=0x7f6abc112980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:44.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.783+0000 7f6abbfff640 1 --2- 172.21.15.112:0/466815244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6abc104bf0 0x7f6abc112980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.783+0000 7f6abbfff640 1 -- 172.21.15.112:0/466815244 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6abc111280 con 0x7f6abc1022c0 2024-03-16T09:16:44.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.783+0000 7f6abb7fe640 1 --2- 172.21.15.112:0/466815244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6abc104bf0 0x7f6abc112980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:44.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.784+0000 7f6abbfff640 1 --2- 172.21.15.112:0/466815244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6abc1022c0 0x7f6abc112440 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00e9f0 tx=0x7f6aac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:44.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.784+0000 7f6ab97fa640 1 -- 172.21.15.112:0/466815244 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac00ccc0 con 0x7f6abc1022c0 2024-03-16T09:16:44.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.784+0000 7f6ab97fa640 1 -- 172.21.15.112:0/466815244 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6aac010930 con 0x7f6abc1022c0 2024-03-16T09:16:44.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.784+0000 7f6ab97fa640 1 -- 172.21.15.112:0/466815244 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac0189f0 con 0x7f6abc1022c0 2024-03-16T09:16:44.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.784+0000 7f6ac275b640 1 -- 172.21.15.112:0/466815244 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6abc111570 con 0x7f6abc1022c0 2024-03-16T09:16:44.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.784+0000 7f6ac275b640 1 -- 172.21.15.112:0/466815244 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6abc1aefd0 con 0x7f6abc1022c0 2024-03-16T09:16:44.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6ac5422740 1 Processor -- start 2024-03-16T09:16:44.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6ac5422740 1 -- start start 2024-03-16T09:16:44.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6ab97fa640 1 -- 172.21.15.112:0/466815244 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7f6aac010ae0 con 0x7f6abc1022c0 2024-03-16T09:16:44.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6ac5422740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c25170 0x55fd02c25590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6ac5422740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25b60 0x55fd02c2a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6ac5422740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c2acd0 0x55fd02c2d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6ac5422740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fd02b8a500 con 0x55fd02c25b60 2024-03-16T09:16:44.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6ac5422740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fd02b32be0 con 0x55fd02c25170 2024-03-16T09:16:44.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6ac5422740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fd02b865a0 con 0x55fd02c2acd0 2024-03-16T09:16:44.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6ac0cf8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c25170 0x55fd02c25590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6ac0cf8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c25170 0x55fd02c25590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:35506/0 (socket says 172.21.15.112:35506) 2024-03-16T09:16:44.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6ac0cf8640 1 -- 172.21.15.112:0/4169425240 learned_addr learned my addr 172.21.15.112:0/4169425240 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:44.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6abb7fe640 1 --2- 172.21.15.112:0/4169425240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c2acd0 0x55fd02c2d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6abbfff640 1 --2- 172.21.15.112:0/4169425240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25b60 0x55fd02c2a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6ac0cf8640 1 -- 172.21.15.112:0/4169425240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c2acd0 msgr2=0x55fd02c2d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:44.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6ac0cf8640 1 --2- 172.21.15.112:0/4169425240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c2acd0 0x55fd02c2d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6ac0cf8640 1 -- 172.21.15.112:0/4169425240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25b60 msgr2=0x55fd02c2a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:44.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6ac0cf8640 1 --2- 172.21.15.112:0/4169425240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25b60 0x55fd02c2a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6ac0cf8640 1 -- 172.21.15.112:0/4169425240 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd02b464b0 con 0x55fd02c25170 2024-03-16T09:16:44.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6abbfff640 1 --2- 172.21.15.112:0/4169425240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25b60 0x55fd02c2a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:16:44.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.787+0000 7f6ab97fa640 1 --2- 172.21.15.112:0/466815244 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6aa40a2900 0x7f6aa40a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.788+0000 7f6ac0cf8640 1 --2- 172.21.15.112:0/4169425240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c25170 0x55fd02c25590 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6ab402c700 tx=0x7f6ab4005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1530b34c0bd40f5b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:44.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.788+0000 7f6abbfff640 1 --2- 172.21.15.112:0/466815244 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6aa40a2900 0x7f6aa40a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.788+0000 7f6aa27fc640 1 -- 172.21.15.112:0/4169425240 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4043070 con 0x55fd02c25170 2024-03-16T09:16:44.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.788+0000 7f6ab97fa640 1 -- 172.21.15.112:0/466815244 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f6aac0cee90 con 0x7f6abc1022c0 2024-03-16T09:16:44.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.788+0000 7f6aa27fc640 1 -- 172.21.15.112:0/4169425240 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6ab4032d30 con 0x55fd02c25170 2024-03-16T09:16:44.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.788+0000 7f6aa27fc640 1 -- 172.21.15.112:0/4169425240 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4039040 con 0x55fd02c25170 2024-03-16T09:16:44.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.789+0000 7f6ac5422740 1 -- 172.21.15.112:0/4169425240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c25170 msgr2=0x55fd02c25590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:44.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.789+0000 7f6abbfff640 1 --2- 172.21.15.112:0/466815244 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6aa40a2900 0x7f6aa40a4dc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55fd02c25f60 tx=0x7f6aac00cf70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:44.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.789+0000 7f6ac5422740 1 --2- 172.21.15.112:0/4169425240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c25170 0x55fd02c25590 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6ab402c700 tx=0x7f6ab4005f80 comp rx=0 tx=0).stop 2024-03-16T09:16:44.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.789+0000 7f6ac5422740 1 -- 172.21.15.112:0/4169425240 shutdown_connections 2024-03-16T09:16:44.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.789+0000 7f6ac5422740 1 --2- 172.21.15.112:0/4169425240 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c2acd0 0x55fd02c2d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.789+0000 7f6ac5422740 1 --2- 172.21.15.112:0/4169425240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25b60 0x55fd02c2a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.789+0000 7f6ac5422740 1 --2- 172.21.15.112:0/4169425240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c25170 0x55fd02c25590 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.789+0000 7f6ac5422740 1 -- 172.21.15.112:0/4169425240 >> 172.21.15.112:0/4169425240 conn(0x55fd02c206e0 msgr2=0x55fd02c20ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:44.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.789+0000 7f6ac5422740 1 -- 172.21.15.112:0/4169425240 shutdown_connections 2024-03-16T09:16:44.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.789+0000 7f6ac5422740 1 -- 172.21.15.112:0/4169425240 wait complete. 2024-03-16T09:16:44.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.789+0000 7f6ac5422740 1 -- start start 2024-03-16T09:16:44.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.789+0000 7f6ac5422740 1 Processor -- start 2024-03-16T09:16:44.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6ac5422740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25170 0x55fd02c2f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6ac5422740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c25b60 0x55fd02c2fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6ac5422740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c2acd0 0x55fd02c36bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6ac5422740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fd02c2dcc0 con 0x55fd02c25170 2024-03-16T09:16:44.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6ac5422740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fd02b46160 con 0x55fd02c2acd0 2024-03-16T09:16:44.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6ac5422740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fd02c2de40 con 0x55fd02c25b60 2024-03-16T09:16:44.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6ac0cf8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25170 0x55fd02c2f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6ac5422740 10 client.0 _ll_register_callbacks cb 0x55fd02b2a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:44.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6ac0cf8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25170 0x55fd02c2f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40674/0 (socket says 172.21.15.112:40674) 2024-03-16T09:16:44.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6ac0cf8640 1 -- 172.21.15.112:0/3621207859 learned_addr learned my addr 172.21.15.112:0/3621207859 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:44.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6abb7fe640 1 --2- 172.21.15.112:0/3621207859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c25b60 0x55fd02c2fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6abb7fe640 1 -- 172.21.15.112:0/3621207859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c2acd0 msgr2=0x55fd02c36bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:44.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6abb7fe640 1 --2- 172.21.15.112:0/3621207859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c2acd0 0x55fd02c36bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6abb7fe640 1 -- 172.21.15.112:0/3621207859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25170 msgr2=0x55fd02c2f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:44.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6abb7fe640 1 --2- 172.21.15.112:0/3621207859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25170 0x55fd02c2f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6abb7fe640 1 -- 172.21.15.112:0/3621207859 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd029687d0 con 0x55fd02c25b60 2024-03-16T09:16:44.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6ac0cf8640 1 --2- 172.21.15.112:0/3621207859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25170 0x55fd02c2f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:44.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6abb7fe640 1 --2- 172.21.15.112:0/3621207859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c25b60 0x55fd02c2fb70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00095e0 tx=0x7f6ab000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3644de6c6dba8bd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:44.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6a9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T09:16:44.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6a9b7fe640 1 -- 172.21.15.112:0/3621207859 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab00120d0 con 0x55fd02c25b60 2024-03-16T09:16:44.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6ac5422740 1 -- 172.21.15.112:0/3621207859 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ab4032d30 con 0x55fd02c25b60 2024-03-16T09:16:44.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.790+0000 7f6a9b7fe640 1 -- 172.21.15.112:0/3621207859 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f6ab00047a0 con 0x55fd02c25b60 2024-03-16T09:16:44.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.791+0000 7f6a9b7fe640 1 -- 172.21.15.112:0/3621207859 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6ab0012270 con 0x55fd02c25b60 2024-03-16T09:16:44.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.791+0000 7f6ac5422740 10 client.25468 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:44.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.791+0000 7f6ac5422740 1 -- 172.21.15.112:0/3621207859 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55fd0289e3f0 con 0x55fd02c25b60 2024-03-16T09:16:44.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.791+0000 7f6a9b7fe640 1 -- 172.21.15.112:0/3621207859 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6ab4032d30 con 0x55fd02c25b60 2024-03-16T09:16:44.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.792+0000 7f6a9b7fe640 1 -- 172.21.15.112:0/3621207859 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f6ab001c470 con 0x55fd02c25b60 2024-03-16T09:16:44.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.792+0000 7f6ac5422740 10 client.25468 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:44.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.792+0000 7f6ac5422740 10 client.25468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:44.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.792+0000 7f6ac5422740 20 client.25468 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:44.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.792+0000 7f6ac5422740 1 --2- 172.21.15.112:0/3621207859 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55fd02c3b170 0x55fd02c3d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.792+0000 7f6ac5422740 4 client.25468 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:16:44.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.792+0000 7f6ac5422740 1 -- 172.21.15.112:0/3621207859 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd02b97a40 con 0x55fd02c3b170 2024-03-16T09:16:44.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.792+0000 7f6ac0cf8640 1 --2- 172.21.15.112:0/3621207859 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55fd02c3b170 0x55fd02c3d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.792+0000 7f6a9b7fe640 10 client.25468 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:44.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.792+0000 7f6ac0cf8640 1 --2- 172.21.15.112:0/3621207859 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55fd02c3b170 0x55fd02c3d5a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d2a027c6d6acd0e server_cookie=c61cedd512141a46 in_seq=0 out_seq=0 2024-03-16T09:16:44.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.793+0000 7f6a9b7fe640 1 -- 172.21.15.112:0/3621207859 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fd02b97a40 con 0x55fd02c3b170 2024-03-16T09:16:44.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.793+0000 7f6a9b7fe640 10 client.25468 handle_command_reply: tid=0 2024-03-16T09:16:44.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.793+0000 7f6ac5422740 1 client.25468 shutdown 2024-03-16T09:16:44.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.793+0000 7f6ac5422740 1 -- 172.21.15.112:0/3621207859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c25b60 msgr2=0x55fd02c2fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:44.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.793+0000 7f6ac5422740 1 --2- 172.21.15.112:0/3621207859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c25b60 0x55fd02c2fb70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00095e0 tx=0x7f6ab000f790 comp rx=0 tx=0).stop 2024-03-16T09:16:44.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.793+0000 7f6ac5422740 1 -- 172.21.15.112:0/3621207859 shutdown_connections 2024-03-16T09:16:44.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.793+0000 7f6ac5422740 1 --2- 172.21.15.112:0/3621207859 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55fd02c3b170 0x55fd02c3d5a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.793+0000 7f6a9b7fe640 0 client.25468 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:44.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.793+0000 7f6ac5422740 1 --2- 172.21.15.112:0/3621207859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c2acd0 0x55fd02c36bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.793+0000 7f6ac5422740 1 --2- 172.21.15.112:0/3621207859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c25b60 0x55fd02c2fb70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.793+0000 7f6ac5422740 1 --2- 172.21.15.112:0/3621207859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25170 0x55fd02c2f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.793+0000 7f6ac5422740 1 -- 172.21.15.112:0/3621207859 >> 172.21.15.112:0/3621207859 conn(0x55fd02c206e0 msgr2=0x55fd02c1f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:44.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.793+0000 7f6ac5422740 1 -- 172.21.15.112:0/3621207859 shutdown_connections 2024-03-16T09:16:44.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.793+0000 7f6ac5422740 1 -- 172.21.15.112:0/3621207859 wait complete. 2024-03-16T09:16:44.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.793+0000 7f6ac5422740 20 client.25468 trim_cache size 0 max 16384 2024-03-16T09:16:44.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.810+0000 7f6ac5422740 1 Processor -- start 2024-03-16T09:16:44.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.810+0000 7f6ac5422740 1 -- start start 2024-03-16T09:16:44.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.811+0000 7f6ac5422740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c25170 0x55fd02c1f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.811+0000 7f6ac5422740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c25b60 0x55fd02c36b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.811+0000 7f6ac5422740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c2acd0 0x55fd02c1f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.811+0000 7f6ac5422740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc1ba450 con 0x55fd02c2acd0 2024-03-16T09:16:44.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.811+0000 7f6ac5422740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc1b7d20 con 0x55fd02c25b60 2024-03-16T09:16:44.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.811+0000 7f6ac5422740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc1b5b70 con 0x55fd02c25170 2024-03-16T09:16:44.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.811+0000 7f6abb7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c25b60 0x55fd02c36b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.811+0000 7f6abb7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c25b60 0x55fd02c36b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:35522/0 (socket says 172.21.15.112:35522) 2024-03-16T09:16:44.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.811+0000 7f6abb7fe640 1 -- 172.21.15.112:0/3299317199 learned_addr learned my addr 172.21.15.112:0/3299317199 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:44.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.811+0000 7f6ac0cf8640 1 --2- 172.21.15.112:0/3299317199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c2acd0 0x55fd02c1f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.811+0000 7f6ac0cf8640 1 --2- 172.21.15.112:0/3299317199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c25170 0x55fd02c1f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.811+0000 7f6abb7fe640 1 -- 172.21.15.112:0/3299317199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c25170 msgr2=0x55fd02c1f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:44.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.811+0000 7f6abb7fe640 1 --2- 172.21.15.112:0/3299317199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c25170 0x55fd02c1f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.811+0000 7f6abb7fe640 1 -- 172.21.15.112:0/3299317199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c2acd0 msgr2=0x55fd02c1f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:44.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.811+0000 7f6abb7fe640 1 --2- 172.21.15.112:0/3299317199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c2acd0 0x55fd02c1f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.811+0000 7f6abb7fe640 1 -- 172.21.15.112:0/3299317199 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd0285e0f0 con 0x55fd02c25b60 2024-03-16T09:16:44.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.811+0000 7f6ac0cf8640 1 --2- 172.21.15.112:0/3299317199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c25170 0x55fd02c1f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:16:44.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.812+0000 7f6abb7fe640 1 --2- 172.21.15.112:0/3299317199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c25b60 0x55fd02c36b70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0012870 tx=0x7f6ab000fd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9f16b02298eb6ae server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:44.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.812+0000 7f6aa1ffb640 1 -- 172.21.15.112:0/3299317199 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab001c5f0 con 0x55fd02c25b60 2024-03-16T09:16:44.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.812+0000 7f6aa1ffb640 1 -- 172.21.15.112:0/3299317199 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6ab002b070 con 0x55fd02c25b60 2024-03-16T09:16:44.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.812+0000 7f6aa1ffb640 1 -- 172.21.15.112:0/3299317199 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0004a00 con 0x55fd02c25b60 2024-03-16T09:16:44.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.813+0000 7f6ac5422740 1 -- 172.21.15.112:0/3299317199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c25b60 msgr2=0x55fd02c36b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:44.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.813+0000 7f6ac5422740 1 --2- 172.21.15.112:0/3299317199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c25b60 0x55fd02c36b70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0012870 tx=0x7f6ab000fd10 comp rx=0 tx=0).stop 2024-03-16T09:16:44.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.813+0000 7f6ac5422740 1 -- 172.21.15.112:0/3299317199 shutdown_connections 2024-03-16T09:16:44.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.813+0000 7f6ac5422740 1 --2- 172.21.15.112:0/3299317199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c2acd0 0x55fd02c1f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.813+0000 7f6ac5422740 1 --2- 172.21.15.112:0/3299317199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c25b60 0x55fd02c36b70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.813+0000 7f6ac5422740 1 --2- 172.21.15.112:0/3299317199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c25170 0x55fd02c1f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.813+0000 7f6ac5422740 1 -- 172.21.15.112:0/3299317199 >> 172.21.15.112:0/3299317199 conn(0x55fd02c206e0 msgr2=0x55fd02c2f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:44.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.813+0000 7f6ac5422740 1 -- 172.21.15.112:0/3299317199 shutdown_connections 2024-03-16T09:16:44.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.813+0000 7f6ac5422740 1 -- 172.21.15.112:0/3299317199 wait complete. 2024-03-16T09:16:44.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.813+0000 7f6ac5422740 1 -- start start 2024-03-16T09:16:44.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6ac5422740 1 Processor -- start 2024-03-16T09:16:44.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6ac5422740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c25170 0x55fd02c34a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6ac5422740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25b60 0x55fd02c34f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6ac5422740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c2acd0 0x55fd02c35f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6ac5422740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc1b61e0 con 0x55fd02c25b60 2024-03-16T09:16:44.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6ac5422740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc1b7650 con 0x55fd02c2acd0 2024-03-16T09:16:44.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6ac5422740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc1b68b0 con 0x55fd02c25170 2024-03-16T09:16:44.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6ac0cf8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c25170 0x55fd02c34a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6ac5422740 10 client.0 _ll_register_callbacks cb 0x7f6ab0009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:44.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6abb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25b60 0x55fd02c34f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6ac0cf8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c25170 0x55fd02c34a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:56466/0 (socket says 172.21.15.112:56466) 2024-03-16T09:16:44.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6abb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25b60 0x55fd02c34f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40688/0 (socket says 172.21.15.112:40688) 2024-03-16T09:16:44.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6abb7fe640 1 -- 172.21.15.112:0/1900034059 learned_addr learned my addr 172.21.15.112:0/1900034059 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:44.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6ac0cf8640 1 --2- 172.21.15.112:0/1900034059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c2acd0 0x55fd02c35f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6abb7fe640 1 -- 172.21.15.112:0/1900034059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c25170 msgr2=0x55fd02c34a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:44.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6abb7fe640 1 --2- 172.21.15.112:0/1900034059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c25170 0x55fd02c34a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6abb7fe640 1 -- 172.21.15.112:0/1900034059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c2acd0 msgr2=0x55fd02c35f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:44.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6abb7fe640 1 --2- 172.21.15.112:0/1900034059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c2acd0 0x55fd02c35f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6abb7fe640 1 -- 172.21.15.112:0/1900034059 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd02b519d0 con 0x55fd02c25b60 2024-03-16T09:16:44.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.814+0000 7f6ac0cf8640 1 --2- 172.21.15.112:0/1900034059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c25170 0x55fd02c34a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:16:44.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.815+0000 7f6abb7fe640 1 --2- 172.21.15.112:0/1900034059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25b60 0x55fd02c34f50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6ab001c8e0 tx=0x7f6ab001c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4185f1487b57dc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:44.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.815+0000 7f6a9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:16:44.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.815+0000 7f6a9b7fe640 1 -- 172.21.15.112:0/1900034059 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0005300 con 0x55fd02c25b60 2024-03-16T09:16:44.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.815+0000 7f6ac5422740 1 -- 172.21.15.112:0/1900034059 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ab002b070 con 0x55fd02c25b60 2024-03-16T09:16:44.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.815+0000 7f6a9b7fe640 1 -- 172.21.15.112:0/1900034059 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f6ab00054a0 con 0x55fd02c25b60 2024-03-16T09:16:44.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.815+0000 7f6a9b7fe640 1 -- 172.21.15.112:0/1900034059 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6ab0004780 con 0x55fd02c25b60 2024-03-16T09:16:44.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.815+0000 7f6ac5422740 10 client.15612 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:44.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.815+0000 7f6ac5422740 1 -- 172.21.15.112:0/1900034059 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55fd02ca19e0 con 0x55fd02c25b60 2024-03-16T09:16:44.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.815+0000 7f6a9b7fe640 1 -- 172.21.15.112:0/1900034059 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6ab002b070 con 0x55fd02c25b60 2024-03-16T09:16:44.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.815+0000 7f6a9b7fe640 1 -- 172.21.15.112:0/1900034059 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f6ab0010c20 con 0x55fd02c25b60 2024-03-16T09:16:44.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.815+0000 7f6ac5422740 10 client.15612 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:44.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.815+0000 7f6ac5422740 10 client.15612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:44.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.815+0000 7f6ac5422740 20 client.15612 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:44.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.815+0000 7f6ac5422740 1 --2- 172.21.15.112:0/1900034059 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55fd02ca2390 0x55fd02ca47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.815+0000 7f6ac5422740 4 client.15612 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T09:16:44.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.815+0000 7f6ac5422740 1 -- 172.21.15.112:0/1900034059 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fd02b91d90 con 0x55fd02ca2390 2024-03-16T09:16:44.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.816+0000 7f6ac0cf8640 1 --2- 172.21.15.112:0/1900034059 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55fd02ca2390 0x55fd02ca47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.816+0000 7f6a9b7fe640 10 client.15612 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:44.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.816+0000 7f6ac0cf8640 1 --2- 172.21.15.112:0/1900034059 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55fd02ca2390 0x55fd02ca47b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=504124020842c77f server_cookie=badc110ba058de55 in_seq=0 out_seq=0 2024-03-16T09:16:44.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.816+0000 7f6a9b7fe640 1 -- 172.21.15.112:0/1900034059 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fd02b91d90 con 0x55fd02ca2390 2024-03-16T09:16:44.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.816+0000 7f6a9b7fe640 10 client.15612 handle_command_reply: tid=0 2024-03-16T09:16:44.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.816+0000 7f6ac5422740 1 client.15612 shutdown 2024-03-16T09:16:44.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.816+0000 7f6ac5422740 1 -- 172.21.15.112:0/1900034059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25b60 msgr2=0x55fd02c34f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:44.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.816+0000 7f6ac5422740 1 --2- 172.21.15.112:0/1900034059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25b60 0x55fd02c34f50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6ab001c8e0 tx=0x7f6ab001c820 comp rx=0 tx=0).stop 2024-03-16T09:16:44.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.817+0000 7f6ac5422740 1 -- 172.21.15.112:0/1900034059 shutdown_connections 2024-03-16T09:16:44.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.817+0000 7f6ac5422740 1 --2- 172.21.15.112:0/1900034059 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55fd02ca2390 0x55fd02ca47b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.817+0000 7f6ac5422740 1 --2- 172.21.15.112:0/1900034059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fd02c2acd0 0x55fd02c35f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.817+0000 7f6ac5422740 1 --2- 172.21.15.112:0/1900034059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd02c25b60 0x55fd02c34f50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.817+0000 7f6a9b7fe640 0 client.15612 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:44.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.817+0000 7f6ac5422740 1 --2- 172.21.15.112:0/1900034059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd02c25170 0x55fd02c34a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.817+0000 7f6ac5422740 1 -- 172.21.15.112:0/1900034059 >> 172.21.15.112:0/1900034059 conn(0x55fd02c206e0 msgr2=0x55fd02c23940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:44.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.817+0000 7f6ac5422740 1 -- 172.21.15.112:0/1900034059 shutdown_connections 2024-03-16T09:16:44.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.817+0000 7f6ac5422740 1 -- 172.21.15.112:0/1900034059 wait complete. 2024-03-16T09:16:44.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.817+0000 7f6ac5422740 20 client.15612 trim_cache size 0 max 16384 2024-03-16T09:16:44.825 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:16:44.825 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-03-16T09:16:44.826 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-03-16T09:16:44.826 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:16:44.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.817+0000 7f6ac275b640 1 -- 172.21.15.112:0/466815244 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6aa40a2900 msgr2=0x7f6aa40a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:44.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.817+0000 7f6ac275b640 1 --2- 172.21.15.112:0/466815244 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6aa40a2900 0x7f6aa40a4dc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55fd02c25f60 tx=0x7f6aac00cf70 comp rx=0 tx=0).stop 2024-03-16T09:16:44.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.817+0000 7f6ac275b640 1 -- 172.21.15.112:0/466815244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6abc1022c0 msgr2=0x7f6abc112440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:44.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.817+0000 7f6ac275b640 1 --2- 172.21.15.112:0/466815244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6abc1022c0 0x7f6abc112440 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00e9f0 tx=0x7f6aac00eeb0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.818+0000 7f6ac275b640 1 -- 172.21.15.112:0/466815244 shutdown_connections 2024-03-16T09:16:44.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.818+0000 7f6ac275b640 1 --2- 172.21.15.112:0/466815244 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f6aa40a2900 0x7f6aa40a4dc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.818+0000 7f6ac275b640 1 --2- 172.21.15.112:0/466815244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6abc107520 0x7f6abc110a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.818+0000 7f6ac275b640 1 --2- 172.21.15.112:0/466815244 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6abc104bf0 0x7f6abc112980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.818+0000 7f6ac275b640 1 --2- 172.21.15.112:0/466815244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6abc1022c0 0x7f6abc112440 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.818+0000 7f6ac275b640 1 -- 172.21.15.112:0/466815244 >> 172.21.15.112:0/466815244 conn(0x7f6abc0fbc70 msgr2=0x7f6abc102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:44.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.818+0000 7f6ac275b640 1 -- 172.21.15.112:0/466815244 shutdown_connections 2024-03-16T09:16:44.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.818+0000 7f6ac275b640 1 -- 172.21.15.112:0/466815244 wait complete. 2024-03-16T09:16:44.834 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T09:16:44.834 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:16:44.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.937+0000 7f37746ff640 1 -- 172.21.15.112:0/166373425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f376c1018f0 msgr2=0x7f376c101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:44.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.937+0000 7f37746ff640 1 --2- 172.21.15.112:0/166373425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f376c1018f0 0x7f376c101d70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f375c009900 tx=0x7f375c02bc30 comp rx=0 tx=0).stop 2024-03-16T09:16:44.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.937+0000 7f37746ff640 1 -- 172.21.15.112:0/166373425 shutdown_connections 2024-03-16T09:16:44.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.937+0000 7f37746ff640 1 --2- 172.21.15.112:0/166373425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f376c1022b0 0x7f376c108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.937+0000 7f37746ff640 1 --2- 172.21.15.112:0/166373425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f376c1018f0 0x7f376c101d70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.937+0000 7f37746ff640 1 --2- 172.21.15.112:0/166373425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f376c100700 0x7f376c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.937+0000 7f37746ff640 1 -- 172.21.15.112:0/166373425 >> 172.21.15.112:0/166373425 conn(0x7f376c0fbc70 msgr2=0x7f376c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:44.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.937+0000 7f37746ff640 1 -- 172.21.15.112:0/166373425 shutdown_connections 2024-03-16T09:16:44.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.937+0000 7f37746ff640 1 -- 172.21.15.112:0/166373425 wait complete. 2024-03-16T09:16:44.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.938+0000 7f37746ff640 1 Processor -- start 2024-03-16T09:16:44.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.938+0000 7f37746ff640 1 -- start start 2024-03-16T09:16:44.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.938+0000 7f37746ff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f376c100700 0x7f376c197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.938+0000 7f37746ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f376c1018f0 0x7f376c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.939+0000 7f37746ff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f376c1022b0 0x7f376c19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.939+0000 7f3771c9a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f376c1018f0 0x7f376c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.939+0000 7f37746ff640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f376c10c460 con 0x7f376c1018f0 2024-03-16T09:16:44.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.939+0000 7f37746ff640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f376c10c2e0 con 0x7f376c100700 2024-03-16T09:16:44.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.939+0000 7f3771c9a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f376c1018f0 0x7f376c198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40714/0 (socket says 172.21.15.112:40714) 2024-03-16T09:16:44.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.939+0000 7f37746ff640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f376c10c5e0 con 0x7f376c1022b0 2024-03-16T09:16:44.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.939+0000 7f3771c9a640 1 -- 172.21.15.112:0/2041218129 learned_addr learned my addr 172.21.15.112:0/2041218129 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:44.943 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.939+0000 7f377249b640 1 --2- 172.21.15.112:0/2041218129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f376c100700 0x7f376c197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.939+0000 7f3772c9c640 1 --2- 172.21.15.112:0/2041218129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f376c1022b0 0x7f376c19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.939+0000 7f377249b640 1 -- 172.21.15.112:0/2041218129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f376c1022b0 msgr2=0x7f376c19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:44.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.939+0000 7f377249b640 1 --2- 172.21.15.112:0/2041218129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f376c1022b0 0x7f376c19f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.939+0000 7f377249b640 1 -- 172.21.15.112:0/2041218129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f376c1018f0 msgr2=0x7f376c198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:44.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.939+0000 7f377249b640 1 --2- 172.21.15.112:0/2041218129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f376c1018f0 0x7f376c198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:44.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.939+0000 7f377249b640 1 -- 172.21.15.112:0/2041218129 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376c19f800 con 0x7f376c100700 2024-03-16T09:16:44.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.939+0000 7f3771c9a640 1 --2- 172.21.15.112:0/2041218129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f376c1018f0 0x7f376c198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:44.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.939+0000 7f377249b640 1 --2- 172.21.15.112:0/2041218129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f376c100700 0x7f376c197be0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f376800e9c0 tx=0x7f376800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:44.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.939+0000 7f37637fe640 1 -- 172.21.15.112:0/2041218129 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376800cd80 con 0x7f376c100700 2024-03-16T09:16:44.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.939+0000 7f37637fe640 1 -- 172.21.15.112:0/2041218129 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3768010930 con 0x7f376c100700 2024-03-16T09:16:44.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.939+0000 7f37637fe640 1 -- 172.21.15.112:0/2041218129 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768018a30 con 0x7f376c100700 2024-03-16T09:16:44.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.940+0000 7f37746ff640 1 -- 172.21.15.112:0/2041218129 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376c102b70 con 0x7f376c100700 2024-03-16T09:16:44.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.940+0000 7f37746ff640 1 -- 172.21.15.112:0/2041218129 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376c103070 con 0x7f376c100700 2024-03-16T09:16:44.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.940+0000 7f37746ff640 1 -- 172.21.15.112:0/2041218129 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f376c103300 con 0x7f376c100700 2024-03-16T09:16:44.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.942+0000 7f37637fe640 1 -- 172.21.15.112:0/2041218129 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7f376801f020 con 0x7f376c100700 2024-03-16T09:16:44.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.942+0000 7f37637fe640 1 --2- 172.21.15.112:0/2041218129 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f37580a2890 0x7f37580a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:44.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.943+0000 7f3771c9a640 1 --2- 172.21.15.112:0/2041218129 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f37580a2890 0x7f37580a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:44.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.943+0000 7f37637fe640 1 -- 172.21.15.112:0/2041218129 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f37680cf250 con 0x7f376c100700 2024-03-16T09:16:44.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.943+0000 7f3771c9a640 1 --2- 172.21.15.112:0/2041218129 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f37580a2890 0x7f37580a4d50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f376c1990f0 tx=0x7f375c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:44.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:44.949+0000 7f37637fe640 1 -- 172.21.15.112:0/2041218129 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f376809c140 con 0x7f376c100700 2024-03-16T09:16:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.195+0000 7f37746ff640 1 -- 172.21.15.112:0/2041218129 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f376c1034d0 con 0x7f376c100700 2024-03-16T09:16:45.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.196+0000 7f37637fe640 1 -- 172.21.15.112:0/2041218129 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11881 (secure 0 0 0) 0x7f37680afe20 con 0x7f376c100700 2024-03-16T09:16:45.199 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:16:45.199 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3939748854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3939748854},{"type":"v1","addr":"172.21.15.112:6839","nonce":3939748854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3118523258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3118523258},{"type":"v1","addr":"172.21.15.112:6841","nonce":3118523258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T09:13:36.450216+0000","modified":"2024-03-16T09:15:34.982470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/1682076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":1682076},{"type":"v1","addr":"172.21.15.194:6837","nonce":1682076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.194:6839/2528943309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2528943309},{"type":"v1","addr":"172.21.15.194:6839","nonce":2528943309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/272895965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":272895965},{"type":"v1","addr":"172.21.15.159:6839","nonce":272895965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24452,24466]},"id":1}]} 2024-03-16T09:16:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 22 2024-03-16T09:16:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.198+0000 7f37746ff640 1 -- 172.21.15.112:0/2041218129 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f37580a2890 msgr2=0x7f37580a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.198+0000 7f37746ff640 1 --2- 172.21.15.112:0/2041218129 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f37580a2890 0x7f37580a4d50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f376c1990f0 tx=0x7f375c005e30 comp rx=0 tx=0).stop 2024-03-16T09:16:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.199+0000 7f37746ff640 1 -- 172.21.15.112:0/2041218129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f376c100700 msgr2=0x7f376c197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.199+0000 7f37746ff640 1 --2- 172.21.15.112:0/2041218129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f376c100700 0x7f376c197be0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f376800e9c0 tx=0x7f376800ee80 comp rx=0 tx=0).stop 2024-03-16T09:16:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.199+0000 7f37746ff640 1 -- 172.21.15.112:0/2041218129 shutdown_connections 2024-03-16T09:16:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.199+0000 7f37746ff640 1 --2- 172.21.15.112:0/2041218129 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f37580a2890 0x7f37580a4d50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.199+0000 7f37746ff640 1 --2- 172.21.15.112:0/2041218129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f376c1022b0 0x7f376c19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.199+0000 7f37746ff640 1 --2- 172.21.15.112:0/2041218129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f376c1018f0 0x7f376c198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.199+0000 7f37746ff640 1 --2- 172.21.15.112:0/2041218129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f376c100700 0x7f376c197be0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.199+0000 7f37746ff640 1 -- 172.21.15.112:0/2041218129 >> 172.21.15.112:0/2041218129 conn(0x7f376c0fbc70 msgr2=0x7f376c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.199+0000 7f37746ff640 1 -- 172.21.15.112:0/2041218129 shutdown_connections 2024-03-16T09:16:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.199+0000 7f37746ff640 1 -- 172.21.15.112:0/2041218129 wait complete. 2024-03-16T09:16:45.213 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T09:16:45.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.320+0000 7f1579d60640 1 -- 172.21.15.112:0/3214272351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1574102290 msgr2=0x7f1574108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.320+0000 7f1579d60640 1 --2- 172.21.15.112:0/3214272351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1574102290 0x7f1574108b20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f156c009ca0 tx=0x7f156c02c0f0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.320+0000 7f1579d60640 1 -- 172.21.15.112:0/3214272351 shutdown_connections 2024-03-16T09:16:45.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.320+0000 7f1579d60640 1 --2- 172.21.15.112:0/3214272351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1574102290 0x7f1574108b20 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.320+0000 7f1579d60640 1 --2- 172.21.15.112:0/3214272351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15741018d0 0x7f1574101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.320+0000 7f1579d60640 1 --2- 172.21.15.112:0/3214272351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15741006e0 0x7f1574100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.320+0000 7f1579d60640 1 -- 172.21.15.112:0/3214272351 >> 172.21.15.112:0/3214272351 conn(0x7f15740fbc70 msgr2=0x7f15740fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:45.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.320+0000 7f1579d60640 1 -- 172.21.15.112:0/3214272351 shutdown_connections 2024-03-16T09:16:45.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.320+0000 7f1579d60640 1 -- 172.21.15.112:0/3214272351 wait complete. 2024-03-16T09:16:45.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.321+0000 7f1579d60640 1 Processor -- start 2024-03-16T09:16:45.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.321+0000 7f1579d60640 1 -- start start 2024-03-16T09:16:45.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.322+0000 7f1579d60640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15741006e0 0x7f1574197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.322+0000 7f1579d60640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15741018d0 0x7f15741980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.322+0000 7f1579d60640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1574102290 0x7f157419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.322+0000 7f1579d60640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f157410d5a0 con 0x7f15741018d0 2024-03-16T09:16:45.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.322+0000 7f1579d60640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f157410d420 con 0x7f1574102290 2024-03-16T09:16:45.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.322+0000 7f1579d60640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f157410d720 con 0x7f15741006e0 2024-03-16T09:16:45.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.322+0000 7f1572ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15741018d0 0x7f15741980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.322+0000 7f1572ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15741018d0 0x7f15741980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40738/0 (socket says 172.21.15.112:40738) 2024-03-16T09:16:45.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.322+0000 7f1572ffd640 1 -- 172.21.15.112:0/2433369490 learned_addr learned my addr 172.21.15.112:0/2433369490 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:45.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.322+0000 7f15737fe640 1 --2- 172.21.15.112:0/2433369490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15741006e0 0x7f1574197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.322+0000 7f1573fff640 1 --2- 172.21.15.112:0/2433369490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1574102290 0x7f157419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.323+0000 7f1572ffd640 1 -- 172.21.15.112:0/2433369490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15741006e0 msgr2=0x7f1574197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.323+0000 7f1572ffd640 1 --2- 172.21.15.112:0/2433369490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15741006e0 0x7f1574197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.323+0000 7f1572ffd640 1 -- 172.21.15.112:0/2433369490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1574102290 msgr2=0x7f157419f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.323+0000 7f1572ffd640 1 --2- 172.21.15.112:0/2433369490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1574102290 0x7f157419f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.323+0000 7f1572ffd640 1 -- 172.21.15.112:0/2433369490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157419f7d0 con 0x7f15741018d0 2024-03-16T09:16:45.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.323+0000 7f1573fff640 1 --2- 172.21.15.112:0/2433369490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1574102290 0x7f157419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:45.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.323+0000 7f15737fe640 1 --2- 172.21.15.112:0/2433369490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15741006e0 0x7f1574197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:45.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.323+0000 7f1572ffd640 1 --2- 172.21.15.112:0/2433369490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15741018d0 0x7f15741980f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f156800e9f0 tx=0x7f156800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:45.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.323+0000 7f1570ff9640 1 -- 172.21.15.112:0/2433369490 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156800cd80 con 0x7f15741018d0 2024-03-16T09:16:45.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.323+0000 7f1579d60640 1 -- 172.21.15.112:0/2433369490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f157419fac0 con 0x7f15741018d0 2024-03-16T09:16:45.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.323+0000 7f1579d60640 1 -- 172.21.15.112:0/2433369490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15741a0010 con 0x7f15741018d0 2024-03-16T09:16:45.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.323+0000 7f1570ff9640 1 -- 172.21.15.112:0/2433369490 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1568005520 con 0x7f15741018d0 2024-03-16T09:16:45.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.323+0000 7f1570ff9640 1 -- 172.21.15.112:0/2433369490 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1568010690 con 0x7f15741018d0 2024-03-16T09:16:45.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.326+0000 7f1570ff9640 1 -- 172.21.15.112:0/2433369490 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7f15680046c0 con 0x7f15741018d0 2024-03-16T09:16:45.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.327+0000 7f1570ff9640 1 --2- 172.21.15.112:0/2433369490 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f155c0a2910 0x7f155c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.327+0000 7f1570ff9640 1 -- 172.21.15.112:0/2433369490 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f15680cef30 con 0x7f15741018d0 2024-03-16T09:16:45.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.327+0000 7f15737fe640 1 --2- 172.21.15.112:0/2433369490 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f155c0a2910 0x7f155c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.328+0000 7f15737fe640 1 --2- 172.21.15.112:0/2433369490 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f155c0a2910 0x7f155c0a4dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1574101730 tx=0x7f1564005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:45.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.328+0000 7f157ca20740 1 Processor -- start 2024-03-16T09:16:45.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.328+0000 7f157ca20740 1 -- start start 2024-03-16T09:16:45.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.328+0000 7f157ca20740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87ad7d0 0x5575d87adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.328+0000 7f157ca20740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ae1c0 0x5575d87b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.328+0000 7f157ca20740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87b1320 0x5575d87be190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.328+0000 7f157ca20740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5575d8712490 con 0x5575d87ae1c0 2024-03-16T09:16:45.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.328+0000 7f157ca20740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5575d86bac90 con 0x5575d87b1320 2024-03-16T09:16:45.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.328+0000 7f157ca20740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5575d870e530 con 0x5575d87ad7d0 2024-03-16T09:16:45.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.328+0000 7f15737fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87ad7d0 0x5575d87adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.328+0000 7f15737fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87ad7d0 0x5575d87adbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:56524/0 (socket says 172.21.15.112:56524) 2024-03-16T09:16:45.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.328+0000 7f15737fe640 1 -- 172.21.15.112:0/169586588 learned_addr learned my addr 172.21.15.112:0/169586588 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:45.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.328+0000 7f1572ffd640 1 --2- 172.21.15.112:0/169586588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ae1c0 0x5575d87b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.328+0000 7f1573fff640 1 --2- 172.21.15.112:0/169586588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87b1320 0x5575d87be190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.329+0000 7f15737fe640 1 -- 172.21.15.112:0/169586588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87b1320 msgr2=0x5575d87be190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.329+0000 7f15737fe640 1 --2- 172.21.15.112:0/169586588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87b1320 0x5575d87be190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.329+0000 7f15737fe640 1 -- 172.21.15.112:0/169586588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ae1c0 msgr2=0x5575d87b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.329+0000 7f15737fe640 1 --2- 172.21.15.112:0/169586588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ae1c0 0x5575d87b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.329+0000 7f15737fe640 1 -- 172.21.15.112:0/169586588 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575d8775500 con 0x5575d87ad7d0 2024-03-16T09:16:45.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.329+0000 7f1572ffd640 1 --2- 172.21.15.112:0/169586588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ae1c0 0x5575d87b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:45.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.329+0000 7f15737fe640 1 --2- 172.21.15.112:0/169586588 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87ad7d0 0x5575d87adbf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1564009ba0 tx=0x7f1564014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d4f51394f817b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:45.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.329+0000 7f1559ffb640 1 -- 172.21.15.112:0/169586588 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156400f480 con 0x5575d87ad7d0 2024-03-16T09:16:45.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.329+0000 7f1559ffb640 1 -- 172.21.15.112:0/169586588 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f156400fb20 con 0x5575d87ad7d0 2024-03-16T09:16:45.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.329+0000 7f1559ffb640 1 -- 172.21.15.112:0/169586588 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156401cca0 con 0x5575d87ad7d0 2024-03-16T09:16:45.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.330+0000 7f157ca20740 1 -- 172.21.15.112:0/169586588 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87ad7d0 msgr2=0x5575d87adbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.330+0000 7f157ca20740 1 --2- 172.21.15.112:0/169586588 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87ad7d0 0x5575d87adbf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1564009ba0 tx=0x7f1564014000 comp rx=0 tx=0).stop 2024-03-16T09:16:45.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.330+0000 7f157ca20740 1 -- 172.21.15.112:0/169586588 shutdown_connections 2024-03-16T09:16:45.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.330+0000 7f157ca20740 1 --2- 172.21.15.112:0/169586588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87b1320 0x5575d87be190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.330+0000 7f157ca20740 1 --2- 172.21.15.112:0/169586588 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ae1c0 0x5575d87b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.330+0000 7f157ca20740 1 --2- 172.21.15.112:0/169586588 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87ad7d0 0x5575d87adbf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.330+0000 7f157ca20740 1 -- 172.21.15.112:0/169586588 >> 172.21.15.112:0/169586588 conn(0x5575d87a8dc0 msgr2=0x5575d87a91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:45.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.330+0000 7f157ca20740 1 -- 172.21.15.112:0/169586588 shutdown_connections 2024-03-16T09:16:45.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.330+0000 7f157ca20740 1 -- 172.21.15.112:0/169586588 wait complete. 2024-03-16T09:16:45.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.331+0000 7f157ca20740 1 -- start start 2024-03-16T09:16:45.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.331+0000 7f157ca20740 1 Processor -- start 2024-03-16T09:16:45.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.331+0000 7f157ca20740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ad7d0 0x5575d87b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.331+0000 7f15737fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ad7d0 0x5575d87b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.331+0000 7f15737fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ad7d0 0x5575d87b7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40756/0 (socket says 172.21.15.112:40756) 2024-03-16T09:16:45.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.331+0000 7f15737fe640 1 -- 172.21.15.112:0/4052778053 learned_addr learned my addr 172.21.15.112:0/4052778053 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:45.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.331+0000 7f157ca20740 1 --2- 172.21.15.112:0/4052778053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87ae1c0 0x5575d87b8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.331+0000 7f157ca20740 1 --2- 172.21.15.112:0/4052778053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87b1320 0x5575d87bd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.331+0000 7f157ca20740 1 -- 172.21.15.112:0/4052778053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5575d87b1d60 con 0x5575d87ad7d0 2024-03-16T09:16:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.332+0000 7f157ca20740 1 -- 172.21.15.112:0/4052778053 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5575d86ce230 con 0x5575d87b1320 2024-03-16T09:16:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.332+0000 7f157ca20740 1 -- 172.21.15.112:0/4052778053 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5575d87b1ee0 con 0x5575d87ae1c0 2024-03-16T09:16:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.332+0000 7f1572ffd640 1 --2- 172.21.15.112:0/4052778053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87ae1c0 0x5575d87b8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.332+0000 7f1573fff640 1 --2- 172.21.15.112:0/4052778053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87b1320 0x5575d87bd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.332+0000 7f157ca20740 10 client.0 _ll_register_callbacks cb 0x5575d870a010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.332+0000 7f15737fe640 1 -- 172.21.15.112:0/4052778053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87ae1c0 msgr2=0x5575d87b8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.332+0000 7f15737fe640 1 --2- 172.21.15.112:0/4052778053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87ae1c0 0x5575d87b8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.332+0000 7f15737fe640 1 -- 172.21.15.112:0/4052778053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87b1320 msgr2=0x5575d87bd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.332+0000 7f15737fe640 1 --2- 172.21.15.112:0/4052778053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87b1320 0x5575d87bd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.332+0000 7f15737fe640 1 -- 172.21.15.112:0/4052778053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575d86dd100 con 0x5575d87ad7d0 2024-03-16T09:16:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.332+0000 7f1573fff640 1 --2- 172.21.15.112:0/4052778053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87b1320 0x5575d87bd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.332+0000 7f1572ffd640 1 --2- 172.21.15.112:0/4052778053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87ae1c0 0x5575d87b8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:16:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.332+0000 7f15737fe640 1 --2- 172.21.15.112:0/4052778053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ad7d0 0x5575d87b7c10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1564003e50 tx=0x7f156401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf66cdc6ef07220b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.332+0000 7f1552ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:16:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.332+0000 7f1552ffd640 1 -- 172.21.15.112:0/4052778053 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156400fcd0 con 0x5575d87ad7d0 2024-03-16T09:16:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.332+0000 7f1552ffd640 1 -- 172.21.15.112:0/4052778053 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f156402b040 con 0x5575d87ad7d0 2024-03-16T09:16:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.332+0000 7f157ca20740 1 -- 172.21.15.112:0/4052778053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f156400fb20 con 0x5575d87ad7d0 2024-03-16T09:16:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.333+0000 7f1552ffd640 1 -- 172.21.15.112:0/4052778053 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f156400fe70 con 0x5575d87ad7d0 2024-03-16T09:16:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.333+0000 7f157ca20740 10 client.15630 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.333+0000 7f157ca20740 1 -- 172.21.15.112:0/4052778053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5575d84263f0 con 0x5575d87ad7d0 2024-03-16T09:16:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.333+0000 7f1552ffd640 1 -- 172.21.15.112:0/4052778053 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f156400f480 con 0x5575d87ad7d0 2024-03-16T09:16:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.333+0000 7f1552ffd640 1 -- 172.21.15.112:0/4052778053 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1564005240 con 0x5575d87ad7d0 2024-03-16T09:16:45.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.333+0000 7f157ca20740 10 client.15630 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:45.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.333+0000 7f157ca20740 10 client.15630 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T09:16:45.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.333+0000 7f157ca20740 20 client.15630 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:45.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.333+0000 7f157ca20740 1 --2- 172.21.15.112:0/4052778053 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x5575d87d47c0 0x5575d87d6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.333+0000 7f157ca20740 4 client.15630 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:16:45.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.333+0000 7f157ca20740 1 -- 172.21.15.112:0/4052778053 --> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f156400fb20 con 0x5575d87d47c0 2024-03-16T09:16:45.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.334+0000 7f1572ffd640 1 --2- 172.21.15.112:0/4052778053 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x5575d87d47c0 0x5575d87d6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.334+0000 7f1552ffd640 10 client.15630 ms_handle_connect on v2:172.21.15.159:6836/4023942203 2024-03-16T09:16:45.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.334+0000 7f1572ffd640 1 --2- 172.21.15.112:0/4052778053 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x5575d87d47c0 0x5575d87d6bb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57368015194d2520 server_cookie=7c9733bc63303208 in_seq=0 out_seq=0 2024-03-16T09:16:45.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.335+0000 7f1552ffd640 1 -- 172.21.15.112:0/4052778053 <== mds.2 v2:172.21.15.159:6836/4023942203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7f156400fb20 con 0x5575d87d47c0 2024-03-16T09:16:45.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.335+0000 7f1552ffd640 10 client.15630 handle_command_reply: tid=0 2024-03-16T09:16:45.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.335+0000 7f157ca20740 1 client.15630 shutdown 2024-03-16T09:16:45.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.335+0000 7f157ca20740 1 -- 172.21.15.112:0/4052778053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ad7d0 msgr2=0x5575d87b7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.335+0000 7f157ca20740 1 --2- 172.21.15.112:0/4052778053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ad7d0 0x5575d87b7c10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1564003e50 tx=0x7f156401c530 comp rx=0 tx=0).stop 2024-03-16T09:16:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.335+0000 7f157ca20740 1 -- 172.21.15.112:0/4052778053 shutdown_connections 2024-03-16T09:16:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.335+0000 7f157ca20740 1 --2- 172.21.15.112:0/4052778053 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x5575d87d47c0 0x5575d87d6bb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.335+0000 7f157ca20740 1 --2- 172.21.15.112:0/4052778053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87b1320 0x5575d87bd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.335+0000 7f1552ffd640 0 client.15630 ms_handle_reset on v2:172.21.15.159:6836/4023942203 2024-03-16T09:16:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.335+0000 7f157ca20740 1 --2- 172.21.15.112:0/4052778053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87ae1c0 0x5575d87b8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.335+0000 7f157ca20740 1 --2- 172.21.15.112:0/4052778053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ad7d0 0x5575d87b7c10 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.335+0000 7f157ca20740 1 -- 172.21.15.112:0/4052778053 >> 172.21.15.112:0/4052778053 conn(0x5575d87a8dc0 msgr2=0x5575d87a7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.336+0000 7f157ca20740 1 -- 172.21.15.112:0/4052778053 shutdown_connections 2024-03-16T09:16:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.336+0000 7f157ca20740 1 -- 172.21.15.112:0/4052778053 wait complete. 2024-03-16T09:16:45.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.336+0000 7f157ca20740 20 client.15630 trim_cache size 0 max 16384 2024-03-16T09:16:45.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f157ca20740 1 Processor -- start 2024-03-16T09:16:45.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f157ca20740 1 -- start start 2024-03-16T09:16:45.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f157ca20740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87ad7d0 0x5575d87b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f157ca20740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ae1c0 0x5575d87b7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f157ca20740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87b1320 0x5575d87a7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f157ca20740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15741a9490 con 0x5575d87ae1c0 2024-03-16T09:16:45.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f157ca20740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f15741a6d60 con 0x5575d87ad7d0 2024-03-16T09:16:45.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f157ca20740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f15741a4bb0 con 0x5575d87b1320 2024-03-16T09:16:45.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f15737fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87ad7d0 0x5575d87b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f1573fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87b1320 0x5575d87a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f15737fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87ad7d0 0x5575d87b7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:35606/0 (socket says 172.21.15.112:35606) 2024-03-16T09:16:45.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f15737fe640 1 -- 172.21.15.112:0/2695539693 learned_addr learned my addr 172.21.15.112:0/2695539693 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:45.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f1572ffd640 1 --2- 172.21.15.112:0/2695539693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ae1c0 0x5575d87b7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f15737fe640 1 -- 172.21.15.112:0/2695539693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87b1320 msgr2=0x5575d87a7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f15737fe640 1 --2- 172.21.15.112:0/2695539693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87b1320 0x5575d87a7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f15737fe640 1 -- 172.21.15.112:0/2695539693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ae1c0 msgr2=0x5575d87b7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f15737fe640 1 --2- 172.21.15.112:0/2695539693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ae1c0 0x5575d87b7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f15737fe640 1 -- 172.21.15.112:0/2695539693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575d84f07d0 con 0x5575d87ad7d0 2024-03-16T09:16:45.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f1573fff640 1 --2- 172.21.15.112:0/2695539693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87b1320 0x5575d87a7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:45.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.362+0000 7f1572ffd640 1 --2- 172.21.15.112:0/2695539693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ae1c0 0x5575d87b7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:16:45.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.363+0000 7f15737fe640 1 --2- 172.21.15.112:0/2695539693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87ad7d0 0x5575d87b7820 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f15640058c0 tx=0x7f1564004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=843a619018cf36cd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:45.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.363+0000 7f15597fa640 1 -- 172.21.15.112:0/2695539693 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564003f80 con 0x5575d87ad7d0 2024-03-16T09:16:45.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.363+0000 7f15597fa640 1 -- 172.21.15.112:0/2695539693 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f15640049a0 con 0x5575d87ad7d0 2024-03-16T09:16:45.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.363+0000 7f15597fa640 1 -- 172.21.15.112:0/2695539693 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156400f870 con 0x5575d87ad7d0 2024-03-16T09:16:45.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.364+0000 7f157ca20740 1 -- 172.21.15.112:0/2695539693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87ad7d0 msgr2=0x5575d87b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.364+0000 7f157ca20740 1 --2- 172.21.15.112:0/2695539693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87ad7d0 0x5575d87b7820 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f15640058c0 tx=0x7f1564004250 comp rx=0 tx=0).stop 2024-03-16T09:16:45.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.364+0000 7f157ca20740 1 -- 172.21.15.112:0/2695539693 shutdown_connections 2024-03-16T09:16:45.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.364+0000 7f157ca20740 1 --2- 172.21.15.112:0/2695539693 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87b1320 0x5575d87a7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.364+0000 7f157ca20740 1 --2- 172.21.15.112:0/2695539693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ae1c0 0x5575d87b7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.364+0000 7f157ca20740 1 --2- 172.21.15.112:0/2695539693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87ad7d0 0x5575d87b7820 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.364+0000 7f157ca20740 1 -- 172.21.15.112:0/2695539693 >> 172.21.15.112:0/2695539693 conn(0x5575d87a8dc0 msgr2=0x5575d87b6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:45.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.364+0000 7f157ca20740 1 -- 172.21.15.112:0/2695539693 shutdown_connections 2024-03-16T09:16:45.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.364+0000 7f157ca20740 1 -- 172.21.15.112:0/2695539693 wait complete. 2024-03-16T09:16:45.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.364+0000 7f157ca20740 1 -- start start 2024-03-16T09:16:45.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.364+0000 7f157ca20740 1 Processor -- start 2024-03-16T09:16:45.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.364+0000 7f157ca20740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87ad7d0 0x5575d87e5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.364+0000 7f157ca20740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ae1c0 0x5575d87e58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.365+0000 7f157ca20740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87b1320 0x5575d87e68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.365+0000 7f157ca20740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15741a5220 con 0x5575d87ae1c0 2024-03-16T09:16:45.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.365+0000 7f157ca20740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f15741a6690 con 0x5575d87b1320 2024-03-16T09:16:45.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.365+0000 7f157ca20740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f15741a58f0 con 0x5575d87ad7d0 2024-03-16T09:16:45.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.365+0000 7f157ca20740 10 client.0 _ll_register_callbacks cb 0x7f15680056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:45.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.365+0000 7f1572ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ae1c0 0x5575d87e58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.365+0000 7f1573fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87b1320 0x5575d87e68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.365+0000 7f15737fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87ad7d0 0x5575d87e5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.365+0000 7f1573fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87b1320 0x5575d87e68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:35618/0 (socket says 172.21.15.112:35618) 2024-03-16T09:16:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.365+0000 7f1573fff640 1 -- 172.21.15.112:0/2569130605 learned_addr learned my addr 172.21.15.112:0/2569130605 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.365+0000 7f1572ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ae1c0 0x5575d87e58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40780/0 (socket says 172.21.15.112:40780) 2024-03-16T09:16:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.365+0000 7f1573fff640 1 -- 172.21.15.112:0/2569130605 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87ad7d0 msgr2=0x5575d87e5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.365+0000 7f1573fff640 1 --2- 172.21.15.112:0/2569130605 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87ad7d0 0x5575d87e5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.365+0000 7f1573fff640 1 -- 172.21.15.112:0/2569130605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ae1c0 msgr2=0x5575d87e58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.365+0000 7f1573fff640 1 --2- 172.21.15.112:0/2569130605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ae1c0 0x5575d87e58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.365+0000 7f1573fff640 1 -- 172.21.15.112:0/2569130605 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575d87e6ec0 con 0x5575d87b1320 2024-03-16T09:16:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.365+0000 7f15737fe640 1 --2- 172.21.15.112:0/2569130605 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87ad7d0 0x5575d87e5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.365+0000 7f1573fff640 1 --2- 172.21.15.112:0/2569130605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87b1320 0x5575d87e68d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f156c009ca0 tx=0x7f156c003f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=947454f3f11bce5a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:45.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.365+0000 7f1552ffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-16T09:16:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.366+0000 7f1552ffd640 1 -- 172.21.15.112:0/2569130605 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c039040 con 0x5575d87b1320 2024-03-16T09:16:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.366+0000 7f1552ffd640 1 -- 172.21.15.112:0/2569130605 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f156c032d30 con 0x5575d87b1320 2024-03-16T09:16:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.366+0000 7f157ca20740 1 -- 172.21.15.112:0/2569130605 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15640049a0 con 0x5575d87b1320 2024-03-16T09:16:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.366+0000 7f1552ffd640 1 -- 172.21.15.112:0/2569130605 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f156c004040 con 0x5575d87b1320 2024-03-16T09:16:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.366+0000 7f157ca20740 10 client.25487 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.366+0000 7f157ca20740 1 -- 172.21.15.112:0/2569130605 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5575d883a500 con 0x5575d87b1320 2024-03-16T09:16:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.366+0000 7f1552ffd640 1 -- 172.21.15.112:0/2569130605 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f15640049a0 con 0x5575d87b1320 2024-03-16T09:16:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.366+0000 7f1552ffd640 1 -- 172.21.15.112:0/2569130605 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f156c035550 con 0x5575d87b1320 2024-03-16T09:16:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.366+0000 7f157ca20740 10 client.25487 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.366+0000 7f157ca20740 10 client.25487 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T09:16:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.366+0000 7f157ca20740 20 client.25487 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.366+0000 7f157ca20740 1 --2- 172.21.15.112:0/2569130605 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x5575d87d47c0 0x5575d883cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.366+0000 7f157ca20740 4 client.25487 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T09:16:45.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.366+0000 7f157ca20740 1 -- 172.21.15.112:0/2569130605 --> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575d871f9d0 con 0x5575d87d47c0 2024-03-16T09:16:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.366+0000 7f15737fe640 1 --2- 172.21.15.112:0/2569130605 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x5575d87d47c0 0x5575d883cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.367+0000 7f1552ffd640 10 client.25487 ms_handle_connect on v2:172.21.15.159:6836/4023942203 2024-03-16T09:16:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.367+0000 7f15737fe640 1 --2- 172.21.15.112:0/2569130605 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x5575d87d47c0 0x5575d883cf10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1cb05278857e73 server_cookie=6e9c601dfe64dc60 in_seq=0 out_seq=0 2024-03-16T09:16:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.367+0000 7f1552ffd640 1 -- 172.21.15.112:0/2569130605 <== mds.2 v2:172.21.15.159:6836/4023942203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575d871f9d0 con 0x5575d87d47c0 2024-03-16T09:16:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.367+0000 7f1552ffd640 10 client.25487 handle_command_reply: tid=0 2024-03-16T09:16:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.367+0000 7f157ca20740 1 client.25487 shutdown 2024-03-16T09:16:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.367+0000 7f157ca20740 1 -- 172.21.15.112:0/2569130605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87b1320 msgr2=0x5575d87e68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.367+0000 7f157ca20740 1 --2- 172.21.15.112:0/2569130605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87b1320 0x5575d87e68d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f156c009ca0 tx=0x7f156c003f20 comp rx=0 tx=0).stop 2024-03-16T09:16:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.367+0000 7f157ca20740 1 -- 172.21.15.112:0/2569130605 shutdown_connections 2024-03-16T09:16:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.367+0000 7f157ca20740 1 --2- 172.21.15.112:0/2569130605 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x5575d87d47c0 0x5575d883cf10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.367+0000 7f157ca20740 1 --2- 172.21.15.112:0/2569130605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575d87b1320 0x5575d87e68d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.367+0000 7f157ca20740 1 --2- 172.21.15.112:0/2569130605 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575d87ae1c0 0x5575d87e58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.367+0000 7f157ca20740 1 --2- 172.21.15.112:0/2569130605 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575d87ad7d0 0x5575d87e5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.367+0000 7f157ca20740 1 -- 172.21.15.112:0/2569130605 >> 172.21.15.112:0/2569130605 conn(0x5575d87a8dc0 msgr2=0x5575d87b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:45.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.367+0000 7f1552ffd640 0 client.25487 ms_handle_reset on v2:172.21.15.159:6836/4023942203 2024-03-16T09:16:45.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.367+0000 7f157ca20740 1 -- 172.21.15.112:0/2569130605 shutdown_connections 2024-03-16T09:16:45.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.367+0000 7f157ca20740 1 -- 172.21.15.112:0/2569130605 wait complete. 2024-03-16T09:16:45.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.368+0000 7f157ca20740 20 client.25487 trim_cache size 0 max 16384 2024-03-16T09:16:45.381 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-03-16T09:16:45.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.368+0000 7f1579d60640 1 -- 172.21.15.112:0/2433369490 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f155c0a2910 msgr2=0x7f155c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.368+0000 7f1579d60640 1 --2- 172.21.15.112:0/2433369490 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f155c0a2910 0x7f155c0a4dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1574101730 tx=0x7f1564005f90 comp rx=0 tx=0).stop 2024-03-16T09:16:45.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.368+0000 7f1579d60640 1 -- 172.21.15.112:0/2433369490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15741018d0 msgr2=0x7f15741980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.368+0000 7f1579d60640 1 --2- 172.21.15.112:0/2433369490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15741018d0 0x7f15741980f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f156800e9f0 tx=0x7f156800edf0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.368+0000 7f1579d60640 1 -- 172.21.15.112:0/2433369490 shutdown_connections 2024-03-16T09:16:45.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.368+0000 7f1579d60640 1 --2- 172.21.15.112:0/2433369490 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f155c0a2910 0x7f155c0a4dd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.368+0000 7f1579d60640 1 --2- 172.21.15.112:0/2433369490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1574102290 0x7f157419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.368+0000 7f1579d60640 1 --2- 172.21.15.112:0/2433369490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15741018d0 0x7f15741980f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.368+0000 7f1579d60640 1 --2- 172.21.15.112:0/2433369490 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15741006e0 0x7f1574197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.368+0000 7f1579d60640 1 -- 172.21.15.112:0/2433369490 >> 172.21.15.112:0/2433369490 conn(0x7f15740fbc70 msgr2=0x7f1574104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:45.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.368+0000 7f1579d60640 1 -- 172.21.15.112:0/2433369490 shutdown_connections 2024-03-16T09:16:45.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.368+0000 7f1579d60640 1 -- 172.21.15.112:0/2433369490 wait complete. 2024-03-16T09:16:45.384 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T09:16:45.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.515+0000 7f929e643640 1 -- 172.21.15.112:0/3658082814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92981006e0 msgr2=0x7f9298100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.515+0000 7f929e643640 1 --2- 172.21.15.112:0/3658082814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92981006e0 0x7f9298100b00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9288009900 tx=0x7f928802bc30 comp rx=0 tx=0).stop 2024-03-16T09:16:45.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.515+0000 7f929e643640 1 -- 172.21.15.112:0/3658082814 shutdown_connections 2024-03-16T09:16:45.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.515+0000 7f929e643640 1 --2- 172.21.15.112:0/3658082814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9298102290 0x7f9298108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.515+0000 7f929e643640 1 --2- 172.21.15.112:0/3658082814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92981018d0 0x7f9298101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.515+0000 7f929e643640 1 --2- 172.21.15.112:0/3658082814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92981006e0 0x7f9298100b00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.515+0000 7f929e643640 1 -- 172.21.15.112:0/3658082814 >> 172.21.15.112:0/3658082814 conn(0x7f92980fbc70 msgr2=0x7f92980fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:45.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.515+0000 7f929e643640 1 -- 172.21.15.112:0/3658082814 shutdown_connections 2024-03-16T09:16:45.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.515+0000 7f929e643640 1 -- 172.21.15.112:0/3658082814 wait complete. 2024-03-16T09:16:45.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.516+0000 7f929e643640 1 Processor -- start 2024-03-16T09:16:45.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.516+0000 7f929e643640 1 -- start start 2024-03-16T09:16:45.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.516+0000 7f929e643640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92981006e0 0x7f9298199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.516+0000 7f929e643640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92981018d0 0x7f929819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.516+0000 7f929e643640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9298102290 0x7f92981a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.516+0000 7f929e643640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f929810f740 con 0x7f92981018d0 2024-03-16T09:16:45.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.516+0000 7f929e643640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f929810f5c0 con 0x7f9298102290 2024-03-16T09:16:45.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.516+0000 7f929e643640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f929810f8c0 con 0x7f92981006e0 2024-03-16T09:16:45.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.516+0000 7f9297fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92981006e0 0x7f9298199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.516+0000 7f92977fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92981018d0 0x7f929819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.516+0000 7f9297fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92981006e0 0x7f9298199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:56562/0 (socket says 172.21.15.112:56562) 2024-03-16T09:16:45.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.516+0000 7f9297fff640 1 -- 172.21.15.112:0/2670913953 learned_addr learned my addr 172.21.15.112:0/2670913953 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:45.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.516+0000 7f929cbe0640 1 --2- 172.21.15.112:0/2670913953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9298102290 0x7f92981a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.517+0000 7f9297fff640 1 -- 172.21.15.112:0/2670913953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9298102290 msgr2=0x7f92981a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.517+0000 7f9297fff640 1 --2- 172.21.15.112:0/2670913953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9298102290 0x7f92981a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.517+0000 7f9297fff640 1 -- 172.21.15.112:0/2670913953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92981018d0 msgr2=0x7f929819a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.517+0000 7f9297fff640 1 --2- 172.21.15.112:0/2670913953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92981018d0 0x7f929819a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.517+0000 7f9297fff640 1 -- 172.21.15.112:0/2670913953 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92981a1a40 con 0x7f92981006e0 2024-03-16T09:16:45.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.517+0000 7f929cbe0640 1 --2- 172.21.15.112:0/2670913953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9298102290 0x7f92981a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:45.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.517+0000 7f92977fe640 1 --2- 172.21.15.112:0/2670913953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92981018d0 0x7f929819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:45.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.517+0000 7f9297fff640 1 --2- 172.21.15.112:0/2670913953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92981006e0 0x7f9298199e20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9288000c00 tx=0x7f9288004a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:45.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.517+0000 7f92957fa640 1 -- 172.21.15.112:0/2670913953 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9288043020 con 0x7f92981006e0 2024-03-16T09:16:45.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.518+0000 7f92957fa640 1 -- 172.21.15.112:0/2670913953 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9288004250 con 0x7f92981006e0 2024-03-16T09:16:45.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.518+0000 7f929e643640 1 -- 172.21.15.112:0/2670913953 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92981a1cd0 con 0x7f92981006e0 2024-03-16T09:16:45.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.518+0000 7f92957fa640 1 -- 172.21.15.112:0/2670913953 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9288003c00 con 0x7f92981006e0 2024-03-16T09:16:45.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.520+0000 7f929e643640 1 -- 172.21.15.112:0/2670913953 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92981a21b0 con 0x7f92981006e0 2024-03-16T09:16:45.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.520+0000 7f92957fa640 1 -- 172.21.15.112:0/2670913953 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7f9288046020 con 0x7f92981006e0 2024-03-16T09:16:45.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.521+0000 7f92957fa640 1 --2- 172.21.15.112:0/2670913953 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f92800a2910 0x7f92800a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.521+0000 7f92957fa640 1 -- 172.21.15.112:0/2670913953 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f92880f21c0 con 0x7f92981006e0 2024-03-16T09:16:45.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.522+0000 7f92977fe640 1 --2- 172.21.15.112:0/2670913953 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f92800a2910 0x7f92800a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.522+0000 7f92977fe640 1 --2- 172.21.15.112:0/2670913953 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f92800a2910 0x7f92800a4dd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f928c00b3c0 tx=0x7f928c00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:45.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.523+0000 7f92a122a740 1 Processor -- start 2024-03-16T09:16:45.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.523+0000 7f92a122a740 1 -- start start 2024-03-16T09:16:45.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.523+0000 7f92a122a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a2850 0x560c338a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.523+0000 7f92a122a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a3240 0x560c338a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.523+0000 7f92a122a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a83b0 0x560c338aa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.524+0000 7f92a122a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560c33807490 con 0x560c338a83b0 2024-03-16T09:16:45.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.524+0000 7f92a122a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560c337afc90 con 0x560c338a2850 2024-03-16T09:16:45.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.524+0000 7f92a122a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560c33803530 con 0x560c338a3240 2024-03-16T09:16:45.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.524+0000 7f929cbe0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a83b0 0x560c338aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.524+0000 7f92977fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a3240 0x560c338a7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.524+0000 7f92977fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a3240 0x560c338a7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:56578/0 (socket says 172.21.15.112:56578) 2024-03-16T09:16:45.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.524+0000 7f92977fe640 1 -- 172.21.15.112:0/2950364538 learned_addr learned my addr 172.21.15.112:0/2950364538 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:45.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.524+0000 7f9297fff640 1 --2- 172.21.15.112:0/2950364538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a2850 0x560c338a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.524+0000 7f92977fe640 1 -- 172.21.15.112:0/2950364538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a2850 msgr2=0x560c338a2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.524+0000 7f92977fe640 1 --2- 172.21.15.112:0/2950364538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a2850 0x560c338a2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.524+0000 7f92977fe640 1 -- 172.21.15.112:0/2950364538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a83b0 msgr2=0x560c338aa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.524+0000 7f92977fe640 1 --2- 172.21.15.112:0/2950364538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a83b0 0x560c338aa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.524+0000 7f92977fe640 1 -- 172.21.15.112:0/2950364538 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c3386a500 con 0x560c338a3240 2024-03-16T09:16:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.524+0000 7f92977fe640 1 --2- 172.21.15.112:0/2950364538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a3240 0x560c338a7d40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f928c018780 tx=0x7f928c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b1ca2f4fdace478 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.524+0000 7f927e7fc640 1 -- 172.21.15.112:0/2950364538 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c01e040 con 0x560c338a3240 2024-03-16T09:16:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.525+0000 7f927e7fc640 1 -- 172.21.15.112:0/2950364538 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f928c027070 con 0x560c338a3240 2024-03-16T09:16:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.525+0000 7f927e7fc640 1 -- 172.21.15.112:0/2950364538 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c022d50 con 0x560c338a3240 2024-03-16T09:16:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.525+0000 7f92a122a740 1 -- 172.21.15.112:0/2950364538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a3240 msgr2=0x560c338a7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.525+0000 7f92a122a740 1 --2- 172.21.15.112:0/2950364538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a3240 0x560c338a7d40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f928c018780 tx=0x7f928c014000 comp rx=0 tx=0).stop 2024-03-16T09:16:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.526+0000 7f92a122a740 1 -- 172.21.15.112:0/2950364538 shutdown_connections 2024-03-16T09:16:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.526+0000 7f92a122a740 1 --2- 172.21.15.112:0/2950364538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a83b0 0x560c338aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.526+0000 7f92a122a740 1 --2- 172.21.15.112:0/2950364538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a3240 0x560c338a7d40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.526+0000 7f92a122a740 1 --2- 172.21.15.112:0/2950364538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a2850 0x560c338a2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.526+0000 7f92a122a740 1 -- 172.21.15.112:0/2950364538 >> 172.21.15.112:0/2950364538 conn(0x560c3389ddc0 msgr2=0x560c3389e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.526+0000 7f92a122a740 1 -- 172.21.15.112:0/2950364538 shutdown_connections 2024-03-16T09:16:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.526+0000 7f92a122a740 1 -- 172.21.15.112:0/2950364538 wait complete. 2024-03-16T09:16:45.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.526+0000 7f92a122a740 1 -- start start 2024-03-16T09:16:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.526+0000 7f92a122a740 1 Processor -- start 2024-03-16T09:16:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.527+0000 7f92a122a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a2850 0x560c338b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.527+0000 7f92a122a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a3240 0x560c338b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.527+0000 7f92a122a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a83b0 0x560c338bcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.527+0000 7f92a122a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560c338ab360 con 0x560c338a3240 2024-03-16T09:16:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.527+0000 7f92977fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a3240 0x560c338b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.527+0000 7f9297fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a2850 0x560c338b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.527+0000 7f92977fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a3240 0x560c338b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40818/0 (socket says 172.21.15.112:40818) 2024-03-16T09:16:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.527+0000 7f92977fe640 1 -- 172.21.15.112:0/656848815 learned_addr learned my addr 172.21.15.112:0/656848815 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.527+0000 7f92a122a740 1 -- 172.21.15.112:0/656848815 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560c337c3230 con 0x560c338a83b0 2024-03-16T09:16:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.527+0000 7f929cbe0640 1 --2- 172.21.15.112:0/656848815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a83b0 0x560c338bcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.527+0000 7f92a122a740 1 -- 172.21.15.112:0/656848815 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560c338ab4e0 con 0x560c338a2850 2024-03-16T09:16:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.528+0000 7f92a122a740 10 client.0 _ll_register_callbacks cb 0x560c337ff010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.528+0000 7f9297fff640 1 -- 172.21.15.112:0/656848815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a83b0 msgr2=0x560c338bcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.528+0000 7f9297fff640 1 --2- 172.21.15.112:0/656848815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a83b0 0x560c338bcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.528+0000 7f9297fff640 1 -- 172.21.15.112:0/656848815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a3240 msgr2=0x560c338b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.528+0000 7f9297fff640 1 --2- 172.21.15.112:0/656848815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a3240 0x560c338b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.528+0000 7f9297fff640 1 -- 172.21.15.112:0/656848815 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c337d2100 con 0x560c338a2850 2024-03-16T09:16:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.528+0000 7f929cbe0640 1 --2- 172.21.15.112:0/656848815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a83b0 0x560c338bcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.528+0000 7f9297fff640 1 --2- 172.21.15.112:0/656848815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a2850 0x560c338b5560 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f92880ed890 tx=0x7f928802c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcfee63cd18337cf server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.528+0000 7f92777fe640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T09:16:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.528+0000 7f92a122a740 1 -- 172.21.15.112:0/656848815 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f928c027070 con 0x560c338a2850 2024-03-16T09:16:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.528+0000 7f92777fe640 1 -- 172.21.15.112:0/656848815 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92880d5070 con 0x560c338a2850 2024-03-16T09:16:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.528+0000 7f92777fe640 1 -- 172.21.15.112:0/656848815 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f92880ed9d0 con 0x560c338a2850 2024-03-16T09:16:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.528+0000 7f92777fe640 1 -- 172.21.15.112:0/656848815 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f92880f4060 con 0x560c338a2850 2024-03-16T09:16:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.528+0000 7f92a122a740 10 client.25507 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.529+0000 7f92a122a740 1 -- 172.21.15.112:0/656848815 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({fsmap=22}) -- 0x560c3351b3f0 con 0x560c338a2850 2024-03-16T09:16:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.529+0000 7f92777fe640 1 -- 172.21.15.112:0/656848815 <== mon.1 v2:172.21.15.194:3300/0 4 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f92880c4a30 con 0x560c338a2850 2024-03-16T09:16:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.529+0000 7f92a122a740 10 client.25507 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.529+0000 7f92a122a740 10 client.25507 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T09:16:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.529+0000 7f92a122a740 20 client.25507 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.529+0000 7f92a122a740 1 --2- 172.21.15.112:0/656848815 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x560c338c1120 0x560c338c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.529+0000 7f92a122a740 4 client.25507 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:16:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.529+0000 7f92a122a740 1 -- 172.21.15.112:0/656848815 --> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c338149d0 con 0x560c338c1120 2024-03-16T09:16:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.529+0000 7f92977fe640 1 --2- 172.21.15.112:0/656848815 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x560c338c1120 0x560c338c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.530+0000 7f92977fe640 1 --2- 172.21.15.112:0/656848815 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x560c338c1120 0x560c338c3570 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=32f0813fcba9d40d server_cookie=529e4d926375bf31 in_seq=0 out_seq=0 2024-03-16T09:16:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.530+0000 7f92777fe640 10 client.25507 ms_handle_connect on v2:172.21.15.194:6838/2528943309 2024-03-16T09:16:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.530+0000 7f92777fe640 1 -- 172.21.15.112:0/656848815 <== mds.1 v2:172.21.15.194:6838/2528943309 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560c338149d0 con 0x560c338c1120 2024-03-16T09:16:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.530+0000 7f92777fe640 10 client.25507 handle_command_reply: tid=0 2024-03-16T09:16:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.530+0000 7f92a122a740 1 client.25507 shutdown 2024-03-16T09:16:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.531+0000 7f92a122a740 1 -- 172.21.15.112:0/656848815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a2850 msgr2=0x560c338b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.531+0000 7f92a122a740 1 --2- 172.21.15.112:0/656848815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a2850 0x560c338b5560 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f92880ed890 tx=0x7f928802c9f0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.531+0000 7f92a122a740 1 -- 172.21.15.112:0/656848815 shutdown_connections 2024-03-16T09:16:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.531+0000 7f92a122a740 1 --2- 172.21.15.112:0/656848815 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x560c338c1120 0x560c338c3570 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.531+0000 7f92a122a740 1 --2- 172.21.15.112:0/656848815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a83b0 0x560c338bcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.531+0000 7f92777fe640 0 client.25507 ms_handle_reset on v2:172.21.15.194:6838/2528943309 2024-03-16T09:16:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.531+0000 7f92a122a740 1 --2- 172.21.15.112:0/656848815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a3240 0x560c338b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.531+0000 7f92a122a740 1 --2- 172.21.15.112:0/656848815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a2850 0x560c338b5560 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.531+0000 7f92a122a740 1 -- 172.21.15.112:0/656848815 >> 172.21.15.112:0/656848815 conn(0x560c3389ddc0 msgr2=0x560c3389cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.531+0000 7f92a122a740 1 -- 172.21.15.112:0/656848815 shutdown_connections 2024-03-16T09:16:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.531+0000 7f92a122a740 1 -- 172.21.15.112:0/656848815 wait complete. 2024-03-16T09:16:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.531+0000 7f92a122a740 20 client.25507 trim_cache size 0 max 16384 2024-03-16T09:16:45.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.557+0000 7f92a122a740 1 Processor -- start 2024-03-16T09:16:45.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.557+0000 7f92a122a740 1 -- start start 2024-03-16T09:16:45.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.557+0000 7f92a122a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a2850 0x560c338a3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.557+0000 7f92a122a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a3240 0x560c3389e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.557+0000 7f92a122a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a83b0 0x560c3389c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.557+0000 7f92a122a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f92981a9450 con 0x560c338a2850 2024-03-16T09:16:45.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.557+0000 7f92a122a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f92981a6d20 con 0x560c338a3240 2024-03-16T09:16:45.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.557+0000 7f92a122a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f92981a4b70 con 0x560c338a83b0 2024-03-16T09:16:45.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.557+0000 7f92977fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a3240 0x560c3389e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.557+0000 7f9297fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a2850 0x560c338a3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.557+0000 7f92977fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a3240 0x560c3389e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:35676/0 (socket says 172.21.15.112:35676) 2024-03-16T09:16:45.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.557+0000 7f929cbe0640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a83b0 0x560c3389c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.557+0000 7f92977fe640 1 -- 172.21.15.112:0/3727807111 learned_addr learned my addr 172.21.15.112:0/3727807111 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:45.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.558+0000 7f92977fe640 1 -- 172.21.15.112:0/3727807111 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a83b0 msgr2=0x560c3389c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.558+0000 7f92977fe640 1 --2- 172.21.15.112:0/3727807111 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a83b0 0x560c3389c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.558+0000 7f92977fe640 1 -- 172.21.15.112:0/3727807111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a2850 msgr2=0x560c338a3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.558+0000 7f92977fe640 1 --2- 172.21.15.112:0/3727807111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a2850 0x560c338a3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.558+0000 7f92977fe640 1 -- 172.21.15.112:0/3727807111 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c335e57d0 con 0x560c338a3240 2024-03-16T09:16:45.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.558+0000 7f92977fe640 1 --2- 172.21.15.112:0/3727807111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a3240 0x560c3389e1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f928c0058c0 tx=0x7f928c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8a3ab5f400484c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:45.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.558+0000 7f927dffb640 1 -- 172.21.15.112:0/3727807111 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c03b070 con 0x560c338a3240 2024-03-16T09:16:45.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.558+0000 7f927dffb640 1 -- 172.21.15.112:0/3727807111 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f928c036cb0 con 0x560c338a3240 2024-03-16T09:16:45.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.558+0000 7f927dffb640 1 -- 172.21.15.112:0/3727807111 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c034070 con 0x560c338a3240 2024-03-16T09:16:45.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.559+0000 7f92a122a740 1 -- 172.21.15.112:0/3727807111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a3240 msgr2=0x560c3389e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.559+0000 7f92a122a740 1 --2- 172.21.15.112:0/3727807111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a3240 0x560c3389e1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f928c0058c0 tx=0x7f928c005850 comp rx=0 tx=0).stop 2024-03-16T09:16:45.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.559+0000 7f92a122a740 1 -- 172.21.15.112:0/3727807111 shutdown_connections 2024-03-16T09:16:45.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.559+0000 7f92a122a740 1 --2- 172.21.15.112:0/3727807111 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a83b0 0x560c3389c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.559+0000 7f92a122a740 1 --2- 172.21.15.112:0/3727807111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a3240 0x560c3389e1a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.559+0000 7f92a122a740 1 --2- 172.21.15.112:0/3727807111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a2850 0x560c338a3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.559+0000 7f92a122a740 1 -- 172.21.15.112:0/3727807111 >> 172.21.15.112:0/3727807111 conn(0x560c3389ddc0 msgr2=0x560c338b4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.559+0000 7f92a122a740 1 -- 172.21.15.112:0/3727807111 shutdown_connections 2024-03-16T09:16:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.559+0000 7f92a122a740 1 -- 172.21.15.112:0/3727807111 wait complete. 2024-03-16T09:16:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.560+0000 7f92a122a740 1 -- start start 2024-03-16T09:16:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.560+0000 7f92a122a740 1 Processor -- start 2024-03-16T09:16:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.560+0000 7f92a122a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a2850 0x560c338b99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.560+0000 7f92a122a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a3240 0x560c338b9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.560+0000 7f9297fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a2850 0x560c338b99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.560+0000 7f9297fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a2850 0x560c338b99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40832/0 (socket says 172.21.15.112:40832) 2024-03-16T09:16:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.560+0000 7f9297fff640 1 -- 172.21.15.112:0/1833975003 learned_addr learned my addr 172.21.15.112:0/1833975003 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.560+0000 7f92a122a740 1 --2- 172.21.15.112:0/1833975003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a83b0 0x560c338baf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.561+0000 7f92a122a740 1 -- 172.21.15.112:0/1833975003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f92981a51e0 con 0x560c338a2850 2024-03-16T09:16:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.561+0000 7f92a122a740 1 -- 172.21.15.112:0/1833975003 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f92981a6650 con 0x560c338a3240 2024-03-16T09:16:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.561+0000 7f92a122a740 1 -- 172.21.15.112:0/1833975003 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f92981a58b0 con 0x560c338a83b0 2024-03-16T09:16:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.561+0000 7f92977fe640 1 --2- 172.21.15.112:0/1833975003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a3240 0x560c338b9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.561+0000 7f92a122a740 10 client.0 _ll_register_callbacks cb 0x7f92880035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.561+0000 7f929cbe0640 1 --2- 172.21.15.112:0/1833975003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a83b0 0x560c338baf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.561+0000 7f9297fff640 1 -- 172.21.15.112:0/1833975003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a83b0 msgr2=0x560c338baf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.561+0000 7f9297fff640 1 --2- 172.21.15.112:0/1833975003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a83b0 0x560c338baf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.561+0000 7f9297fff640 1 -- 172.21.15.112:0/1833975003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a3240 msgr2=0x560c338b9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.561+0000 7f9297fff640 1 --2- 172.21.15.112:0/1833975003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a3240 0x560c338b9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.561+0000 7f9297fff640 1 -- 172.21.15.112:0/1833975003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c338bb4f0 con 0x560c338a2850 2024-03-16T09:16:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.561+0000 7f929cbe0640 1 --2- 172.21.15.112:0/1833975003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a83b0 0x560c338baf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:16:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.561+0000 7f9297fff640 1 --2- 172.21.15.112:0/1833975003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a2850 0x560c338b99b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9288005c20 tx=0x7f92880f2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07c7e93f8c5b920 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.561+0000 7f92777fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:16:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.561+0000 7f92777fe640 1 -- 172.21.15.112:0/1833975003 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92880d5040 con 0x560c338a2850 2024-03-16T09:16:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.561+0000 7f92777fe640 1 -- 172.21.15.112:0/1833975003 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f92880bbb70 con 0x560c338a2850 2024-03-16T09:16:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.561+0000 7f92a122a740 1 -- 172.21.15.112:0/1833975003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f928c036cb0 con 0x560c338a2850 2024-03-16T09:16:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.562+0000 7f92777fe640 1 -- 172.21.15.112:0/1833975003 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f92880c4ec0 con 0x560c338a2850 2024-03-16T09:16:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.562+0000 7f92a122a740 10 client.15654 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.562+0000 7f92a122a740 1 -- 172.21.15.112:0/1833975003 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560c338bc840 con 0x560c338a2850 2024-03-16T09:16:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.562+0000 7f92777fe640 1 -- 172.21.15.112:0/1833975003 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f928c027070 con 0x560c338a2850 2024-03-16T09:16:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.563+0000 7f92777fe640 1 -- 172.21.15.112:0/1833975003 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f92880bb5e0 con 0x560c338a2850 2024-03-16T09:16:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.563+0000 7f92a122a740 10 client.15654 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.563+0000 7f92a122a740 10 client.15654 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T09:16:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.563+0000 7f92a122a740 20 client.15654 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.563+0000 7f92a122a740 1 --2- 172.21.15.112:0/1833975003 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x560c339277c0 0x560c33929be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.563+0000 7f92a122a740 4 client.15654 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T09:16:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.563+0000 7f92a122a740 1 -- 172.21.15.112:0/1833975003 --> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f928c036cb0 con 0x560c339277c0 2024-03-16T09:16:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.563+0000 7f92977fe640 1 --2- 172.21.15.112:0/1833975003 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x560c339277c0 0x560c33929be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.563+0000 7f92777fe640 10 client.15654 ms_handle_connect on v2:172.21.15.194:6838/2528943309 2024-03-16T09:16:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.563+0000 7f92977fe640 1 --2- 172.21.15.112:0/1833975003 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x560c339277c0 0x560c33929be0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0179a341abe848f server_cookie=47e450f19d6c4aa6 in_seq=0 out_seq=0 2024-03-16T09:16:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.563+0000 7f92777fe640 1 -- 172.21.15.112:0/1833975003 <== mds.1 v2:172.21.15.194:6838/2528943309 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f928c036cb0 con 0x560c339277c0 2024-03-16T09:16:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.563+0000 7f92777fe640 10 client.15654 handle_command_reply: tid=0 2024-03-16T09:16:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.563+0000 7f92a122a740 1 client.15654 shutdown 2024-03-16T09:16:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.564+0000 7f92a122a740 1 -- 172.21.15.112:0/1833975003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a2850 msgr2=0x560c338b99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.564+0000 7f92a122a740 1 --2- 172.21.15.112:0/1833975003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a2850 0x560c338b99b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9288005c20 tx=0x7f92880f2000 comp rx=0 tx=0).stop 2024-03-16T09:16:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.564+0000 7f92a122a740 1 -- 172.21.15.112:0/1833975003 shutdown_connections 2024-03-16T09:16:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.564+0000 7f92a122a740 1 --2- 172.21.15.112:0/1833975003 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x560c339277c0 0x560c33929be0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.564+0000 7f92a122a740 1 --2- 172.21.15.112:0/1833975003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560c338a83b0 0x560c338baf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.564+0000 7f92a122a740 1 --2- 172.21.15.112:0/1833975003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c338a3240 0x560c338b9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.564+0000 7f92a122a740 1 --2- 172.21.15.112:0/1833975003 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560c338a2850 0x560c338b99b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.564+0000 7f92a122a740 1 -- 172.21.15.112:0/1833975003 >> 172.21.15.112:0/1833975003 conn(0x560c3389ddc0 msgr2=0x560c338a3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.564+0000 7f92777fe640 0 client.15654 ms_handle_reset on v2:172.21.15.194:6838/2528943309 2024-03-16T09:16:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.564+0000 7f92a122a740 1 -- 172.21.15.112:0/1833975003 shutdown_connections 2024-03-16T09:16:45.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.564+0000 7f92a122a740 1 -- 172.21.15.112:0/1833975003 wait complete. 2024-03-16T09:16:45.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.564+0000 7f92a122a740 20 client.15654 trim_cache size 0 max 16384 2024-03-16T09:16:45.571 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-03-16T09:16:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.565+0000 7f929e643640 1 -- 172.21.15.112:0/2670913953 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f92800a2910 msgr2=0x7f92800a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.565+0000 7f929e643640 1 --2- 172.21.15.112:0/2670913953 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f92800a2910 0x7f92800a4dd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f928c00b3c0 tx=0x7f928c00d040 comp rx=0 tx=0).stop 2024-03-16T09:16:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.565+0000 7f929e643640 1 -- 172.21.15.112:0/2670913953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92981006e0 msgr2=0x7f9298199e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.565+0000 7f929e643640 1 --2- 172.21.15.112:0/2670913953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92981006e0 0x7f9298199e20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9288000c00 tx=0x7f9288004a30 comp rx=0 tx=0).stop 2024-03-16T09:16:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.565+0000 7f929e643640 1 -- 172.21.15.112:0/2670913953 shutdown_connections 2024-03-16T09:16:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.565+0000 7f929e643640 1 --2- 172.21.15.112:0/2670913953 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f92800a2910 0x7f92800a4dd0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.565+0000 7f929e643640 1 --2- 172.21.15.112:0/2670913953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9298102290 0x7f92981a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.565+0000 7f929e643640 1 --2- 172.21.15.112:0/2670913953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92981018d0 0x7f929819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.565+0000 7f929e643640 1 --2- 172.21.15.112:0/2670913953 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92981006e0 0x7f9298199e20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.565+0000 7f929e643640 1 -- 172.21.15.112:0/2670913953 >> 172.21.15.112:0/2670913953 conn(0x7f92980fbc70 msgr2=0x7f9298104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:45.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.565+0000 7f929e643640 1 -- 172.21.15.112:0/2670913953 shutdown_connections 2024-03-16T09:16:45.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.565+0000 7f929e643640 1 -- 172.21.15.112:0/2670913953 wait complete. 2024-03-16T09:16:45.582 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T09:16:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.743+0000 7f802a675640 1 -- 172.21.15.112:0/1900038246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8024102290 msgr2=0x7f8024108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.743+0000 7f802a675640 1 --2- 172.21.15.112:0/1900038246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8024102290 0x7f8024108b20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f801c009ca0 tx=0x7f801c02c0f0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.743+0000 7f802a675640 1 -- 172.21.15.112:0/1900038246 shutdown_connections 2024-03-16T09:16:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.743+0000 7f802a675640 1 --2- 172.21.15.112:0/1900038246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8024102290 0x7f8024108b20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.743+0000 7f802a675640 1 --2- 172.21.15.112:0/1900038246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80241018d0 0x7f8024101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.743+0000 7f802a675640 1 --2- 172.21.15.112:0/1900038246 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f80241006e0 0x7f8024100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.743+0000 7f802a675640 1 -- 172.21.15.112:0/1900038246 >> 172.21.15.112:0/1900038246 conn(0x7f80240fbc70 msgr2=0x7f80240fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:45.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.744+0000 7f802a675640 1 -- 172.21.15.112:0/1900038246 shutdown_connections 2024-03-16T09:16:45.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.744+0000 7f802a675640 1 -- 172.21.15.112:0/1900038246 wait complete. 2024-03-16T09:16:45.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.744+0000 7f802a675640 1 Processor -- start 2024-03-16T09:16:45.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.744+0000 7f802a675640 1 -- start start 2024-03-16T09:16:45.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.745+0000 7f802a675640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80241006e0 0x7f8024197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.745+0000 7f802a675640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80241018d0 0x7f8024198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.745+0000 7f802a675640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8024102290 0x7f802419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.745+0000 7f802a675640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f802410d5a0 con 0x7f80241006e0 2024-03-16T09:16:45.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.745+0000 7f802a675640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f802410d420 con 0x7f80241018d0 2024-03-16T09:16:45.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.745+0000 7f802a675640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f802410d720 con 0x7f8024102290 2024-03-16T09:16:45.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.745+0000 7f80237fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80241018d0 0x7f8024198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.745+0000 7f8023fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80241006e0 0x7f8024197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.745+0000 7f80237fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80241018d0 0x7f8024198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:35708/0 (socket says 172.21.15.112:35708) 2024-03-16T09:16:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.745+0000 7f80237fe640 1 -- 172.21.15.112:0/3163761591 learned_addr learned my addr 172.21.15.112:0/3163761591 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.745+0000 7f8028c12640 1 --2- 172.21.15.112:0/3163761591 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8024102290 0x7f802419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.745+0000 7f80237fe640 1 -- 172.21.15.112:0/3163761591 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8024102290 msgr2=0x7f802419f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.745+0000 7f80237fe640 1 --2- 172.21.15.112:0/3163761591 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8024102290 0x7f802419f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.745+0000 7f80237fe640 1 -- 172.21.15.112:0/3163761591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80241006e0 msgr2=0x7f8024197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.745+0000 7f80237fe640 1 --2- 172.21.15.112:0/3163761591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80241006e0 0x7f8024197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.745+0000 7f80237fe640 1 -- 172.21.15.112:0/3163761591 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802419f7e0 con 0x7f80241018d0 2024-03-16T09:16:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.746+0000 7f80237fe640 1 --2- 172.21.15.112:0/3163761591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80241018d0 0x7f8024198100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f801800ea60 tx=0x7f801800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.746+0000 7f80217fa640 1 -- 172.21.15.112:0/3163761591 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801800cde0 con 0x7f80241018d0 2024-03-16T09:16:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.746+0000 7f80217fa640 1 -- 172.21.15.112:0/3163761591 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f80180035f0 con 0x7f80241018d0 2024-03-16T09:16:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.746+0000 7f802a675640 1 -- 172.21.15.112:0/3163761591 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802419fad0 con 0x7f80241018d0 2024-03-16T09:16:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.746+0000 7f80217fa640 1 -- 172.21.15.112:0/3163761591 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018010720 con 0x7f80241018d0 2024-03-16T09:16:45.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.746+0000 7f802a675640 1 -- 172.21.15.112:0/3163761591 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80241a0070 con 0x7f80241018d0 2024-03-16T09:16:45.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.749+0000 7f802d25b740 1 Processor -- start 2024-03-16T09:16:45.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.749+0000 7f802d25b740 1 -- start start 2024-03-16T09:16:45.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.749+0000 7f802d25b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa4c60 0x5575edfa5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.749+0000 7f802d25b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfa5650 0x5575edfaa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.749+0000 7f802d25b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfaa7c0 0x5575edfacc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.749+0000 7f802d25b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5575edf05490 con 0x5575edfaa7c0 2024-03-16T09:16:45.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.749+0000 7f802d25b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5575edeadc90 con 0x5575edfa5650 2024-03-16T09:16:45.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.749+0000 7f802d25b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5575edf01530 con 0x5575edfa4c60 2024-03-16T09:16:45.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.750+0000 7f80217fa640 1 -- 172.21.15.112:0/3163761591 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7f8018004020 con 0x7f80241018d0 2024-03-16T09:16:45.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.750+0000 7f8028c12640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa4c60 0x5575edfa5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.750+0000 7f8023fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfa5650 0x5575edfaa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.750+0000 7f8028c12640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa4c60 0x5575edfa5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:56642/0 (socket says 172.21.15.112:56642) 2024-03-16T09:16:45.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.750+0000 7f8023fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfa5650 0x5575edfaa150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:35714/0 (socket says 172.21.15.112:35714) 2024-03-16T09:16:45.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.750+0000 7f8028c12640 1 -- 172.21.15.112:0/3235001913 learned_addr learned my addr 172.21.15.112:0/3235001913 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:45.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.750+0000 7f80237fe640 1 --2- 172.21.15.112:0/3235001913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfaa7c0 0x5575edfacc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.750+0000 7f8028c12640 1 -- 172.21.15.112:0/3235001913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfa5650 msgr2=0x5575edfaa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.750+0000 7f8028c12640 1 --2- 172.21.15.112:0/3235001913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfa5650 0x5575edfaa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.750+0000 7f8028c12640 1 -- 172.21.15.112:0/3235001913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfaa7c0 msgr2=0x5575edfacc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.750+0000 7f8028c12640 1 --2- 172.21.15.112:0/3235001913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfaa7c0 0x5575edfacc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.750+0000 7f8028c12640 1 -- 172.21.15.112:0/3235001913 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575edf68500 con 0x5575edfa4c60 2024-03-16T09:16:45.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.750+0000 7f80237fe640 1 --2- 172.21.15.112:0/3235001913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfaa7c0 0x5575edfacc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:16:45.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.750+0000 7f8028c12640 1 --2- 172.21.15.112:0/3235001913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa4c60 0x5575edfa5080 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f801c02c700 tx=0x7f801c005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74e9f97006482a8b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:45.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.750+0000 7f80217fa640 1 --2- 172.21.15.112:0/3163761591 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f800c0a2910 0x7f800c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.751+0000 7f800a7fc640 1 -- 172.21.15.112:0/3235001913 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c043070 con 0x5575edfa4c60 2024-03-16T09:16:45.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.751+0000 7f800a7fc640 1 -- 172.21.15.112:0/3235001913 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f801c032d30 con 0x5575edfa4c60 2024-03-16T09:16:45.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.751+0000 7f800a7fc640 1 -- 172.21.15.112:0/3235001913 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c039040 con 0x5575edfa4c60 2024-03-16T09:16:45.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.751+0000 7f8023fff640 1 --2- 172.21.15.112:0/3163761591 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f800c0a2910 0x7f800c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.751+0000 7f80217fa640 1 -- 172.21.15.112:0/3163761591 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f80180ceec0 con 0x7f80241018d0 2024-03-16T09:16:45.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.752+0000 7f8023fff640 1 --2- 172.21.15.112:0/3163761591 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f800c0a2910 0x7f800c0a4dd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x5575edfa5a50 tx=0x7f8014005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:45.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.752+0000 7f802d25b740 1 -- 172.21.15.112:0/3235001913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa4c60 msgr2=0x5575edfa5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.752+0000 7f802d25b740 1 --2- 172.21.15.112:0/3235001913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa4c60 0x5575edfa5080 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f801c02c700 tx=0x7f801c005f80 comp rx=0 tx=0).stop 2024-03-16T09:16:45.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.752+0000 7f802d25b740 1 -- 172.21.15.112:0/3235001913 shutdown_connections 2024-03-16T09:16:45.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.752+0000 7f802d25b740 1 --2- 172.21.15.112:0/3235001913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfaa7c0 0x5575edfacc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.752+0000 7f802d25b740 1 --2- 172.21.15.112:0/3235001913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfa5650 0x5575edfaa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.752+0000 7f802d25b740 1 --2- 172.21.15.112:0/3235001913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa4c60 0x5575edfa5080 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.752+0000 7f802d25b740 1 -- 172.21.15.112:0/3235001913 >> 172.21.15.112:0/3235001913 conn(0x5575edf9bdc0 msgr2=0x5575edf9c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:45.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.752+0000 7f802d25b740 1 -- 172.21.15.112:0/3235001913 shutdown_connections 2024-03-16T09:16:45.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.752+0000 7f802d25b740 1 -- 172.21.15.112:0/3235001913 wait complete. 2024-03-16T09:16:45.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.752+0000 7f802d25b740 1 -- start start 2024-03-16T09:16:45.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.753+0000 7f802d25b740 1 Processor -- start 2024-03-16T09:16:45.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.753+0000 7f802d25b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfa4c60 0x5575edfaf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.753+0000 7f802d25b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa5650 0x5575edfaf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.753+0000 7f802d25b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfaa7c0 0x5575edfb6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.753+0000 7f802d25b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5575edfad710 con 0x5575edfaa7c0 2024-03-16T09:16:45.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.753+0000 7f802d25b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5575edec1230 con 0x5575edfa4c60 2024-03-16T09:16:45.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.753+0000 7f802d25b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5575edfad890 con 0x5575edfa5650 2024-03-16T09:16:45.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.753+0000 7f802d25b740 10 client.0 _ll_register_callbacks cb 0x5575edefd010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:45.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.753+0000 7f8028c12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfaa7c0 0x5575edfb6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.753+0000 7f8023fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfa4c60 0x5575edfaf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.753+0000 7f8028c12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfaa7c0 0x5575edfb6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40858/0 (socket says 172.21.15.112:40858) 2024-03-16T09:16:45.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.753+0000 7f8028c12640 1 -- 172.21.15.112:0/3236468072 learned_addr learned my addr 172.21.15.112:0/3236468072 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:45.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.753+0000 7f80237fe640 1 --2- 172.21.15.112:0/3236468072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa5650 0x5575edfaf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.754+0000 7f80237fe640 1 -- 172.21.15.112:0/3236468072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfa4c60 msgr2=0x5575edfaf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.754+0000 7f80237fe640 1 --2- 172.21.15.112:0/3236468072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfa4c60 0x5575edfaf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.754+0000 7f80237fe640 1 -- 172.21.15.112:0/3236468072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfaa7c0 msgr2=0x5575edfb6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.754+0000 7f80237fe640 1 --2- 172.21.15.112:0/3236468072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfaa7c0 0x5575edfb6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.754+0000 7f80237fe640 1 -- 172.21.15.112:0/3236468072 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575eded0100 con 0x5575edfa5650 2024-03-16T09:16:45.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.754+0000 7f8023fff640 1 --2- 172.21.15.112:0/3236468072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfa4c60 0x5575edfaf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:45.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.754+0000 7f8028c12640 1 --2- 172.21.15.112:0/3236468072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfaa7c0 0x5575edfb6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:45.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.754+0000 7f80237fe640 1 --2- 172.21.15.112:0/3236468072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa5650 0x5575edfaf710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f80180cc880 tx=0x7f801809b4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fb73123491bbdb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:45.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.754+0000 7f80037fe640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T09:16:45.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.754+0000 7f80037fe640 1 -- 172.21.15.112:0/3236468072 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018003bc0 con 0x5575edfa5650 2024-03-16T09:16:45.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.754+0000 7f80037fe640 1 -- 172.21.15.112:0/3236468072 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f80180ccad0 con 0x5575edfa5650 2024-03-16T09:16:45.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.754+0000 7f802d25b740 1 -- 172.21.15.112:0/3236468072 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f801c032d30 con 0x5575edfa5650 2024-03-16T09:16:45.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.754+0000 7f80037fe640 1 -- 172.21.15.112:0/3236468072 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f801809be10 con 0x5575edfa5650 2024-03-16T09:16:45.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.754+0000 7f802d25b740 10 client.25525 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:45.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.754+0000 7f802d25b740 1 -- 172.21.15.112:0/3236468072 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5575edc193f0 con 0x5575edfa5650 2024-03-16T09:16:45.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.754+0000 7f80037fe640 1 -- 172.21.15.112:0/3236468072 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f801c032d30 con 0x5575edfa5650 2024-03-16T09:16:45.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.755+0000 7f80037fe640 1 -- 172.21.15.112:0/3236468072 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f80180a3a20 con 0x5575edfa5650 2024-03-16T09:16:45.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.755+0000 7f802d25b740 10 client.25525 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:45.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.755+0000 7f802d25b740 10 client.25525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:45.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.755+0000 7f802d25b740 20 client.25525 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:45.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.755+0000 7f802d25b740 1 --2- 172.21.15.112:0/3236468072 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5575edfbafc0 0x5575edfbd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.755+0000 7f802d25b740 4 client.25525 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:16:45.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.755+0000 7f802d25b740 1 -- 172.21.15.112:0/3236468072 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575edf129d0 con 0x5575edfbafc0 2024-03-16T09:16:45.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.755+0000 7f8023fff640 1 --2- 172.21.15.112:0/3236468072 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5575edfbafc0 0x5575edfbd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.755+0000 7f8023fff640 1 --2- 172.21.15.112:0/3236468072 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5575edfbafc0 0x5575edfbd400 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54aa24c309ffc6cf server_cookie=bfa2664c7f4d895f in_seq=0 out_seq=0 2024-03-16T09:16:45.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.755+0000 7f80037fe640 10 client.25525 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:45.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.756+0000 7f80037fe640 1 -- 172.21.15.112:0/3236468072 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5575edf129d0 con 0x5575edfbafc0 2024-03-16T09:16:45.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.756+0000 7f80037fe640 10 client.25525 handle_command_reply: tid=0 2024-03-16T09:16:45.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.756+0000 7f802d25b740 1 client.25525 shutdown 2024-03-16T09:16:45.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.756+0000 7f802d25b740 1 -- 172.21.15.112:0/3236468072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa5650 msgr2=0x5575edfaf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.756+0000 7f802d25b740 1 --2- 172.21.15.112:0/3236468072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa5650 0x5575edfaf710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f80180cc880 tx=0x7f801809b4e0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.756+0000 7f802d25b740 1 -- 172.21.15.112:0/3236468072 shutdown_connections 2024-03-16T09:16:45.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.756+0000 7f802d25b740 1 --2- 172.21.15.112:0/3236468072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfaa7c0 0x5575edfb6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.756+0000 7f802d25b740 1 --2- 172.21.15.112:0/3236468072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa5650 0x5575edfaf710 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.756+0000 7f802d25b740 1 --2- 172.21.15.112:0/3236468072 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5575edfbafc0 0x5575edfbd400 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.756+0000 7f802d25b740 1 --2- 172.21.15.112:0/3236468072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfa4c60 0x5575edfaf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.756+0000 7f80037fe640 0 client.25525 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:45.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.756+0000 7f802d25b740 1 -- 172.21.15.112:0/3236468072 >> 172.21.15.112:0/3236468072 conn(0x5575edf9bdc0 msgr2=0x5575edf9afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:45.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.756+0000 7f802d25b740 1 -- 172.21.15.112:0/3236468072 shutdown_connections 2024-03-16T09:16:45.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.756+0000 7f802d25b740 1 -- 172.21.15.112:0/3236468072 wait complete. 2024-03-16T09:16:45.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.756+0000 7f802d25b740 20 client.25525 trim_cache size 0 max 16384 2024-03-16T09:16:45.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.782+0000 7f802d25b740 1 Processor -- start 2024-03-16T09:16:45.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.782+0000 7f802d25b740 1 -- start start 2024-03-16T09:16:45.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.782+0000 7f802d25b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa4c60 0x5575edf9a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.782+0000 7f802d25b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfa5650 0x5575edfb6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.783+0000 7f80237fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfa5650 0x5575edfb6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.783+0000 7f802d25b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfaa7c0 0x5575edf9ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.783+0000 7f8023fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa4c60 0x5575edf9a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.783+0000 7f80237fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfa5650 0x5575edfb6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40868/0 (socket says 172.21.15.112:40868) 2024-03-16T09:16:45.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.783+0000 7f8023fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa4c60 0x5575edf9a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:56664/0 (socket says 172.21.15.112:56664) 2024-03-16T09:16:45.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.783+0000 7f802d25b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80241a94e0 con 0x5575edfa5650 2024-03-16T09:16:45.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.783+0000 7f802d25b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f80241a6db0 con 0x5575edfaa7c0 2024-03-16T09:16:45.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.783+0000 7f802d25b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f80241a4c00 con 0x5575edfa4c60 2024-03-16T09:16:45.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.783+0000 7f80237fe640 1 -- 172.21.15.112:0/1357228164 learned_addr learned my addr 172.21.15.112:0/1357228164 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:45.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.783+0000 7f8028c12640 1 --2- 172.21.15.112:0/1357228164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfaa7c0 0x5575edf9ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.783+0000 7f8023fff640 1 -- 172.21.15.112:0/1357228164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfaa7c0 msgr2=0x5575edf9ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.783+0000 7f8023fff640 1 --2- 172.21.15.112:0/1357228164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfaa7c0 0x5575edf9ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.783+0000 7f8023fff640 1 -- 172.21.15.112:0/1357228164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfa5650 msgr2=0x5575edfb6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.783+0000 7f8023fff640 1 --2- 172.21.15.112:0/1357228164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfa5650 0x5575edfb6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.783+0000 7f8023fff640 1 -- 172.21.15.112:0/1357228164 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575edce37d0 con 0x5575edfa4c60 2024-03-16T09:16:45.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.783+0000 7f80237fe640 1 --2- 172.21.15.112:0/1357228164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfa5650 0x5575edfb6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:45.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.783+0000 7f8023fff640 1 --2- 172.21.15.112:0/1357228164 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa4c60 0x5575edf9a750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x5575edfbdc70 tx=0x7f801401b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3643c9cc600f3a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:45.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.783+0000 7f8009ffb640 1 -- 172.21.15.112:0/1357228164 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014033030 con 0x5575edfa4c60 2024-03-16T09:16:45.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.784+0000 7f8009ffb640 1 -- 172.21.15.112:0/1357228164 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f801402e770 con 0x5575edfa4c60 2024-03-16T09:16:45.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.784+0000 7f8009ffb640 1 -- 172.21.15.112:0/1357228164 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801402ccd0 con 0x5575edfa4c60 2024-03-16T09:16:45.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.785+0000 7f802d25b740 1 -- 172.21.15.112:0/1357228164 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa4c60 msgr2=0x5575edf9a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.785+0000 7f802d25b740 1 --2- 172.21.15.112:0/1357228164 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa4c60 0x5575edf9a750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x5575edfbdc70 tx=0x7f801401b000 comp rx=0 tx=0).stop 2024-03-16T09:16:45.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.785+0000 7f802d25b740 1 -- 172.21.15.112:0/1357228164 shutdown_connections 2024-03-16T09:16:45.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.785+0000 7f802d25b740 1 --2- 172.21.15.112:0/1357228164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfaa7c0 0x5575edf9ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.785+0000 7f802d25b740 1 --2- 172.21.15.112:0/1357228164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfa5650 0x5575edfb6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.785+0000 7f802d25b740 1 --2- 172.21.15.112:0/1357228164 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa4c60 0x5575edf9a750 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.785+0000 7f802d25b740 1 -- 172.21.15.112:0/1357228164 >> 172.21.15.112:0/1357228164 conn(0x5575edf9bdc0 msgr2=0x5575edfa6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:45.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.785+0000 7f802d25b740 1 -- 172.21.15.112:0/1357228164 shutdown_connections 2024-03-16T09:16:45.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.785+0000 7f802d25b740 1 -- 172.21.15.112:0/1357228164 wait complete. 2024-03-16T09:16:45.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.785+0000 7f802d25b740 1 -- start start 2024-03-16T09:16:45.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.786+0000 7f802d25b740 1 Processor -- start 2024-03-16T09:16:45.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.786+0000 7f802d25b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfa4c60 0x5575edfb3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.786+0000 7f802d25b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa5650 0x5575edfb3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.786+0000 7f802d25b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfaa7c0 0x5575ee01f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.786+0000 7f8023fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfa4c60 0x5575edfb3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.786+0000 7f8023fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfa4c60 0x5575edfb3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40880/0 (socket says 172.21.15.112:40880) 2024-03-16T09:16:45.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.786+0000 7f8023fff640 1 -- 172.21.15.112:0/3902651172 learned_addr learned my addr 172.21.15.112:0/3902651172 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:45.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.786+0000 7f802d25b740 1 -- 172.21.15.112:0/3902651172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80241a5270 con 0x5575edfa4c60 2024-03-16T09:16:45.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.786+0000 7f80237fe640 1 --2- 172.21.15.112:0/3902651172 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa5650 0x5575edfb3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.786+0000 7f802d25b740 1 -- 172.21.15.112:0/3902651172 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f80241a66e0 con 0x5575edfaa7c0 2024-03-16T09:16:45.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.786+0000 7f8028c12640 1 --2- 172.21.15.112:0/3902651172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfaa7c0 0x5575ee01f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.786+0000 7f802d25b740 1 -- 172.21.15.112:0/3902651172 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f80241a5940 con 0x5575edfa5650 2024-03-16T09:16:45.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f802d25b740 10 client.0 _ll_register_callbacks cb 0x7f8018009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:45.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f80237fe640 1 -- 172.21.15.112:0/3902651172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfaa7c0 msgr2=0x5575ee01f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f80237fe640 1 --2- 172.21.15.112:0/3902651172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfaa7c0 0x5575ee01f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f80237fe640 1 -- 172.21.15.112:0/3902651172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfa4c60 msgr2=0x5575edfb3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:45.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f80237fe640 1 --2- 172.21.15.112:0/3902651172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfa4c60 0x5575edfb3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f80237fe640 1 -- 172.21.15.112:0/3902651172 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575ee01f830 con 0x5575edfa5650 2024-03-16T09:16:45.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f8023fff640 1 --2- 172.21.15.112:0/3902651172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfa4c60 0x5575edfb3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:45.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f8028c12640 1 --2- 172.21.15.112:0/3902651172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfaa7c0 0x5575ee01f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:45.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f802d25b740 1 -- 172.21.15.112:0/3902651172 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f801402e770 con 0x5575edfa5650 2024-03-16T09:16:45.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f80237fe640 1 --2- 172.21.15.112:0/3902651172 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa5650 0x5575edfb3b90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8018097b70 tx=0x7f80180cc7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=723ad3f29f781903 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:45.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f80037fe640 10 client.25531 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T09:16:45.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f80037fe640 1 -- 172.21.15.112:0/3902651172 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018003bc0 con 0x5575edfa5650 2024-03-16T09:16:45.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f80037fe640 1 -- 172.21.15.112:0/3902651172 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f801809b970 con 0x5575edfa5650 2024-03-16T09:16:45.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f80037fe640 1 -- 172.21.15.112:0/3902651172 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f80180a3eb0 con 0x5575edfa5650 2024-03-16T09:16:45.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f802d25b740 10 client.25531 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:45.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f802d25b740 1 -- 172.21.15.112:0/3902651172 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({fsmap=22}) -- 0x5575ee0209e0 con 0x5575edfa5650 2024-03-16T09:16:45.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f80037fe640 1 -- 172.21.15.112:0/3902651172 <== mon.1 v2:172.21.15.194:3300/0 4 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f80180b4370 con 0x5575edfa5650 2024-03-16T09:16:45.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f802d25b740 10 client.25531 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:45.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f802d25b740 10 client.25531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:45.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.787+0000 7f802d25b740 20 client.25531 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:45.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.788+0000 7f802d25b740 1 --2- 172.21.15.112:0/3902651172 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5575ee0210f0 0x5575ee023510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:45.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.788+0000 7f802d25b740 4 client.25531 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T09:16:45.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.788+0000 7f802d25b740 1 -- 172.21.15.112:0/3902651172 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575edf0cd20 con 0x5575ee0210f0 2024-03-16T09:16:45.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.788+0000 7f8023fff640 1 --2- 172.21.15.112:0/3902651172 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5575ee0210f0 0x5575ee023510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:45.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.788+0000 7f8023fff640 1 --2- 172.21.15.112:0/3902651172 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5575ee0210f0 0x5575ee023510 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=282491a6511b3bd server_cookie=b1bf19ea2b82ff27 in_seq=0 out_seq=0 2024-03-16T09:16:45.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.788+0000 7f80037fe640 10 client.25531 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:45.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.788+0000 7f80037fe640 1 -- 172.21.15.112:0/3902651172 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575edf0cd20 con 0x5575ee0210f0 2024-03-16T09:16:45.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.788+0000 7f80037fe640 10 client.25531 handle_command_reply: tid=0 2024-03-16T09:16:45.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.788+0000 7f802d25b740 1 client.25531 shutdown 2024-03-16T09:16:45.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.788+0000 7f802d25b740 1 -- 172.21.15.112:0/3902651172 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa5650 msgr2=0x5575edfb3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.788+0000 7f802d25b740 1 --2- 172.21.15.112:0/3902651172 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa5650 0x5575edfb3b90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8018097b70 tx=0x7f80180cc7c0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.789+0000 7f802d25b740 1 -- 172.21.15.112:0/3902651172 shutdown_connections 2024-03-16T09:16:45.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.789+0000 7f802d25b740 1 --2- 172.21.15.112:0/3902651172 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5575ee0210f0 0x5575ee023510 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.789+0000 7f80037fe640 0 client.25531 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:45.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.789+0000 7f802d25b740 1 --2- 172.21.15.112:0/3902651172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575edfaa7c0 0x5575ee01f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.789+0000 7f802d25b740 1 --2- 172.21.15.112:0/3902651172 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5575edfa5650 0x5575edfb3b90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.789+0000 7f802d25b740 1 --2- 172.21.15.112:0/3902651172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5575edfa4c60 0x5575edfb3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.789+0000 7f802d25b740 1 -- 172.21.15.112:0/3902651172 >> 172.21.15.112:0/3902651172 conn(0x5575edf9bdc0 msgr2=0x5575edf9f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:45.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.789+0000 7f802d25b740 1 -- 172.21.15.112:0/3902651172 shutdown_connections 2024-03-16T09:16:45.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.789+0000 7f802d25b740 1 -- 172.21.15.112:0/3902651172 wait complete. 2024-03-16T09:16:45.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.789+0000 7f802d25b740 20 client.25531 trim_cache size 0 max 16384 2024-03-16T09:16:45.798 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-03-16T09:16:45.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.790+0000 7f802a675640 1 -- 172.21.15.112:0/3163761591 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f800c0a2910 msgr2=0x7f800c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.790+0000 7f802a675640 1 --2- 172.21.15.112:0/3163761591 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f800c0a2910 0x7f800c0a4dd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x5575edfa5a50 tx=0x7f8014005f90 comp rx=0 tx=0).stop 2024-03-16T09:16:45.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.790+0000 7f802a675640 1 -- 172.21.15.112:0/3163761591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80241018d0 msgr2=0x7f8024198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:45.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.790+0000 7f802a675640 1 --2- 172.21.15.112:0/3163761591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80241018d0 0x7f8024198100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f801800ea60 tx=0x7f801800ef20 comp rx=0 tx=0).stop 2024-03-16T09:16:45.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.790+0000 7f802a675640 1 -- 172.21.15.112:0/3163761591 shutdown_connections 2024-03-16T09:16:45.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.790+0000 7f802a675640 1 --2- 172.21.15.112:0/3163761591 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f800c0a2910 0x7f800c0a4dd0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.790+0000 7f802a675640 1 --2- 172.21.15.112:0/3163761591 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8024102290 0x7f802419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.790+0000 7f802a675640 1 --2- 172.21.15.112:0/3163761591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80241018d0 0x7f8024198100 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.790+0000 7f802a675640 1 --2- 172.21.15.112:0/3163761591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80241006e0 0x7f8024197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:45.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.790+0000 7f802a675640 1 -- 172.21.15.112:0/3163761591 >> 172.21.15.112:0/3163761591 conn(0x7f80240fbc70 msgr2=0x7f8024104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:45.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.790+0000 7f802a675640 1 -- 172.21.15.112:0/3163761591 shutdown_connections 2024-03-16T09:16:45.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:45.790+0000 7f802a675640 1 -- 172.21.15.112:0/3163761591 wait complete. 2024-03-16T09:16:46.287 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:46 smithi159 ceph-mon[36739]: pgmap v384: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T09:16:46.287 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:46 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2041218129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:16:46.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:46 smithi112 ceph-mon[32307]: pgmap v384: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T09:16:46.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:46 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2041218129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:16:46.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:46 smithi194 ceph-mon[36729]: pgmap v384: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-16T09:16:46.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:46 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2041218129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:16:46.807 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T09:16:46.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:16:46.808 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T09:16:46.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.914+0000 7fdfc87cd640 1 -- 172.21.15.112:0/2944951083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc0102320 msgr2=0x7fdfc0104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:46.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.914+0000 7fdfc87cd640 1 --2- 172.21.15.112:0/2944951083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc0102320 0x7fdfc0104710 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009900 tx=0x7fdfbc02bc30 comp rx=0 tx=0).stop 2024-03-16T09:16:46.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.914+0000 7fdfc87cd640 1 -- 172.21.15.112:0/2944951083 shutdown_connections 2024-03-16T09:16:46.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.914+0000 7fdfc87cd640 1 --2- 172.21.15.112:0/2944951083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfc0107580 0x7fdfc0109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.914+0000 7fdfc87cd640 1 --2- 172.21.15.112:0/2944951083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfc0104c50 0x7fdfc0107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.914+0000 7fdfc87cd640 1 --2- 172.21.15.112:0/2944951083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc0102320 0x7fdfc0104710 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.914+0000 7fdfc87cd640 1 -- 172.21.15.112:0/2944951083 >> 172.21.15.112:0/2944951083 conn(0x7fdfc00fbc70 msgr2=0x7fdfc00fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.914+0000 7fdfc87cd640 1 -- 172.21.15.112:0/2944951083 shutdown_connections 2024-03-16T09:16:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.915+0000 7fdfc87cd640 1 -- 172.21.15.112:0/2944951083 wait complete. 2024-03-16T09:16:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.915+0000 7fdfc87cd640 1 Processor -- start 2024-03-16T09:16:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.915+0000 7fdfc87cd640 1 -- start start 2024-03-16T09:16:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.915+0000 7fdfc87cd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfc0102320 0x7fdfc0197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.915+0000 7fdfc87cd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfc0104c50 0x7fdfc0198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.915+0000 7fdfc87cd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc0107580 0x7fdfc019f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.915+0000 7fdfc87cd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc010d550 con 0x7fdfc0107580 2024-03-16T09:16:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.915+0000 7fdfc87cd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc010d3d0 con 0x7fdfc0104c50 2024-03-16T09:16:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.915+0000 7fdfc87cd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc010d6d0 con 0x7fdfc0102320 2024-03-16T09:16:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.915+0000 7fdfc6d6a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc0107580 0x7fdfc019f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.915+0000 7fdfc6d6a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc0107580 0x7fdfc019f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40906/0 (socket says 172.21.15.112:40906) 2024-03-16T09:16:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.915+0000 7fdfc6d6a640 1 -- 172.21.15.112:0/3403218727 learned_addr learned my addr 172.21.15.112:0/3403218727 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.916+0000 7fdfc5d68640 1 --2- 172.21.15.112:0/3403218727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfc0104c50 0x7fdfc0198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:46.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.916+0000 7fdfc6569640 1 --2- 172.21.15.112:0/3403218727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfc0102320 0x7fdfc0197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:46.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.916+0000 7fdfc6d6a640 1 -- 172.21.15.112:0/3403218727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfc0102320 msgr2=0x7fdfc0197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:46.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.916+0000 7fdfc6d6a640 1 --2- 172.21.15.112:0/3403218727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfc0102320 0x7fdfc0197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.916+0000 7fdfc6d6a640 1 -- 172.21.15.112:0/3403218727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfc0104c50 msgr2=0x7fdfc0198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:46.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.916+0000 7fdfc6d6a640 1 --2- 172.21.15.112:0/3403218727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfc0104c50 0x7fdfc0198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.916+0000 7fdfc6d6a640 1 -- 172.21.15.112:0/3403218727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc019f850 con 0x7fdfc0107580 2024-03-16T09:16:46.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.916+0000 7fdfc5d68640 1 --2- 172.21.15.112:0/3403218727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfc0104c50 0x7fdfc0198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:46.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.916+0000 7fdfc6d6a640 1 --2- 172.21.15.112:0/3403218727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc0107580 0x7fdfc019f1b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdfb800eb10 tx=0x7fdfb800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:46.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.916+0000 7fdfb77fe640 1 -- 172.21.15.112:0/3403218727 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8004b20 con 0x7fdfc0107580 2024-03-16T09:16:46.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.916+0000 7fdfc87cd640 1 -- 172.21.15.112:0/3403218727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc019fb40 con 0x7fdfc0107580 2024-03-16T09:16:46.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.916+0000 7fdfb77fe640 1 -- 172.21.15.112:0/3403218727 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdfb8004cc0 con 0x7fdfc0107580 2024-03-16T09:16:46.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.916+0000 7fdfc87cd640 1 -- 172.21.15.112:0/3403218727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc01003a0 con 0x7fdfc0107580 2024-03-16T09:16:46.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.916+0000 7fdfb77fe640 1 -- 172.21.15.112:0/3403218727 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80107f0 con 0x7fdfc0107580 2024-03-16T09:16:46.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.919+0000 7fdfb77fe640 1 -- 172.21.15.112:0/3403218727 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7fdfb8005020 con 0x7fdfc0107580 2024-03-16T09:16:46.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.919+0000 7fdfcb3ba740 1 Processor -- start 2024-03-16T09:16:46.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.920+0000 7fdfcb3ba740 1 -- start start 2024-03-16T09:16:46.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.920+0000 7fdfcb3ba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b44300b990 0x55b44300bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:46.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.920+0000 7fdfcb3ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b44300c380 0x55b443010e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:46.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.920+0000 7fdfcb3ba740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4430114f0 0x55b443013980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:46.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.920+0000 7fdfcb3ba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b442f700d0 con 0x55b44300b990 2024-03-16T09:16:46.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.920+0000 7fdfcb3ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b442f18c70 con 0x55b44300c380 2024-03-16T09:16:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.920+0000 7fdfcb3ba740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b442f6c170 con 0x55b4430114f0 2024-03-16T09:16:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.920+0000 7fdfc5d68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b44300b990 0x55b44300bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.920+0000 7fdfc6569640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b44300c380 0x55b443010e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.920+0000 7fdfc5d68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b44300b990 0x55b44300bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40908/0 (socket says 172.21.15.112:40908) 2024-03-16T09:16:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.920+0000 7fdfc5d68640 1 -- 172.21.15.112:0/3563674763 learned_addr learned my addr 172.21.15.112:0/3563674763 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.920+0000 7fdfc6569640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b44300c380 0x55b443010e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:35782/0 (socket says 172.21.15.112:35782) 2024-03-16T09:16:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.920+0000 7fdfc5d68640 1 --2- 172.21.15.112:0/3563674763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4430114f0 0x55b443013980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.920+0000 7fdfc6569640 1 -- 172.21.15.112:0/3563674763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4430114f0 msgr2=0x55b443013980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.920+0000 7fdfc6569640 1 --2- 172.21.15.112:0/3563674763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4430114f0 0x55b443013980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.920+0000 7fdfc6569640 1 -- 172.21.15.112:0/3563674763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b44300b990 msgr2=0x55b44300bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.920+0000 7fdfc6569640 1 --2- 172.21.15.112:0/3563674763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b44300b990 0x55b44300bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.920+0000 7fdfc6569640 1 -- 172.21.15.112:0/3563674763 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b442d4c7e0 con 0x55b44300c380 2024-03-16T09:16:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.920+0000 7fdfb77fe640 1 --2- 172.21.15.112:0/3403218727 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdfac0a2910 0x7fdfac0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:46.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.921+0000 7fdfc5d68640 1 --2- 172.21.15.112:0/3403218727 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdfac0a2910 0x7fdfac0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:46.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.921+0000 7fdfb77fe640 1 -- 172.21.15.112:0/3403218727 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdfb80cf1b0 con 0x7fdfc0107580 2024-03-16T09:16:46.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.921+0000 7fdfc6569640 1 --2- 172.21.15.112:0/3563674763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b44300c380 0x55b443010e80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009230 tx=0x7fdfbc009670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7e3ec1ca9bba99 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:46.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.921+0000 7fdfb4ff9640 1 -- 172.21.15.112:0/3563674763 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc02c5d0 con 0x55b44300c380 2024-03-16T09:16:46.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.921+0000 7fdfb4ff9640 1 -- 172.21.15.112:0/3563674763 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdfbc004040 con 0x55b44300c380 2024-03-16T09:16:46.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.921+0000 7fdfc5d68640 1 --2- 172.21.15.112:0/3403218727 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdfac0a2910 0x7fdfac0a4dd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55b44300cbe0 tx=0x7fdfb0009c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:46.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.921+0000 7fdfb4ff9640 1 -- 172.21.15.112:0/3563674763 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0046c0 con 0x55b44300c380 2024-03-16T09:16:46.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.922+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/3563674763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b44300c380 msgr2=0x55b443010e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:46.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.922+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/3563674763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b44300c380 0x55b443010e80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009230 tx=0x7fdfbc009670 comp rx=0 tx=0).stop 2024-03-16T09:16:46.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.922+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/3563674763 shutdown_connections 2024-03-16T09:16:46.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.922+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/3563674763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4430114f0 0x55b443013980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.922+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/3563674763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b44300c380 0x55b443010e80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.922+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/3563674763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b44300b990 0x55b44300bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.922+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/3563674763 >> 172.21.15.112:0/3563674763 conn(0x55b442fd3c50 msgr2=0x55b443007300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:46.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.922+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/3563674763 shutdown_connections 2024-03-16T09:16:46.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.922+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/3563674763 wait complete. 2024-03-16T09:16:46.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.923+0000 7fdfcb3ba740 1 -- start start 2024-03-16T09:16:46.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.923+0000 7fdfcb3ba740 1 Processor -- start 2024-03-16T09:16:46.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.923+0000 7fdfcb3ba740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b44300b990 0x55b443015e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:46.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.923+0000 7fdfcb3ba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b44300c380 0x55b443016350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:46.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.923+0000 7fdfcb3ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4430114f0 0x55b44301d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:46.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.923+0000 7fdfcb3ba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b443014440 con 0x55b44300c380 2024-03-16T09:16:46.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.923+0000 7fdfcb3ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b442f2c0d0 con 0x55b4430114f0 2024-03-16T09:16:46.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.923+0000 7fdfcb3ba740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b4430145c0 con 0x55b44300b990 2024-03-16T09:16:46.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.923+0000 7fdfc5d68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b44300c380 0x55b443016350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:46.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.923+0000 7fdfc6569640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b44300b990 0x55b443015e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:46.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.923+0000 7fdfc6569640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b44300b990 0x55b443015e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:56712/0 (socket says 172.21.15.112:56712) 2024-03-16T09:16:46.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.923+0000 7fdfc5d68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b44300c380 0x55b443016350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40922/0 (socket says 172.21.15.112:40922) 2024-03-16T09:16:46.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.923+0000 7fdfc6569640 1 -- 172.21.15.112:0/2589191878 learned_addr learned my addr 172.21.15.112:0/2589191878 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:46.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.923+0000 7fdfc6d6a640 1 --2- 172.21.15.112:0/2589191878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4430114f0 0x55b44301d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:46.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.924+0000 7fdfcb3ba740 10 client.0 _ll_register_callbacks cb 0x55b442ef5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:46.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.924+0000 7fdfc6569640 1 -- 172.21.15.112:0/2589191878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4430114f0 msgr2=0x55b44301d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:46.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.924+0000 7fdfc6569640 1 --2- 172.21.15.112:0/2589191878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4430114f0 0x55b44301d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.924+0000 7fdfc6569640 1 -- 172.21.15.112:0/2589191878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b44300c380 msgr2=0x55b443016350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:46.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.924+0000 7fdfc6569640 1 --2- 172.21.15.112:0/2589191878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b44300c380 0x55b443016350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.924+0000 7fdfc6569640 1 -- 172.21.15.112:0/2589191878 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b442dacf70 con 0x55b44300b990 2024-03-16T09:16:46.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.924+0000 7fdfc5d68640 1 --2- 172.21.15.112:0/2589191878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b44300c380 0x55b443016350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:46.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.924+0000 7fdfc6d6a640 1 --2- 172.21.15.112:0/2589191878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4430114f0 0x55b44301d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:46.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.924+0000 7fdfc6569640 1 --2- 172.21.15.112:0/2589191878 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b44300b990 0x55b443015e10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc02c170 tx=0x7fdfbc02c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55ce0d5c3e6790da server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:46.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.924+0000 7fdfa9ffb640 10 client.25543 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T09:16:46.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.924+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/2589191878 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfbc004040 con 0x55b44300b990 2024-03-16T09:16:46.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.924+0000 7fdfa9ffb640 1 -- 172.21.15.112:0/2589191878 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc02cad0 con 0x55b44300b990 2024-03-16T09:16:46.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.924+0000 7fdfa9ffb640 1 -- 172.21.15.112:0/2589191878 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdfbc02cc70 con 0x55b44300b990 2024-03-16T09:16:46.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.924+0000 7fdfa9ffb640 1 -- 172.21.15.112:0/2589191878 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fdfbc0035f0 con 0x55b44300b990 2024-03-16T09:16:46.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.924+0000 7fdfcb3ba740 10 client.25543 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:46.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.924+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/2589191878 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({fsmap=22}) -- 0x55b442c44170 con 0x55b44300b990 2024-03-16T09:16:46.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.925+0000 7fdfa9ffb640 1 -- 172.21.15.112:0/2589191878 <== mon.1 v2:172.21.15.194:3300/0 4 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fdfbc003790 con 0x55b44300b990 2024-03-16T09:16:46.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.925+0000 7fdfcb3ba740 10 client.25543 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:46.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.925+0000 7fdfcb3ba740 10 client.25543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:46.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.925+0000 7fdfcb3ba740 20 client.25543 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:46.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.925+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/2589191878 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55b443021370 0x55b443023760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:46.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.925+0000 7fdfcb3ba740 4 client.25543 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:16:46.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.925+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/2589191878 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdfbc004040 con 0x55b443021370 2024-03-16T09:16:46.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.926+0000 7fdfc5d68640 1 --2- 172.21.15.112:0/2589191878 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55b443021370 0x55b443023760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:46.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.926+0000 7fdfc5d68640 1 --2- 172.21.15.112:0/2589191878 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55b443021370 0x55b443023760 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=380cf9295740ff4a server_cookie=1fd9456de355be1e in_seq=0 out_seq=0 2024-03-16T09:16:46.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.926+0000 7fdfa9ffb640 10 client.25543 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:46.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.926+0000 7fdfa9ffb640 1 -- 172.21.15.112:0/2589191878 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fdfbc004040 con 0x55b443021370 2024-03-16T09:16:46.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.926+0000 7fdfa9ffb640 10 client.25543 handle_command_reply: tid=0 2024-03-16T09:16:46.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.927+0000 7fdfcb3ba740 1 client.25543 shutdown 2024-03-16T09:16:46.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.927+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/2589191878 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b44300b990 msgr2=0x55b443015e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:46.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.927+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/2589191878 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b44300b990 0x55b443015e10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc02c170 tx=0x7fdfbc02c7e0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.927+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/2589191878 shutdown_connections 2024-03-16T09:16:46.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.927+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/2589191878 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55b443021370 0x55b443023760 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.927+0000 7fdfa9ffb640 0 client.25543 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:46.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.927+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/2589191878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4430114f0 0x55b44301d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.927+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/2589191878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b44300c380 0x55b443016350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.927+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/2589191878 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b44300b990 0x55b443015e10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.927+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/2589191878 >> 172.21.15.112:0/2589191878 conn(0x55b442fd3c50 msgr2=0x55b4430064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:46.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.927+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/2589191878 shutdown_connections 2024-03-16T09:16:46.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.927+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/2589191878 wait complete. 2024-03-16T09:16:46.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.927+0000 7fdfcb3ba740 20 client.25543 trim_cache size 0 max 16384 2024-03-16T09:16:46.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.961+0000 7fdfcb3ba740 1 Processor -- start 2024-03-16T09:16:46.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.961+0000 7fdfcb3ba740 1 -- start start 2024-03-16T09:16:46.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.961+0000 7fdfcb3ba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b442fd3c50 0x55b443015430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:46.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.961+0000 7fdfcb3ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b44300b990 0x55b44301d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:46.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.961+0000 7fdfcb3ba740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b44300c380 0x55b443015970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:46.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.961+0000 7fdfcb3ba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc01ad770 con 0x55b442fd3c50 2024-03-16T09:16:46.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.961+0000 7fdfcb3ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc01ab040 con 0x55b44300b990 2024-03-16T09:16:46.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.961+0000 7fdfcb3ba740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc01a8e90 con 0x55b44300c380 2024-03-16T09:16:46.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.962+0000 7fdfc6569640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b442fd3c50 0x55b443015430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:46.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.962+0000 7fdfc5d68640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b44300b990 0x55b44301d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.962+0000 7fdfc6569640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b442fd3c50 0x55b443015430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40934/0 (socket says 172.21.15.112:40934) 2024-03-16T09:16:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.962+0000 7fdfc6569640 1 -- 172.21.15.112:0/2668711484 learned_addr learned my addr 172.21.15.112:0/2668711484 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.962+0000 7fdfc6d6a640 1 --2- 172.21.15.112:0/2668711484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b44300c380 0x55b443015970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.962+0000 7fdfc6569640 1 -- 172.21.15.112:0/2668711484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b44300c380 msgr2=0x55b443015970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.962+0000 7fdfc6569640 1 --2- 172.21.15.112:0/2668711484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b44300c380 0x55b443015970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.962+0000 7fdfc6569640 1 -- 172.21.15.112:0/2668711484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b44300b990 msgr2=0x55b44301d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.962+0000 7fdfc6569640 1 --2- 172.21.15.112:0/2668711484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b44300b990 0x55b44301d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.962+0000 7fdfc6569640 1 -- 172.21.15.112:0/2668711484 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b443014090 con 0x55b442fd3c50 2024-03-16T09:16:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.962+0000 7fdfc6569640 1 --2- 172.21.15.112:0/2668711484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b442fd3c50 0x55b443015430 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc000c00 tx=0x7fdfbc04a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb2108f27f55bde server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.962+0000 7fdfabfff640 1 -- 172.21.15.112:0/2668711484 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc02ccd0 con 0x55b442fd3c50 2024-03-16T09:16:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.962+0000 7fdfabfff640 1 -- 172.21.15.112:0/2668711484 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdfbc0344e0 con 0x55b442fd3c50 2024-03-16T09:16:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.962+0000 7fdfabfff640 1 -- 172.21.15.112:0/2668711484 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc05b680 con 0x55b442fd3c50 2024-03-16T09:16:46.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.963+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/2668711484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b442fd3c50 msgr2=0x55b443015430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.963+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/2668711484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b442fd3c50 0x55b443015430 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc000c00 tx=0x7fdfbc04a000 comp rx=0 tx=0).stop 2024-03-16T09:16:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.963+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/2668711484 shutdown_connections 2024-03-16T09:16:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.963+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/2668711484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b44300c380 0x55b443015970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.964+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/2668711484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b44300b990 0x55b44301d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.964+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/2668711484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b442fd3c50 0x55b443015430 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.964+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/2668711484 >> 172.21.15.112:0/2668711484 conn(0x55b4430073c0 msgr2=0x55b4430064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.964+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/2668711484 shutdown_connections 2024-03-16T09:16:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.964+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/2668711484 wait complete. 2024-03-16T09:16:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.964+0000 7fdfcb3ba740 1 -- start start 2024-03-16T09:16:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.964+0000 7fdfcb3ba740 1 Processor -- start 2024-03-16T09:16:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.964+0000 7fdfcb3ba740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b442fd3c50 0x55b44301b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.964+0000 7fdfcb3ba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b44300b990 0x55b44301b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.965+0000 7fdfc6569640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b442fd3c50 0x55b44301b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:46.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.965+0000 7fdfc6569640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b442fd3c50 0x55b44301b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:56726/0 (socket says 172.21.15.112:56726) 2024-03-16T09:16:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.965+0000 7fdfc6569640 1 -- 172.21.15.112:0/910893859 learned_addr learned my addr 172.21.15.112:0/910893859 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.965+0000 7fdfc5d68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b44300b990 0x55b44301b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.965+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/910893859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b44300c380 0x55b44301c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.965+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/910893859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc01a9500 con 0x55b44300b990 2024-03-16T09:16:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.965+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/910893859 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc01aa970 con 0x55b44300c380 2024-03-16T09:16:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.965+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/910893859 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc01a9bd0 con 0x55b442fd3c50 2024-03-16T09:16:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.965+0000 7fdfc6d6a640 1 --2- 172.21.15.112:0/910893859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b44300c380 0x55b44301c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.965+0000 7fdfc6569640 1 -- 172.21.15.112:0/910893859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b44300c380 msgr2=0x55b44301c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.965+0000 7fdfc6569640 1 --2- 172.21.15.112:0/910893859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b44300c380 0x55b44301c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.965+0000 7fdfc6569640 1 -- 172.21.15.112:0/910893859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b44300b990 msgr2=0x55b44301b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.965+0000 7fdfc6569640 1 --2- 172.21.15.112:0/910893859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b44300b990 0x55b44301b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.965+0000 7fdfc6569640 1 -- 172.21.15.112:0/910893859 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b44301ce60 con 0x55b442fd3c50 2024-03-16T09:16:46.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.965+0000 7fdfc5d68640 1 --2- 172.21.15.112:0/910893859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b44300b990 0x55b44301b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:46.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.966+0000 7fdfcb3ba740 10 client.0 _ll_register_callbacks cb 0x7fdfb0009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:46.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.966+0000 7fdfc6569640 1 --2- 172.21.15.112:0/910893859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b442fd3c50 0x55b44301b320 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc004650 tx=0x7fdfbc003ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e62d0e92177eaa71 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:46.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.966+0000 7fdfa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T09:16:46.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.966+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/910893859 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfbc0344e0 con 0x55b442fd3c50 2024-03-16T09:16:46.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.966+0000 7fdfa9ffb640 1 -- 172.21.15.112:0/910893859 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc034aa0 con 0x55b442fd3c50 2024-03-16T09:16:46.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.966+0000 7fdfa9ffb640 1 -- 172.21.15.112:0/910893859 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdfbc03f430 con 0x55b442fd3c50 2024-03-16T09:16:46.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.966+0000 7fdfa9ffb640 1 -- 172.21.15.112:0/910893859 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fdfbc03f6b0 con 0x55b442fd3c50 2024-03-16T09:16:46.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.966+0000 7fdfcb3ba740 10 client.25549 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:46.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.966+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/910893859 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({fsmap=22}) -- 0x55b443087be0 con 0x55b442fd3c50 2024-03-16T09:16:46.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.966+0000 7fdfa9ffb640 1 -- 172.21.15.112:0/910893859 <== mon.1 v2:172.21.15.194:3300/0 4 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fdfbc03f850 con 0x55b442fd3c50 2024-03-16T09:16:46.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.966+0000 7fdfcb3ba740 10 client.25549 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:46.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.966+0000 7fdfcb3ba740 10 client.25549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:46.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.966+0000 7fdfcb3ba740 20 client.25549 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:46.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.966+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/910893859 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55b4430114f0 0x55b44308a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:46.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.967+0000 7fdfcb3ba740 4 client.25549 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T09:16:46.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.967+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/910893859 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fdfbc0344e0 con 0x55b4430114f0 2024-03-16T09:16:46.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.967+0000 7fdfc5d68640 1 --2- 172.21.15.112:0/910893859 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55b4430114f0 0x55b44308a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:46.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.967+0000 7fdfa9ffb640 10 client.25549 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:46.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.967+0000 7fdfc5d68640 1 --2- 172.21.15.112:0/910893859 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55b4430114f0 0x55b44308a430 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd8eb855d3981a7d server_cookie=e3e040599fc42240 in_seq=0 out_seq=0 2024-03-16T09:16:46.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.968+0000 7fdfa9ffb640 1 -- 172.21.15.112:0/910893859 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fdfbc0344e0 con 0x55b4430114f0 2024-03-16T09:16:46.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.968+0000 7fdfa9ffb640 10 client.25549 handle_command_reply: tid=0 2024-03-16T09:16:46.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.968+0000 7fdfcb3ba740 1 client.25549 shutdown 2024-03-16T09:16:46.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.968+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/910893859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b442fd3c50 msgr2=0x55b44301b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:46.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.969+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/910893859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b442fd3c50 0x55b44301b320 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc004650 tx=0x7fdfbc003ec0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.969+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/910893859 shutdown_connections 2024-03-16T09:16:46.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.969+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/910893859 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55b4430114f0 0x55b44308a430 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.969+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/910893859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b44300c380 0x55b44301c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.969+0000 7fdfa9ffb640 0 client.25549 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:46.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.969+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/910893859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b44300b990 0x55b44301b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.969+0000 7fdfcb3ba740 1 --2- 172.21.15.112:0/910893859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b442fd3c50 0x55b44301b320 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.969+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/910893859 >> 172.21.15.112:0/910893859 conn(0x55b4430073c0 msgr2=0x55b44300a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:46.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.969+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/910893859 shutdown_connections 2024-03-16T09:16:46.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.969+0000 7fdfcb3ba740 1 -- 172.21.15.112:0/910893859 wait complete. 2024-03-16T09:16:46.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.969+0000 7fdfcb3ba740 20 client.25549 trim_cache size 0 max 16384 2024-03-16T09:16:46.977 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:16:46.977 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-03-16T09:16:46.977 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "a04fa4f2-b33b-4628-9fc1-5899669d0ecd", 2024-03-16T09:16:46.977 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-03-16T09:16:46.977 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:16:46.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.970+0000 7fdfc87cd640 1 -- 172.21.15.112:0/3403218727 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdfac0a2910 msgr2=0x7fdfac0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:46.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.970+0000 7fdfc87cd640 1 --2- 172.21.15.112:0/3403218727 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdfac0a2910 0x7fdfac0a4dd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55b44300cbe0 tx=0x7fdfb0009c00 comp rx=0 tx=0).stop 2024-03-16T09:16:46.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.970+0000 7fdfc87cd640 1 -- 172.21.15.112:0/3403218727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc0107580 msgr2=0x7fdfc019f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:46.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.970+0000 7fdfc87cd640 1 --2- 172.21.15.112:0/3403218727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc0107580 0x7fdfc019f1b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdfb800eb10 tx=0x7fdfb800c2d0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.970+0000 7fdfc87cd640 1 -- 172.21.15.112:0/3403218727 shutdown_connections 2024-03-16T09:16:46.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.970+0000 7fdfc87cd640 1 --2- 172.21.15.112:0/3403218727 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdfac0a2910 0x7fdfac0a4dd0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.971+0000 7fdfc87cd640 1 --2- 172.21.15.112:0/3403218727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfc0107580 0x7fdfc019f1b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.971+0000 7fdfc87cd640 1 --2- 172.21.15.112:0/3403218727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfc0104c50 0x7fdfc0198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.971+0000 7fdfc87cd640 1 --2- 172.21.15.112:0/3403218727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdfc0102320 0x7fdfc0197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:46.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.971+0000 7fdfc87cd640 1 -- 172.21.15.112:0/3403218727 >> 172.21.15.112:0/3403218727 conn(0x7fdfc00fbc70 msgr2=0x7fdfc0102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:46.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.971+0000 7fdfc87cd640 1 -- 172.21.15.112:0/3403218727 shutdown_connections 2024-03-16T09:16:46.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:46.971+0000 7fdfc87cd640 1 -- 172.21.15.112:0/3403218727 wait complete. 2024-03-16T09:16:46.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T09:16:46.987 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T09:16:47.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.094+0000 7fbf4aff3640 1 -- 172.21.15.112:0/1200570691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf44107580 msgr2=0x7fbf44109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:47.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.094+0000 7fbf4aff3640 1 --2- 172.21.15.112:0/1200570691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf44107580 0x7fbf44109970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009ca0 tx=0x7fbf3c02c100 comp rx=0 tx=0).stop 2024-03-16T09:16:47.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.094+0000 7fbf4aff3640 1 -- 172.21.15.112:0/1200570691 shutdown_connections 2024-03-16T09:16:47.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.094+0000 7fbf4aff3640 1 --2- 172.21.15.112:0/1200570691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf44107580 0x7fbf44109970 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.094+0000 7fbf4aff3640 1 --2- 172.21.15.112:0/1200570691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf44104c50 0x7fbf44107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.094+0000 7fbf4aff3640 1 --2- 172.21.15.112:0/1200570691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf44102320 0x7fbf44104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.094+0000 7fbf4aff3640 1 -- 172.21.15.112:0/1200570691 >> 172.21.15.112:0/1200570691 conn(0x7fbf440fbc70 msgr2=0x7fbf440fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:47.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.094+0000 7fbf4aff3640 1 -- 172.21.15.112:0/1200570691 shutdown_connections 2024-03-16T09:16:47.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.094+0000 7fbf4aff3640 1 -- 172.21.15.112:0/1200570691 wait complete. 2024-03-16T09:16:47.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf4aff3640 1 Processor -- start 2024-03-16T09:16:47.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf4aff3640 1 -- start start 2024-03-16T09:16:47.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf4aff3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf44102320 0x7fbf44199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf4aff3640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf44104c50 0x7fbf4419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf4aff3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf44107580 0x7fbf441a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf4aff3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4410d5a0 con 0x7fbf44107580 2024-03-16T09:16:47.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf4aff3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4410d420 con 0x7fbf44102320 2024-03-16T09:16:47.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf4aff3640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4410d720 con 0x7fbf44104c50 2024-03-16T09:16:47.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf48d8f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf44102320 0x7fbf44199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf43fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf44104c50 0x7fbf4419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf43fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf44104c50 0x7fbf4419a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:56734/0 (socket says 172.21.15.112:56734) 2024-03-16T09:16:47.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf48d8f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf44102320 0x7fbf44199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:35816/0 (socket says 172.21.15.112:35816) 2024-03-16T09:16:47.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf43fff640 1 -- 172.21.15.112:0/2223425890 learned_addr learned my addr 172.21.15.112:0/2223425890 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:47.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf49590640 1 --2- 172.21.15.112:0/2223425890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf44107580 0x7fbf441a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf43fff640 1 -- 172.21.15.112:0/2223425890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf44102320 msgr2=0x7fbf44199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:47.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf43fff640 1 --2- 172.21.15.112:0/2223425890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf44102320 0x7fbf44199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf43fff640 1 -- 172.21.15.112:0/2223425890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf44107580 msgr2=0x7fbf441a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:47.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf43fff640 1 --2- 172.21.15.112:0/2223425890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf44107580 0x7fbf441a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf43fff640 1 -- 172.21.15.112:0/2223425890 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf441a1830 con 0x7fbf44104c50 2024-03-16T09:16:47.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.095+0000 7fbf49590640 1 --2- 172.21.15.112:0/2223425890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf44107580 0x7fbf441a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:47.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.096+0000 7fbf43fff640 1 --2- 172.21.15.112:0/2223425890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf44104c50 0x7fbf4419a160 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000e9a0 tx=0x7fbf3000ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:47.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.096+0000 7fbf41ffb640 1 -- 172.21.15.112:0/2223425890 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3000cd80 con 0x7fbf44104c50 2024-03-16T09:16:47.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.096+0000 7fbf4aff3640 1 -- 172.21.15.112:0/2223425890 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf441a1b20 con 0x7fbf44104c50 2024-03-16T09:16:47.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.096+0000 7fbf41ffb640 1 -- 172.21.15.112:0/2223425890 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbf300035f0 con 0x7fbf44104c50 2024-03-16T09:16:47.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.096+0000 7fbf41ffb640 1 -- 172.21.15.112:0/2223425890 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30010700 con 0x7fbf44104c50 2024-03-16T09:16:47.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.096+0000 7fbf4aff3640 1 -- 172.21.15.112:0/2223425890 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf441a20c0 con 0x7fbf44104c50 2024-03-16T09:16:47.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.099+0000 7fbf41ffb640 1 -- 172.21.15.112:0/2223425890 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7fbf300037a0 con 0x7fbf44104c50 2024-03-16T09:16:47.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.099+0000 7fbf41ffb640 1 --2- 172.21.15.112:0/2223425890 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fbf340a2910 0x7fbf340a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.099+0000 7fbf4dbd9740 1 Processor -- start 2024-03-16T09:16:47.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.100+0000 7fbf48d8f640 1 --2- 172.21.15.112:0/2223425890 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fbf340a2910 0x7fbf340a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.100+0000 7fbf4dbd9740 1 -- start start 2024-03-16T09:16:47.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.100+0000 7fbf4dbd9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e79170 0x558072e79590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.100+0000 7fbf4dbd9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e79b60 0x558072e7e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.100+0000 7fbf4dbd9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072e7ecd0 0x558072e81160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.100+0000 7fbf41ffb640 1 -- 172.21.15.112:0/2223425890 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fbf300ceed0 con 0x7fbf44104c50 2024-03-16T09:16:47.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.100+0000 7fbf4dbd9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558072dde500 con 0x558072e79b60 2024-03-16T09:16:47.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.100+0000 7fbf48d8f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e79170 0x558072e79590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.100+0000 7fbf43fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e79b60 0x558072e7e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.100+0000 7fbf48d8f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e79170 0x558072e79590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:56746/0 (socket says 172.21.15.112:56746) 2024-03-16T09:16:47.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.100+0000 7fbf48d8f640 1 -- 172.21.15.112:0/4106414800 learned_addr learned my addr 172.21.15.112:0/4106414800 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:47.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.100+0000 7fbf49590640 1 --2- 172.21.15.112:0/4106414800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072e7ecd0 0x558072e81160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.100+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/4106414800 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558072d86be0 con 0x558072e7ecd0 2024-03-16T09:16:47.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.100+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/4106414800 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558072dda5a0 con 0x558072e79170 2024-03-16T09:16:47.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.100+0000 7fbf48d8f640 1 --2- 172.21.15.112:0/2223425890 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fbf340a2910 0x7fbf340a4dd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbf4419b0f0 tx=0x7fbf38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:47.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.100+0000 7fbf43fff640 1 -- 172.21.15.112:0/4106414800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e79170 msgr2=0x558072e79590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:47.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.101+0000 7fbf43fff640 1 --2- 172.21.15.112:0/4106414800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e79170 0x558072e79590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.101+0000 7fbf43fff640 1 -- 172.21.15.112:0/4106414800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072e7ecd0 msgr2=0x558072e81160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:47.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.101+0000 7fbf43fff640 1 --2- 172.21.15.112:0/4106414800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072e7ecd0 0x558072e81160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.101+0000 7fbf43fff640 1 -- 172.21.15.112:0/4106414800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558072d9a4b0 con 0x558072e79b60 2024-03-16T09:16:47.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.101+0000 7fbf49590640 1 --2- 172.21.15.112:0/4106414800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072e7ecd0 0x558072e81160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:47.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.101+0000 7fbf43fff640 1 --2- 172.21.15.112:0/4106414800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e79b60 0x558072e7e660 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbf300cc890 tx=0x7fbf3009b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec70fc5c2a0563d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:47.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.101+0000 7fbf2effd640 1 -- 172.21.15.112:0/4106414800 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf300b4070 con 0x558072e79b60 2024-03-16T09:16:47.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.101+0000 7fbf2effd640 1 -- 172.21.15.112:0/4106414800 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbf300860c0 con 0x558072e79b60 2024-03-16T09:16:47.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.101+0000 7fbf2effd640 1 -- 172.21.15.112:0/4106414800 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3009ab70 con 0x558072e79b60 2024-03-16T09:16:47.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.102+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/4106414800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e79b60 msgr2=0x558072e7e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:47.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.102+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/4106414800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e79b60 0x558072e7e660 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbf300cc890 tx=0x7fbf3009b320 comp rx=0 tx=0).stop 2024-03-16T09:16:47.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.102+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/4106414800 shutdown_connections 2024-03-16T09:16:47.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.102+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/4106414800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072e7ecd0 0x558072e81160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.102+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/4106414800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e79b60 0x558072e7e660 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.102+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/4106414800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e79170 0x558072e79590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.102+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/4106414800 >> 172.21.15.112:0/4106414800 conn(0x558072e746e0 msgr2=0x558072e74ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.102+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/4106414800 shutdown_connections 2024-03-16T09:16:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.102+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/4106414800 wait complete. 2024-03-16T09:16:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.102+0000 7fbf4dbd9740 1 -- start start 2024-03-16T09:16:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.102+0000 7fbf4dbd9740 1 Processor -- start 2024-03-16T09:16:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.103+0000 7fbf4dbd9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072e79170 0x558072e83810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.103+0000 7fbf4dbd9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e7ecd0 0x558072e83d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.103+0000 7fbf4dbd9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e88d80 0x558072e8b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.103+0000 7fbf4dbd9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558072e81c80 con 0x558072e88d80 2024-03-16T09:16:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.103+0000 7fbf4dbd9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558072d9a160 con 0x558072e79170 2024-03-16T09:16:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.103+0000 7fbf4dbd9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558072e81e00 con 0x558072e7ecd0 2024-03-16T09:16:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.103+0000 7fbf4dbd9740 10 client.0 _ll_register_callbacks cb 0x558072d7e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:47.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.103+0000 7fbf48d8f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072e79170 0x558072e83810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.103+0000 7fbf43fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e7ecd0 0x558072e83d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.103+0000 7fbf49590640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e88d80 0x558072e8b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.103+0000 7fbf48d8f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072e79170 0x558072e83810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:35832/0 (socket says 172.21.15.112:35832) 2024-03-16T09:16:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.103+0000 7fbf48d8f640 1 -- 172.21.15.112:0/2143256633 learned_addr learned my addr 172.21.15.112:0/2143256633 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.103+0000 7fbf48d8f640 1 -- 172.21.15.112:0/2143256633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e7ecd0 msgr2=0x558072e83d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.103+0000 7fbf48d8f640 1 --2- 172.21.15.112:0/2143256633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e7ecd0 0x558072e83d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.103+0000 7fbf48d8f640 1 -- 172.21.15.112:0/2143256633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e88d80 msgr2=0x558072e8b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.103+0000 7fbf48d8f640 1 --2- 172.21.15.112:0/2143256633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e88d80 0x558072e8b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.103+0000 7fbf48d8f640 1 -- 172.21.15.112:0/2143256633 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558072bbc7d0 con 0x558072e79170 2024-03-16T09:16:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.103+0000 7fbf49590640 1 --2- 172.21.15.112:0/2143256633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e88d80 0x558072e8b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.104+0000 7fbf48d8f640 1 --2- 172.21.15.112:0/2143256633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072e79170 0x558072e83810 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbf380045d0 tx=0x7fbf38004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1abedf326763b92f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.104+0000 7fbf23fff640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-16T09:16:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.104+0000 7fbf23fff640 1 -- 172.21.15.112:0/2143256633 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3802a060 con 0x558072e79170 2024-03-16T09:16:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.104+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/2143256633 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558072deba40 con 0x558072e79170 2024-03-16T09:16:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.104+0000 7fbf23fff640 1 -- 172.21.15.112:0/2143256633 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fbf3801c710 con 0x558072e79170 2024-03-16T09:16:47.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.104+0000 7fbf23fff640 1 -- 172.21.15.112:0/2143256633 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fbf3801c910 con 0x558072e79170 2024-03-16T09:16:47.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.104+0000 7fbf4dbd9740 10 client.25553 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:47.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.104+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/2143256633 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558072af23f0 con 0x558072e79170 2024-03-16T09:16:47.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.104+0000 7fbf23fff640 1 -- 172.21.15.112:0/2143256633 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x558072deba40 con 0x558072e79170 2024-03-16T09:16:47.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.104+0000 7fbf23fff640 1 -- 172.21.15.112:0/2143256633 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fbf3801bca0 con 0x558072e79170 2024-03-16T09:16:47.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.104+0000 7fbf4dbd9740 10 client.25553 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:47.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.104+0000 7fbf4dbd9740 10 client.25553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:47.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.104+0000 7fbf4dbd9740 20 client.25553 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:47.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.105+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/2143256633 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x558072e8f6b0 0x558072e91b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.105+0000 7fbf4dbd9740 4 client.25553 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:16:47.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.105+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/2143256633 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558072de5d90 con 0x558072e8f6b0 2024-03-16T09:16:47.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.105+0000 7fbf43fff640 1 --2- 172.21.15.112:0/2143256633 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x558072e8f6b0 0x558072e91b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.106+0000 7fbf43fff640 1 --2- 172.21.15.112:0/2143256633 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x558072e8f6b0 0x558072e91b20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5280ae2632512435 server_cookie=e46e931c1d41968b in_seq=0 out_seq=0 2024-03-16T09:16:47.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.106+0000 7fbf23fff640 10 client.25553 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:47.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.106+0000 7fbf23fff640 1 -- 172.21.15.112:0/2143256633 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558072de5d90 con 0x558072e8f6b0 2024-03-16T09:16:47.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.106+0000 7fbf23fff640 10 client.25553 handle_command_reply: tid=0 2024-03-16T09:16:47.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.106+0000 7fbf4dbd9740 1 client.25553 shutdown 2024-03-16T09:16:47.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.107+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/2143256633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072e79170 msgr2=0x558072e83810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:47.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.107+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/2143256633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072e79170 0x558072e83810 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbf380045d0 tx=0x7fbf38004730 comp rx=0 tx=0).stop 2024-03-16T09:16:47.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.107+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/2143256633 shutdown_connections 2024-03-16T09:16:47.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.107+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/2143256633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e88d80 0x558072e8b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.107+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/2143256633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e7ecd0 0x558072e83d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.107+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/2143256633 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x558072e8f6b0 0x558072e91b20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.107+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/2143256633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072e79170 0x558072e83810 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.107+0000 7fbf23fff640 0 client.25553 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:47.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.107+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/2143256633 >> 172.21.15.112:0/2143256633 conn(0x558072e746e0 msgr2=0x558072e73900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:47.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.107+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/2143256633 shutdown_connections 2024-03-16T09:16:47.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.107+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/2143256633 wait complete. 2024-03-16T09:16:47.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.107+0000 7fbf4dbd9740 20 client.25553 trim_cache size 0 max 16384 2024-03-16T09:16:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.140+0000 7fbf4dbd9740 1 Processor -- start 2024-03-16T09:16:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.141+0000 7fbf4dbd9740 1 -- start start 2024-03-16T09:16:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.141+0000 7fbf4dbd9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072e79170 0x558072e83cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.141+0000 7fbf4dbd9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e7ecd0 0x558072e78050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.141+0000 7fbf4dbd9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e88d80 0x558072e963e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.141+0000 7fbf4dbd9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbf441ad6d0 con 0x558072e7ecd0 2024-03-16T09:16:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.141+0000 7fbf4dbd9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbf441aafa0 con 0x558072e79170 2024-03-16T09:16:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.141+0000 7fbf4dbd9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbf441a8df0 con 0x558072e88d80 2024-03-16T09:16:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.141+0000 7fbf43fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e7ecd0 0x558072e78050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.141+0000 7fbf43fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e7ecd0 0x558072e78050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40992/0 (socket says 172.21.15.112:40992) 2024-03-16T09:16:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.141+0000 7fbf48d8f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072e79170 0x558072e83cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.141+0000 7fbf43fff640 1 -- 172.21.15.112:0/3130582596 learned_addr learned my addr 172.21.15.112:0/3130582596 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:47.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.141+0000 7fbf49590640 1 --2- 172.21.15.112:0/3130582596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e88d80 0x558072e963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.141+0000 7fbf43fff640 1 -- 172.21.15.112:0/3130582596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e88d80 msgr2=0x558072e963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:47.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.141+0000 7fbf43fff640 1 --2- 172.21.15.112:0/3130582596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e88d80 0x558072e963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.141+0000 7fbf43fff640 1 -- 172.21.15.112:0/3130582596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072e79170 msgr2=0x558072e83cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:47.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.141+0000 7fbf43fff640 1 --2- 172.21.15.112:0/3130582596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072e79170 0x558072e83cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.141+0000 7fbf43fff640 1 -- 172.21.15.112:0/3130582596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558072e91e90 con 0x558072e7ecd0 2024-03-16T09:16:47.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.141+0000 7fbf48d8f640 1 --2- 172.21.15.112:0/3130582596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072e79170 0x558072e83cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:47.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.142+0000 7fbf43fff640 1 --2- 172.21.15.112:0/3130582596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e7ecd0 0x558072e78050 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbf3009b420 tx=0x7fbf30099c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6b0647052bc73 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:47.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.142+0000 7fbf2e7fc640 1 -- 172.21.15.112:0/3130582596 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf300b4040 con 0x558072e7ecd0 2024-03-16T09:16:47.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.142+0000 7fbf2e7fc640 1 -- 172.21.15.112:0/3130582596 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbf30003ba0 con 0x558072e7ecd0 2024-03-16T09:16:47.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.142+0000 7fbf2e7fc640 1 -- 172.21.15.112:0/3130582596 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3009add0 con 0x558072e7ecd0 2024-03-16T09:16:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.143+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/3130582596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e7ecd0 msgr2=0x558072e78050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.143+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/3130582596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e7ecd0 0x558072e78050 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbf3009b420 tx=0x7fbf30099c30 comp rx=0 tx=0).stop 2024-03-16T09:16:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.143+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/3130582596 shutdown_connections 2024-03-16T09:16:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.143+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/3130582596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e88d80 0x558072e963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.143+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/3130582596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e7ecd0 0x558072e78050 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.143+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/3130582596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072e79170 0x558072e83cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.143+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/3130582596 >> 172.21.15.112:0/3130582596 conn(0x558072e746e0 msgr2=0x558072e77940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.143+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/3130582596 shutdown_connections 2024-03-16T09:16:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.143+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/3130582596 wait complete. 2024-03-16T09:16:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.143+0000 7fbf4dbd9740 1 -- start start 2024-03-16T09:16:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.143+0000 7fbf4dbd9740 1 Processor -- start 2024-03-16T09:16:47.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.144+0000 7fbf4dbd9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e79170 0x558072ea24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.144+0000 7fbf4dbd9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e88d80 0x558072ea2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.144+0000 7fbf4dbd9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072ea0b00 0x558072ea0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.144+0000 7fbf4dbd9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbf441a9460 con 0x558072e79170 2024-03-16T09:16:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.144+0000 7fbf48d8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e79170 0x558072ea24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.144+0000 7fbf48d8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e79170 0x558072ea24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41004/0 (socket says 172.21.15.112:41004) 2024-03-16T09:16:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.144+0000 7fbf48d8f640 1 -- 172.21.15.112:0/2794085458 learned_addr learned my addr 172.21.15.112:0/2794085458 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.144+0000 7fbf43fff640 1 --2- 172.21.15.112:0/2794085458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e88d80 0x558072ea2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.144+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/2794085458 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbf441aa8d0 con 0x558072ea0b00 2024-03-16T09:16:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.144+0000 7fbf49590640 1 --2- 172.21.15.112:0/2794085458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072ea0b00 0x558072ea0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.144+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/2794085458 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbf441a9b30 con 0x558072e88d80 2024-03-16T09:16:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.145+0000 7fbf4dbd9740 10 client.0 _ll_register_callbacks cb 0x7fbf38012e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.145+0000 7fbf43fff640 1 -- 172.21.15.112:0/2794085458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072ea0b00 msgr2=0x558072ea0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:47.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.145+0000 7fbf43fff640 1 --2- 172.21.15.112:0/2794085458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072ea0b00 0x558072ea0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.145+0000 7fbf43fff640 1 -- 172.21.15.112:0/2794085458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e79170 msgr2=0x558072ea24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.145+0000 7fbf43fff640 1 --2- 172.21.15.112:0/2794085458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e79170 0x558072ea24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.145+0000 7fbf43fff640 1 -- 172.21.15.112:0/2794085458 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558072ab20f0 con 0x558072e88d80 2024-03-16T09:16:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.145+0000 7fbf48d8f640 1 --2- 172.21.15.112:0/2794085458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e79170 0x558072ea24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.145+0000 7fbf43fff640 1 --2- 172.21.15.112:0/2794085458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e88d80 0x558072ea2a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x558072e7ea90 tx=0x7fbf30003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3c5a09b7852b4a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.145+0000 7fbf23fff640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T09:16:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.145+0000 7fbf23fff640 1 -- 172.21.15.112:0/2794085458 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3009ad70 con 0x558072e88d80 2024-03-16T09:16:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.145+0000 7fbf23fff640 1 -- 172.21.15.112:0/2794085458 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fbf300afb00 con 0x558072e88d80 2024-03-16T09:16:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.145+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/2794085458 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf30003ba0 con 0x558072e88d80 2024-03-16T09:16:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.145+0000 7fbf23fff640 1 -- 172.21.15.112:0/2794085458 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fbf3009b720 con 0x558072e88d80 2024-03-16T09:16:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.146+0000 7fbf4dbd9740 10 client.25567 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.146+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/2794085458 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558072da59d0 con 0x558072e88d80 2024-03-16T09:16:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.146+0000 7fbf23fff640 1 -- 172.21.15.112:0/2794085458 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbf30003ba0 con 0x558072e88d80 2024-03-16T09:16:47.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.146+0000 7fbf23fff640 1 -- 172.21.15.112:0/2794085458 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fbf30003fc0 con 0x558072e88d80 2024-03-16T09:16:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.146+0000 7fbf4dbd9740 10 client.25567 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.146+0000 7fbf4dbd9740 10 client.25567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.146+0000 7fbf4dbd9740 20 client.25567 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.146+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/2794085458 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x558072ef6200 0x558072ef85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.146+0000 7fbf4dbd9740 4 client.25567 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T09:16:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.146+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/2794085458 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558072d63540 con 0x558072ef6200 2024-03-16T09:16:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.146+0000 7fbf48d8f640 1 --2- 172.21.15.112:0/2794085458 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x558072ef6200 0x558072ef85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.147+0000 7fbf23fff640 10 client.25567 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.147+0000 7fbf48d8f640 1 --2- 172.21.15.112:0/2794085458 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x558072ef6200 0x558072ef85f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5d42941e0741ee7 server_cookie=5622724b86730b46 in_seq=0 out_seq=0 2024-03-16T09:16:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.147+0000 7fbf23fff640 1 -- 172.21.15.112:0/2794085458 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558072d63540 con 0x558072ef6200 2024-03-16T09:16:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.147+0000 7fbf23fff640 10 client.25567 handle_command_reply: tid=0 2024-03-16T09:16:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.147+0000 7fbf4dbd9740 1 client.25567 shutdown 2024-03-16T09:16:47.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.147+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/2794085458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e88d80 msgr2=0x558072ea2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.147+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/2794085458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e88d80 0x558072ea2a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x558072e7ea90 tx=0x7fbf30003a90 comp rx=0 tx=0).stop 2024-03-16T09:16:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.147+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/2794085458 shutdown_connections 2024-03-16T09:16:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.147+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/2794085458 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x558072ef6200 0x558072ef85f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.147+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/2794085458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558072ea0b00 0x558072ea0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.147+0000 7fbf23fff640 0 client.25567 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.147+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/2794085458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558072e88d80 0x558072ea2a30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.147+0000 7fbf4dbd9740 1 --2- 172.21.15.112:0/2794085458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558072e79170 0x558072ea24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.147+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/2794085458 >> 172.21.15.112:0/2794085458 conn(0x558072e746e0 msgr2=0x558072e781d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.147+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/2794085458 shutdown_connections 2024-03-16T09:16:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.147+0000 7fbf4dbd9740 1 -- 172.21.15.112:0/2794085458 wait complete. 2024-03-16T09:16:47.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.147+0000 7fbf4dbd9740 20 client.25567 trim_cache size 0 max 16384 2024-03-16T09:16:47.156 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:16:47.156 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-16T09:16:47.156 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-03-16T09:16:47.156 INFO:teuthology.orchestra.run.smithi112.stdout: "a04fa4f2-b33b-4628-9fc1-5899669d0ecd": { 2024-03-16T09:16:47.156 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-03-16T09:16:47.156 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "a04fa4f2-b33b-4628-9fc1-5899669d0ecd", 2024-03-16T09:16:47.156 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-03-16T09:16:47.156 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:16:47.156 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:16:47.156 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:16:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.148+0000 7fbf4aff3640 1 -- 172.21.15.112:0/2223425890 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fbf340a2910 msgr2=0x7fbf340a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.148+0000 7fbf4aff3640 1 --2- 172.21.15.112:0/2223425890 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fbf340a2910 0x7fbf340a4dd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbf4419b0f0 tx=0x7fbf38005f90 comp rx=0 tx=0).stop 2024-03-16T09:16:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.148+0000 7fbf4aff3640 1 -- 172.21.15.112:0/2223425890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf44104c50 msgr2=0x7fbf4419a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.148+0000 7fbf4aff3640 1 --2- 172.21.15.112:0/2223425890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf44104c50 0x7fbf4419a160 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000e9a0 tx=0x7fbf3000ee60 comp rx=0 tx=0).stop 2024-03-16T09:16:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.148+0000 7fbf4aff3640 1 -- 172.21.15.112:0/2223425890 shutdown_connections 2024-03-16T09:16:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.148+0000 7fbf4aff3640 1 --2- 172.21.15.112:0/2223425890 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fbf340a2910 0x7fbf340a4dd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.148+0000 7fbf4aff3640 1 --2- 172.21.15.112:0/2223425890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf44107580 0x7fbf441a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.148+0000 7fbf4aff3640 1 --2- 172.21.15.112:0/2223425890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf44104c50 0x7fbf4419a160 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.148+0000 7fbf4aff3640 1 --2- 172.21.15.112:0/2223425890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf44102320 0x7fbf44199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.148+0000 7fbf4aff3640 1 -- 172.21.15.112:0/2223425890 >> 172.21.15.112:0/2223425890 conn(0x7fbf440fbc70 msgr2=0x7fbf44100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.149+0000 7fbf4aff3640 1 -- 172.21.15.112:0/2223425890 shutdown_connections 2024-03-16T09:16:47.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:16:47.149+0000 7fbf4aff3640 1 -- 172.21.15.112:0/2223425890 wait complete. 2024-03-16T09:16:47.164 INFO:tasks.cephfs.filesystem:scrub status for tag:a04fa4f2-b33b-4628-9fc1-5899669d0ecd - {'path': '/', 'tag': 'a04fa4f2-b33b-4628-9fc1-5899669d0ecd', 'options': 'recursive,force'} 2024-03-16T09:16:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.742+0000 7fd9e7d10640 1 -- 172.21.15.112:0/2494037705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e0101840 msgr2=0x7fd9e0101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.742+0000 7fd9e7d10640 1 --2- 172.21.15.112:0/2494037705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e0101840 0x7fd9e0101c60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009900 tx=0x7fd9dc02bc30 comp rx=0 tx=0).stop 2024-03-16T09:16:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.742+0000 7fd9e7d10640 1 -- 172.21.15.112:0/2494037705 shutdown_connections 2024-03-16T09:16:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.742+0000 7fd9e7d10640 1 --2- 172.21.15.112:0/2494037705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9e01033f0 0x7fd9e0109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.742+0000 7fd9e7d10640 1 --2- 172.21.15.112:0/2494037705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9e0102a30 0x7fd9e0102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.742+0000 7fd9e7d10640 1 --2- 172.21.15.112:0/2494037705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e0101840 0x7fd9e0101c60 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.742+0000 7fd9e7d10640 1 -- 172.21.15.112:0/2494037705 >> 172.21.15.112:0/2494037705 conn(0x7fd9e00fcdb0 msgr2=0x7fd9e00ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.742+0000 7fd9e7d10640 1 -- 172.21.15.112:0/2494037705 shutdown_connections 2024-03-16T09:16:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.742+0000 7fd9e7d10640 1 -- 172.21.15.112:0/2494037705 wait complete. 2024-03-16T09:16:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.743+0000 7fd9e7d10640 1 Processor -- start 2024-03-16T09:16:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.743+0000 7fd9e7d10640 1 -- start start 2024-03-16T09:16:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.743+0000 7fd9e7d10640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9e0101840 0x7fd9e0197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.743+0000 7fd9e7d10640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e0102a30 0x7fd9e01980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.743+0000 7fd9e7d10640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9e01033f0 0x7fd9e019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.743+0000 7fd9e7d10640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e010d5a0 con 0x7fd9e0102a30 2024-03-16T09:16:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.743+0000 7fd9e5aac640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9e0101840 0x7fd9e0197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.743+0000 7fd9e5aac640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9e0101840 0x7fd9e0197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:56796/0 (socket says 172.21.15.112:56796) 2024-03-16T09:16:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.743+0000 7fd9e5aac640 1 -- 172.21.15.112:0/685061040 learned_addr learned my addr 172.21.15.112:0/685061040 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.743+0000 7fd9e62ad640 1 --2- 172.21.15.112:0/685061040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9e01033f0 0x7fd9e019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.743+0000 7fd9e52ab640 1 --2- 172.21.15.112:0/685061040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e0102a30 0x7fd9e01980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.743+0000 7fd9e7d10640 1 -- 172.21.15.112:0/685061040 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e010d420 con 0x7fd9e01033f0 2024-03-16T09:16:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.743+0000 7fd9e7d10640 1 -- 172.21.15.112:0/685061040 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e010d720 con 0x7fd9e0101840 2024-03-16T09:16:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.743+0000 7fd9e5aac640 1 -- 172.21.15.112:0/685061040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9e01033f0 msgr2=0x7fd9e019f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.743+0000 7fd9e5aac640 1 --2- 172.21.15.112:0/685061040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9e01033f0 0x7fd9e019f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.744+0000 7fd9e5aac640 1 -- 172.21.15.112:0/685061040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e0102a30 msgr2=0x7fd9e01980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.744+0000 7fd9e5aac640 1 --2- 172.21.15.112:0/685061040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e0102a30 0x7fd9e01980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.744+0000 7fd9e5aac640 1 -- 172.21.15.112:0/685061040 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e019f7b0 con 0x7fd9e0101840 2024-03-16T09:16:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.744+0000 7fd9e62ad640 1 --2- 172.21.15.112:0/685061040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9e01033f0 0x7fd9e019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:16:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.744+0000 7fd9e52ab640 1 --2- 172.21.15.112:0/685061040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e0102a30 0x7fd9e01980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:16:47.748 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.744+0000 7fd9e5aac640 1 --2- 172.21.15.112:0/685061040 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9e0101840 0x7fd9e0197b90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc000c00 tx=0x7fd9dc004a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.744+0000 7fd9d6ffd640 1 -- 172.21.15.112:0/685061040 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc043020 con 0x7fd9e0101840 2024-03-16T09:16:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.744+0000 7fd9d6ffd640 1 -- 172.21.15.112:0/685061040 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd9dc004250 con 0x7fd9e0101840 2024-03-16T09:16:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.744+0000 7fd9d6ffd640 1 -- 172.21.15.112:0/685061040 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc003b60 con 0x7fd9e0101840 2024-03-16T09:16:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.744+0000 7fd9e7d10640 1 -- 172.21.15.112:0/685061040 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e0073a10 con 0x7fd9e0101840 2024-03-16T09:16:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.744+0000 7fd9e7d10640 1 -- 172.21.15.112:0/685061040 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e0073f10 con 0x7fd9e0101840 2024-03-16T09:16:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.747+0000 7fd9d6ffd640 1 -- 172.21.15.112:0/685061040 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7fd9dc046020 con 0x7fd9e0101840 2024-03-16T09:16:47.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.748+0000 7fd9ea9da740 1 Processor -- start 2024-03-16T09:16:47.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.748+0000 7fd9ea9da740 1 -- start start 2024-03-16T09:16:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.748+0000 7fd9ea9da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d1985d340 0x560d1985d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.748+0000 7fd9ea9da740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1985dd30 0x560d19862830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.748+0000 7fd9ea9da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d19862ea0 0x560d19865330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.748+0000 7fd9ea9da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560d197c2a40 con 0x560d19862ea0 2024-03-16T09:16:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.748+0000 7fd9ea9da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560d1976b920 con 0x560d1985d340 2024-03-16T09:16:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.748+0000 7fd9ea9da740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560d197beae0 con 0x560d1985dd30 2024-03-16T09:16:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.748+0000 7fd9e62ad640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d1985d340 0x560d1985d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.748+0000 7fd9e5aac640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1985dd30 0x560d19862830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.748+0000 7fd9e62ad640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d1985d340 0x560d1985d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:35870/0 (socket says 172.21.15.112:35870) 2024-03-16T09:16:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.748+0000 7fd9e62ad640 1 -- 172.21.15.112:0/2867371316 learned_addr learned my addr 172.21.15.112:0/2867371316 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.748+0000 7fd9e52ab640 1 --2- 172.21.15.112:0/2867371316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d19862ea0 0x560d19865330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.749+0000 7fd9d6ffd640 1 --2- 172.21.15.112:0/685061040 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd9cc0a6dd0 0x7fd9cc0a9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.749+0000 7fd9e62ad640 1 -- 172.21.15.112:0/2867371316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1985dd30 msgr2=0x560d19862830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:47.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.749+0000 7fd9e62ad640 1 --2- 172.21.15.112:0/2867371316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1985dd30 0x560d19862830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.749+0000 7fd9e62ad640 1 -- 172.21.15.112:0/2867371316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d19862ea0 msgr2=0x560d19865330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.749+0000 7fd9d6ffd640 1 -- 172.21.15.112:0/685061040 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd9dc0f0c10 con 0x7fd9e0101840 2024-03-16T09:16:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.749+0000 7fd9e62ad640 1 --2- 172.21.15.112:0/2867371316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d19862ea0 0x560d19865330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.749+0000 7fd9e62ad640 1 -- 172.21.15.112:0/2867371316 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d19783460 con 0x560d1985d340 2024-03-16T09:16:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.749+0000 7fd9e5aac640 1 --2- 172.21.15.112:0/2867371316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1985dd30 0x560d19862830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.749+0000 7fd9e52ab640 1 --2- 172.21.15.112:0/2867371316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d19862ea0 0x560d19865330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.749+0000 7fd9e62ad640 1 --2- 172.21.15.112:0/685061040 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd9cc0a6dd0 0x7fd9cc0a9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.749+0000 7fd9e62ad640 1 --2- 172.21.15.112:0/2867371316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d1985d340 0x560d1985d760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009f60 tx=0x7fd9d8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91290b57f6adafe1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.750+0000 7fd9cbfff640 1 -- 172.21.15.112:0/2867371316 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8029070 con 0x560d1985d340 2024-03-16T09:16:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.750+0000 7fd9cbfff640 1 -- 172.21.15.112:0/2867371316 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd9d8018e40 con 0x560d1985d340 2024-03-16T09:16:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.750+0000 7fd9cbfff640 1 -- 172.21.15.112:0/2867371316 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80217a0 con 0x560d1985d340 2024-03-16T09:16:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.750+0000 7fd9e62ad640 1 --2- 172.21.15.112:0/685061040 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd9cc0a6dd0 0x7fd9cc0a9290 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8026000 tx=0x7fd9d8027000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.750+0000 7fd9ea9da740 1 -- 172.21.15.112:0/2867371316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d1985d340 msgr2=0x560d1985d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.751+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/2867371316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d1985d340 0x560d1985d760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009f60 tx=0x7fd9d8009f90 comp rx=0 tx=0).stop 2024-03-16T09:16:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.751+0000 7fd9ea9da740 1 -- 172.21.15.112:0/2867371316 shutdown_connections 2024-03-16T09:16:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.751+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/2867371316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d19862ea0 0x560d19865330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.751+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/2867371316 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1985dd30 0x560d19862830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.751+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/2867371316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d1985d340 0x560d1985d760 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.751+0000 7fd9ea9da740 1 -- 172.21.15.112:0/2867371316 >> 172.21.15.112:0/2867371316 conn(0x560d198588b0 msgr2=0x560d19858cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:47.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.751+0000 7fd9ea9da740 1 -- 172.21.15.112:0/2867371316 shutdown_connections 2024-03-16T09:16:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.751+0000 7fd9ea9da740 1 -- 172.21.15.112:0/2867371316 wait complete. 2024-03-16T09:16:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.751+0000 7fd9ea9da740 1 -- start start 2024-03-16T09:16:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.751+0000 7fd9ea9da740 1 Processor -- start 2024-03-16T09:16:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.752+0000 7fd9ea9da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1985d340 0x560d19867670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.752+0000 7fd9ea9da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d1985dd30 0x560d19867bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.752+0000 7fd9ea9da740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d19862ea0 0x560d1986ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.752+0000 7fd9ea9da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560d19865e50 con 0x560d1985d340 2024-03-16T09:16:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.752+0000 7fd9ea9da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560d1977ed70 con 0x560d1985dd30 2024-03-16T09:16:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.752+0000 7fd9e5aac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1985d340 0x560d19867670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.752+0000 7fd9e5aac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1985d340 0x560d19867670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41030/0 (socket says 172.21.15.112:41030) 2024-03-16T09:16:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.752+0000 7fd9ea9da740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560d19865fd0 con 0x560d19862ea0 2024-03-16T09:16:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.752+0000 7fd9e5aac640 1 -- 172.21.15.112:0/3891584971 learned_addr learned my addr 172.21.15.112:0/3891584971 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.752+0000 7fd9e62ad640 1 --2- 172.21.15.112:0/3891584971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d19862ea0 0x560d1986ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.752+0000 7fd9ea9da740 10 client.0 _ll_register_callbacks cb 0x560d194d6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:47.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.752+0000 7fd9e52ab640 1 --2- 172.21.15.112:0/3891584971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d1985dd30 0x560d19867bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.752+0000 7fd9e62ad640 1 -- 172.21.15.112:0/3891584971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d1985dd30 msgr2=0x560d19867bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.752+0000 7fd9e62ad640 1 --2- 172.21.15.112:0/3891584971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d1985dd30 0x560d19867bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.752+0000 7fd9e62ad640 1 -- 172.21.15.112:0/3891584971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1985d340 msgr2=0x560d19867670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.752+0000 7fd9e62ad640 1 --2- 172.21.15.112:0/3891584971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1985d340 0x560d19867670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.752+0000 7fd9e62ad640 1 -- 172.21.15.112:0/3891584971 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d194d7210 con 0x560d19862ea0 2024-03-16T09:16:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.752+0000 7fd9e52ab640 1 --2- 172.21.15.112:0/3891584971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d1985dd30 0x560d19867bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:16:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.752+0000 7fd9e5aac640 1 --2- 172.21.15.112:0/3891584971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1985d340 0x560d19867670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.753+0000 7fd9e62ad640 1 --2- 172.21.15.112:0/3891584971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d19862ea0 0x560d1986ebf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800fc80 tx=0x7fd9d8009d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80844eaa5a820a82 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.753+0000 7fd9c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T09:16:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.753+0000 7fd9c97fa640 1 -- 172.21.15.112:0/3891584971 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8029050 con 0x560d19862ea0 2024-03-16T09:16:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.753+0000 7fd9c97fa640 1 -- 172.21.15.112:0/3891584971 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd9d8004c00 con 0x560d19862ea0 2024-03-16T09:16:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.753+0000 7fd9ea9da740 1 -- 172.21.15.112:0/3891584971 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560d197cff80 con 0x560d19862ea0 2024-03-16T09:16:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.753+0000 7fd9c97fa640 1 -- 172.21.15.112:0/3891584971 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd9d8021750 con 0x560d19862ea0 2024-03-16T09:16:47.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.753+0000 7fd9ea9da740 10 client.25579 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.753+0000 7fd9ea9da740 1 -- 172.21.15.112:0/3891584971 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560d195a3620 con 0x560d19862ea0 2024-03-16T09:16:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.753+0000 7fd9c97fa640 1 -- 172.21.15.112:0/3891584971 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x560d197cff80 con 0x560d19862ea0 2024-03-16T09:16:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.753+0000 7fd9c97fa640 1 -- 172.21.15.112:0/3891584971 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd9d8022a90 con 0x560d19862ea0 2024-03-16T09:16:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.754+0000 7fd9ea9da740 10 client.25579 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.754+0000 7fd9ea9da740 10 client.25579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.754+0000 7fd9ea9da740 20 client.25579 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.754+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/3891584971 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560d198732b0 0x560d198756a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.754+0000 7fd9ea9da740 4 client.25579 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:16:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.754+0000 7fd9ea9da740 1 -- 172.21.15.112:0/3891584971 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d197ca2d0 con 0x560d198732b0 2024-03-16T09:16:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.754+0000 7fd9e5aac640 1 --2- 172.21.15.112:0/3891584971 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560d198732b0 0x560d198756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.754+0000 7fd9e5aac640 1 --2- 172.21.15.112:0/3891584971 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560d198732b0 0x560d198756a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58741770798eb959 server_cookie=6e7ae4f852b312e7 in_seq=0 out_seq=0 2024-03-16T09:16:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.754+0000 7fd9c97fa640 10 client.25579 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:47.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.755+0000 7fd9c97fa640 1 -- 172.21.15.112:0/3891584971 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560d197ca2d0 con 0x560d198732b0 2024-03-16T09:16:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.755+0000 7fd9c97fa640 10 client.25579 handle_command_reply: tid=0 2024-03-16T09:16:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.755+0000 7fd9ea9da740 1 client.25579 shutdown 2024-03-16T09:16:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.755+0000 7fd9ea9da740 1 -- 172.21.15.112:0/3891584971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d19862ea0 msgr2=0x560d1986ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.755+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/3891584971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d19862ea0 0x560d1986ebf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800fc80 tx=0x7fd9d8009d50 comp rx=0 tx=0).stop 2024-03-16T09:16:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.755+0000 7fd9ea9da740 1 -- 172.21.15.112:0/3891584971 shutdown_connections 2024-03-16T09:16:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.755+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/3891584971 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d19862ea0 0x560d1986ebf0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.755+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/3891584971 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560d198732b0 0x560d198756a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.755+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/3891584971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d1985dd30 0x560d19867bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.755+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/3891584971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1985d340 0x560d19867670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.755+0000 7fd9c97fa640 0 client.25579 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.755+0000 7fd9ea9da740 1 -- 172.21.15.112:0/3891584971 >> 172.21.15.112:0/3891584971 conn(0x560d198588b0 msgr2=0x560d19857bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.755+0000 7fd9ea9da740 1 -- 172.21.15.112:0/3891584971 shutdown_connections 2024-03-16T09:16:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.756+0000 7fd9ea9da740 1 -- 172.21.15.112:0/3891584971 wait complete. 2024-03-16T09:16:47.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.756+0000 7fd9ea9da740 20 client.25579 trim_cache size 0 max 16384 2024-03-16T09:16:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.784+0000 7fd9ea9da740 1 Processor -- start 2024-03-16T09:16:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.784+0000 7fd9ea9da740 1 -- start start 2024-03-16T09:16:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.784+0000 7fd9ea9da740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1985d340 0x560d1985f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.784+0000 7fd9ea9da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1985dd30 0x560d1986ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.784+0000 7fd9ea9da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d19862ea0 0x560d1985f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.784+0000 7fd9ea9da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01a93e0 con 0x560d1985dd30 2024-03-16T09:16:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.784+0000 7fd9ea9da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01a6cb0 con 0x560d19862ea0 2024-03-16T09:16:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.784+0000 7fd9ea9da740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01a4b00 con 0x560d1985d340 2024-03-16T09:16:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.784+0000 7fd9e62ad640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d19862ea0 0x560d1985f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.784+0000 7fd9e52ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1985dd30 0x560d1986ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.784+0000 7fd9e5aac640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1985d340 0x560d1985f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.784+0000 7fd9e62ad640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d19862ea0 0x560d1985f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:35878/0 (socket says 172.21.15.112:35878) 2024-03-16T09:16:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.784+0000 7fd9e62ad640 1 -- 172.21.15.112:0/1277727154 learned_addr learned my addr 172.21.15.112:0/1277727154 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.784+0000 7fd9e62ad640 1 -- 172.21.15.112:0/1277727154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1985d340 msgr2=0x560d1985f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.785+0000 7fd9e62ad640 1 --2- 172.21.15.112:0/1277727154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1985d340 0x560d1985f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.785+0000 7fd9e62ad640 1 -- 172.21.15.112:0/1277727154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1985dd30 msgr2=0x560d1986ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.785+0000 7fd9e62ad640 1 --2- 172.21.15.112:0/1277727154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1985dd30 0x560d1986ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.785+0000 7fd9e62ad640 1 -- 172.21.15.112:0/1277727154 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d1985e110 con 0x560d19862ea0 2024-03-16T09:16:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.785+0000 7fd9e5aac640 1 --2- 172.21.15.112:0/1277727154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1985d340 0x560d1985f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.785+0000 7fd9e52ab640 1 --2- 172.21.15.112:0/1277727154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1985dd30 0x560d1986ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.785+0000 7fd9e62ad640 1 --2- 172.21.15.112:0/1277727154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d19862ea0 0x560d1985f590 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009bc0 tx=0x7fd9d8009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92099fe523a8d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.785+0000 7fd9cb7fe640 1 -- 172.21.15.112:0/1277727154 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8029050 con 0x560d19862ea0 2024-03-16T09:16:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.785+0000 7fd9cb7fe640 1 -- 172.21.15.112:0/1277727154 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd9d80057d0 con 0x560d19862ea0 2024-03-16T09:16:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.785+0000 7fd9cb7fe640 1 -- 172.21.15.112:0/1277727154 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8004c60 con 0x560d19862ea0 2024-03-16T09:16:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.786+0000 7fd9ea9da740 1 -- 172.21.15.112:0/1277727154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d19862ea0 msgr2=0x560d1985f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.786+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/1277727154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d19862ea0 0x560d1985f590 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009bc0 tx=0x7fd9d8009500 comp rx=0 tx=0).stop 2024-03-16T09:16:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.786+0000 7fd9ea9da740 1 -- 172.21.15.112:0/1277727154 shutdown_connections 2024-03-16T09:16:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.786+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/1277727154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d19862ea0 0x560d1985f590 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.786+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/1277727154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d1985dd30 0x560d1986ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.786+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/1277727154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1985d340 0x560d1985f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.786+0000 7fd9ea9da740 1 -- 172.21.15.112:0/1277727154 >> 172.21.15.112:0/1277727154 conn(0x560d198588b0 msgr2=0x560d19858c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.786+0000 7fd9ea9da740 1 -- 172.21.15.112:0/1277727154 shutdown_connections 2024-03-16T09:16:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.786+0000 7fd9ea9da740 1 -- 172.21.15.112:0/1277727154 wait complete. 2024-03-16T09:16:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.786+0000 7fd9ea9da740 1 -- start start 2024-03-16T09:16:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.787+0000 7fd9ea9da740 1 Processor -- start 2024-03-16T09:16:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.787+0000 7fd9ea9da740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1985d340 0x560d1986bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.787+0000 7fd9ea9da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d1985dd30 0x560d1986c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.787+0000 7fd9ea9da740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d19862ea0 0x560d1986d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.787+0000 7fd9ea9da740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01a5170 con 0x560d19862ea0 2024-03-16T09:16:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.787+0000 7fd9ea9da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01a65e0 con 0x560d1985dd30 2024-03-16T09:16:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.787+0000 7fd9ea9da740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e01a5840 con 0x560d1985d340 2024-03-16T09:16:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.787+0000 7fd9e62ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d19862ea0 0x560d1986d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.787+0000 7fd9e62ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d19862ea0 0x560d1986d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41044/0 (socket says 172.21.15.112:41044) 2024-03-16T09:16:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.787+0000 7fd9e62ad640 1 -- 172.21.15.112:0/257499445 learned_addr learned my addr 172.21.15.112:0/257499445 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:16:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.787+0000 7fd9e5aac640 1 --2- 172.21.15.112:0/257499445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1985d340 0x560d1986bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.787+0000 7fd9e52ab640 1 --2- 172.21.15.112:0/257499445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d1985dd30 0x560d1986c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.787+0000 7fd9ea9da740 10 client.0 _ll_register_callbacks cb 0x7fd9d8027070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:16:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.787+0000 7fd9e5aac640 1 -- 172.21.15.112:0/257499445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d1985dd30 msgr2=0x560d1986c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.787+0000 7fd9e5aac640 1 --2- 172.21.15.112:0/257499445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d1985dd30 0x560d1986c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.787+0000 7fd9e5aac640 1 -- 172.21.15.112:0/257499445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d19862ea0 msgr2=0x560d1986d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:16:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.787+0000 7fd9e5aac640 1 --2- 172.21.15.112:0/257499445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d19862ea0 0x560d1986d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.787+0000 7fd9e5aac640 1 -- 172.21.15.112:0/257499445 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d1986d920 con 0x560d1985d340 2024-03-16T09:16:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.787+0000 7fd9e62ad640 1 --2- 172.21.15.112:0/257499445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d19862ea0 0x560d1986d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:16:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.788+0000 7fd9e5aac640 1 --2- 172.21.15.112:0/257499445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1985d340 0x560d1986bde0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc02c900 tx=0x7fd9dc0a8040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=803bd42f2ee1b1aa server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:16:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.788+0000 7fd9c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T09:16:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.788+0000 7fd9c97fa640 1 -- 172.21.15.112:0/257499445 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc0a6020 con 0x560d1985d340 2024-03-16T09:16:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.788+0000 7fd9c97fa640 1 -- 172.21.15.112:0/257499445 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fd9dc003d00 con 0x560d1985d340 2024-03-16T09:16:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.788+0000 7fd9ea9da740 1 -- 172.21.15.112:0/257499445 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9d80057d0 con 0x560d1985d340 2024-03-16T09:16:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.788+0000 7fd9c97fa640 1 -- 172.21.15.112:0/257499445 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd9dc0bd0b0 con 0x560d1985d340 2024-03-16T09:16:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.788+0000 7fd9ea9da740 10 client.25585 fetch_fsmap learned FSMap version 22 2024-03-16T09:16:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.788+0000 7fd9ea9da740 1 -- 172.21.15.112:0/257499445 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560d1986eca0 con 0x560d1985d340 2024-03-16T09:16:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.788+0000 7fd9c97fa640 1 -- 172.21.15.112:0/257499445 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd9d80057d0 con 0x560d1985d340 2024-03-16T09:16:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.788+0000 7fd9c97fa640 1 -- 172.21.15.112:0/257499445 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd9dc0b9be0 con 0x560d1985d340 2024-03-16T09:16:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.789+0000 7fd9ea9da740 10 client.25585 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:16:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.789+0000 7fd9ea9da740 10 client.25585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:16:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.789+0000 7fd9ea9da740 20 client.25585 populate_metadata read hostname 'smithi112' 2024-03-16T09:16:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.789+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/257499445 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560d198d9b90 0x560d198dbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:16:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.789+0000 7fd9ea9da740 4 client.25585 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T09:16:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.789+0000 7fd9ea9da740 1 -- 172.21.15.112:0/257499445 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d19748770 con 0x560d198d9b90 2024-03-16T09:16:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.789+0000 7fd9e52ab640 1 --2- 172.21.15.112:0/257499445 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560d198d9b90 0x560d198dbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:16:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.790+0000 7fd9c97fa640 10 client.25585 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.790+0000 7fd9e52ab640 1 --2- 172.21.15.112:0/257499445 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560d198d9b90 0x560d198dbfb0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b70665dd2e20bbf server_cookie=1aec7162c65ecce3 in_seq=0 out_seq=0 2024-03-16T09:16:47.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.793+0000 7fd9c97fa640 1 -- 172.21.15.112:0/257499445 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560d19748770 con 0x560d198d9b90 2024-03-16T09:16:47.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.793+0000 7fd9c97fa640 10 client.25585 handle_command_reply: tid=0 2024-03-16T09:16:47.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.793+0000 7fd9ea9da740 1 client.25585 shutdown 2024-03-16T09:16:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.793+0000 7fd9ea9da740 1 -- 172.21.15.112:0/257499445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1985d340 msgr2=0x560d1986bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.793+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/257499445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1985d340 0x560d1986bde0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc02c900 tx=0x7fd9dc0a8040 comp rx=0 tx=0).stop 2024-03-16T09:16:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.793+0000 7fd9ea9da740 1 -- 172.21.15.112:0/257499445 shutdown_connections 2024-03-16T09:16:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.793+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/257499445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560d19862ea0 0x560d1986d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.793+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/257499445 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x560d198d9b90 0x560d198dbfb0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.793+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/257499445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560d1985dd30 0x560d1986c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.793+0000 7fd9ea9da740 1 --2- 172.21.15.112:0/257499445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560d1985d340 0x560d1986bde0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.793+0000 7fd9ea9da740 1 -- 172.21.15.112:0/257499445 >> 172.21.15.112:0/257499445 conn(0x560d198588b0 msgr2=0x560d198675b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.793+0000 7fd9c97fa640 0 client.25585 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:16:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.793+0000 7fd9ea9da740 1 -- 172.21.15.112:0/257499445 shutdown_connections 2024-03-16T09:16:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.793+0000 7fd9ea9da740 1 -- 172.21.15.112:0/257499445 wait complete. 2024-03-16T09:16:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.793+0000 7fd9ea9da740 20 client.25585 trim_cache size 0 max 16384 2024-03-16T09:16:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-03-16T09:16:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-03-16T09:16:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-03-16T09:16:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-03-16T09:16:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.794+0000 7fd9e7d10640 1 -- 172.21.15.112:0/685061040 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd9cc0a6dd0 msgr2=0x7fd9cc0a9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.794+0000 7fd9e7d10640 1 --2- 172.21.15.112:0/685061040 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd9cc0a6dd0 0x7fd9cc0a9290 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8026000 tx=0x7fd9d8027000 comp rx=0 tx=0).stop 2024-03-16T09:16:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.794+0000 7fd9e7d10640 1 -- 172.21.15.112:0/685061040 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9e0101840 msgr2=0x7fd9e0197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:16:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.794+0000 7fd9e7d10640 1 --2- 172.21.15.112:0/685061040 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9e0101840 0x7fd9e0197b90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc000c00 tx=0x7fd9dc004a30 comp rx=0 tx=0).stop 2024-03-16T09:16:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.794+0000 7fd9e7d10640 1 -- 172.21.15.112:0/685061040 shutdown_connections 2024-03-16T09:16:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.794+0000 7fd9e7d10640 1 --2- 172.21.15.112:0/685061040 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd9cc0a6dd0 0x7fd9cc0a9290 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.794+0000 7fd9e7d10640 1 --2- 172.21.15.112:0/685061040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9e01033f0 0x7fd9e019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.794+0000 7fd9e7d10640 1 --2- 172.21.15.112:0/685061040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9e0102a30 0x7fd9e01980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.794+0000 7fd9e7d10640 1 --2- 172.21.15.112:0/685061040 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9e0101840 0x7fd9e0197b90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:16:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.794+0000 7fd9e7d10640 1 -- 172.21.15.112:0/685061040 >> 172.21.15.112:0/685061040 conn(0x7fd9e00fcdb0 msgr2=0x7fd9e0105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:16:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.794+0000 7fd9e7d10640 1 -- 172.21.15.112:0/685061040 shutdown_connections 2024-03-16T09:16:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:16:47.794+0000 7fd9e7d10640 1 -- 172.21.15.112:0/685061040 wait complete. 2024-03-16T09:16:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:48 smithi112 ceph-mon[32307]: pgmap v385: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:16:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:48 smithi112 ceph-mon[32307]: scrub queued for path: / 2024-03-16T09:16:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:48 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:16:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:48 smithi112 ceph-mon[32307]: scrub summary: active paths [/] 2024-03-16T09:16:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:48 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:16:48.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:48 smithi194 ceph-mon[36729]: pgmap v385: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:16:48.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:48 smithi194 ceph-mon[36729]: scrub queued for path: / 2024-03-16T09:16:48.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:48 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:16:48.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:48 smithi194 ceph-mon[36729]: scrub summary: active paths [/] 2024-03-16T09:16:48.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:48 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:16:48.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:48 smithi159 ceph-mon[36739]: pgmap v385: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:16:48.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:48 smithi159 ceph-mon[36739]: scrub queued for path: / 2024-03-16T09:16:48.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:48 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:16:48.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:48 smithi159 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-16T09:16:48.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:48 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:16:50.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:50 smithi112 ceph-mon[32307]: pgmap v386: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:16:50.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:50 smithi194 ceph-mon[36729]: pgmap v386: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:16:50.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:50 smithi159 ceph-mon[36739]: pgmap v386: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-16T09:16:52.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:52 smithi112 ceph-mon[32307]: pgmap v387: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-16T09:16:52.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:52 smithi112 ceph-mon[32307]: scrub summary: idle 2024-03-16T09:16:52.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:52 smithi194 ceph-mon[36729]: pgmap v387: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-16T09:16:52.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:52 smithi194 ceph-mon[36729]: scrub summary: idle 2024-03-16T09:16:52.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:52 smithi159 ceph-mon[36739]: pgmap v387: 97 pgs: 97 active+clean; 668 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-03-16T09:16:52.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:52 smithi159 ceph-mon[36739]: scrub summary: idle 2024-03-16T09:16:54.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:54 smithi112 ceph-mon[32307]: pgmap v388: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-16T09:16:54.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:54 smithi194 ceph-mon[36729]: pgmap v388: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-16T09:16:54.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:54 smithi159 ceph-mon[36739]: pgmap v388: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-16T09:16:56.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:56 smithi112 ceph-mon[32307]: pgmap v389: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:16:56.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:56 smithi194 ceph-mon[36729]: pgmap v389: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:16:56.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:56 smithi159 ceph-mon[36739]: pgmap v389: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:16:58.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:58 smithi112 ceph-mon[32307]: pgmap v390: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T09:16:58.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:58 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:16:58.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:58 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:16:58.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:58 smithi194 ceph-mon[36729]: pgmap v390: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T09:16:58.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:58 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:16:58.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:58 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:16:58.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:58 smithi159 ceph-mon[36739]: pgmap v390: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T09:16:58.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:58 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:16:58.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:58 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:16:59.219 INFO:tasks.workunit.client.0.smithi112.stderr:Updating files: 99% (12199/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-16T09:16:59.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:16:59.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:16:59.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:16:59.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:16:59.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:16:59.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:16:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:16:59.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:59 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:16:59.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:59 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:16:59.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:59 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:16:59.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:59 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:16:59.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:59 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:16:59.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:16:59 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:16:59.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:59 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:16:59.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:59 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:16:59.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:59 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:16:59.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:59 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:16:59.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:59 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:16:59.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:16:59 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:16:59.622 INFO:tasks.workunit.client.0.smithi112.stderr:Note: switching to '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'. 2024-03-16T09:16:59.622 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-03-16T09:16:59.622 INFO:tasks.workunit.client.0.smithi112.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-16T09:16:59.622 INFO:tasks.workunit.client.0.smithi112.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-16T09:16:59.622 INFO:tasks.workunit.client.0.smithi112.stderr:state without impacting any branches by switching back to a branch. 2024-03-16T09:16:59.622 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-03-16T09:16:59.622 INFO:tasks.workunit.client.0.smithi112.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-16T09:16:59.623 INFO:tasks.workunit.client.0.smithi112.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-16T09:16:59.623 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-03-16T09:16:59.623 INFO:tasks.workunit.client.0.smithi112.stderr: git switch -c 2024-03-16T09:16:59.623 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-03-16T09:16:59.623 INFO:tasks.workunit.client.0.smithi112.stderr:Or undo this operation with: 2024-03-16T09:16:59.623 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-03-16T09:16:59.623 INFO:tasks.workunit.client.0.smithi112.stderr: git switch - 2024-03-16T09:16:59.623 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-03-16T09:16:59.623 INFO:tasks.workunit.client.0.smithi112.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-16T09:16:59.623 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-03-16T09:16:59.623 INFO:tasks.workunit.client.0.smithi112.stderr:HEAD is now at 3d78f5ba07b Merge PR #54581 into wip-batrick-testing-20240314.183243 2024-03-16T09:16:59.627 DEBUG:teuthology.orchestra.run.smithi112:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-16T09:16:59.694 INFO:tasks.workunit.client.0.smithi112.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-16T09:16:59.696 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-16T09:16:59.696 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-16T09:17:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:00 smithi112 ceph-mon[32307]: pgmap v391: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T09:17:00.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:00 smithi194 ceph-mon[36729]: pgmap v391: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T09:17:00.469 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-16T09:17:00.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:00 smithi159 ceph-mon[36739]: pgmap v391: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-16T09:17:00.533 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-16T09:17:00.575 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-16T09:17:00.577 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-16T09:17:00.577 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-16T09:17:00.617 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-16T09:17:00.620 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T09:17:00.620 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-16T09:17:00.675 INFO:tasks.workunit:Running workunits matching fs/norstats on client.0... 2024-03-16T09:17:00.676 INFO:tasks.workunit:Running workunit fs/norstats/kernel_untar_tar.sh... 2024-03-16T09:17:00.676 DEBUG:teuthology.orchestra.run.smithi112:workunit test fs/norstats/kernel_untar_tar.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh 2024-03-16T09:17:00.735 INFO:tasks.workunit.client.0.smithi112.stderr:+ KERNEL=linux-4.0.5 2024-03-16T09:17:00.735 INFO:tasks.workunit.client.0.smithi112.stderr:+ wget -q http://download.ceph.com/qa/linux-4.0.5.tar.xz 2024-03-16T09:17:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.882+0000 7f820eb10640 1 -- 172.21.15.112:0/4030332314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8208102a00 msgr2=0x7f8208102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.882+0000 7f820eb10640 1 --2- 172.21.15.112:0/4030332314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8208102a00 0x7f8208102e80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009900 tx=0x7f81f402bc30 comp rx=0 tx=0).stop 2024-03-16T09:17:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.882+0000 7f820eb10640 1 -- 172.21.15.112:0/4030332314 shutdown_connections 2024-03-16T09:17:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.882+0000 7f820eb10640 1 --2- 172.21.15.112:0/4030332314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82081033c0 0x7f8208109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.882+0000 7f820eb10640 1 --2- 172.21.15.112:0/4030332314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8208102a00 0x7f8208102e80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.882+0000 7f820eb10640 1 --2- 172.21.15.112:0/4030332314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8208101810 0x7f8208101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.882+0000 7f820eb10640 1 -- 172.21.15.112:0/4030332314 >> 172.21.15.112:0/4030332314 conn(0x7f82080fcda0 msgr2=0x7f82080ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.882+0000 7f820eb10640 1 -- 172.21.15.112:0/4030332314 shutdown_connections 2024-03-16T09:17:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.882+0000 7f820eb10640 1 -- 172.21.15.112:0/4030332314 wait complete. 2024-03-16T09:17:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f820eb10640 1 Processor -- start 2024-03-16T09:17:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f820eb10640 1 -- start start 2024-03-16T09:17:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f820eb10640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8208101810 0x7f8208197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f820eb10640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8208102a00 0x7f82081980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f820eb10640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82081033c0 0x7f820819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f820eb10640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f820810d570 con 0x7f82081033c0 2024-03-16T09:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f820eb10640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f820810d3f0 con 0x7f8208101810 2024-03-16T09:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f820eb10640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f820810d6f0 con 0x7f8208102a00 2024-03-16T09:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f820d0ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82081033c0 0x7f820819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f820d0ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82081033c0 0x7f820819f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50896/0 (socket says 172.21.15.112:50896) 2024-03-16T09:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f820d0ad640 1 -- 172.21.15.112:0/824175393 learned_addr learned my addr 172.21.15.112:0/824175393 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f820c8ac640 1 --2- 172.21.15.112:0/824175393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8208101810 0x7f8208197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f8207fff640 1 --2- 172.21.15.112:0/824175393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8208102a00 0x7f82081980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f820d0ad640 1 -- 172.21.15.112:0/824175393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8208102a00 msgr2=0x7f82081980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f820d0ad640 1 --2- 172.21.15.112:0/824175393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8208102a00 0x7f82081980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f820d0ad640 1 -- 172.21.15.112:0/824175393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8208101810 msgr2=0x7f8208197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f820d0ad640 1 --2- 172.21.15.112:0/824175393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8208101810 0x7f8208197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f820d0ad640 1 -- 172.21.15.112:0/824175393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820819f7a0 con 0x7f82081033c0 2024-03-16T09:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f820d0ad640 1 --2- 172.21.15.112:0/824175393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82081033c0 0x7f820819f130 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f820000eb10 tx=0x7f820000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f8205ffb640 1 -- 172.21.15.112:0/824175393 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8200004b20 con 0x7f82081033c0 2024-03-16T09:17:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f8205ffb640 1 -- 172.21.15.112:0/824175393 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8200004cc0 con 0x7f82081033c0 2024-03-16T09:17:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f820eb10640 1 -- 172.21.15.112:0/824175393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f820819fa90 con 0x7f82081033c0 2024-03-16T09:17:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.883+0000 7f8205ffb640 1 -- 172.21.15.112:0/824175393 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82000107f0 con 0x7f82081033c0 2024-03-16T09:17:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.884+0000 7f820eb10640 1 -- 172.21.15.112:0/824175393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f820819ff60 con 0x7f82081033c0 2024-03-16T09:17:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.886+0000 7f8205ffb640 1 -- 172.21.15.112:0/824175393 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 137342+0+0 (secure 0 0 0) 0x7f8200005020 con 0x7f82081033c0 2024-03-16T09:17:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.886+0000 7f82117d8740 1 Processor -- start 2024-03-16T09:17:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.886+0000 7f8205ffb640 1 --2- 172.21.15.112:0/824175393 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f81f80a2940 0x7f81f80a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.886+0000 7f82117d8740 1 -- start start 2024-03-16T09:17:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.886+0000 7f8207fff640 1 --2- 172.21.15.112:0/824175393 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f81f80a2940 0x7f81f80a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.886+0000 7f8205ffb640 1 -- 172.21.15.112:0/824175393 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f82000cf100 con 0x7f82081033c0 2024-03-16T09:17:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.886+0000 7f82117d8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9340 0x564e7d5b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.886+0000 7f82117d8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5b9d30 0x564e7d5be830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.886+0000 7f82117d8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5beea0 0x564e7d5c1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.886+0000 7f82117d8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564e7d51ea40 con 0x564e7d5b9340 2024-03-16T09:17:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.886+0000 7f82117d8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564e7d4c7920 con 0x564e7d5beea0 2024-03-16T09:17:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.886+0000 7f82117d8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564e7d51aae0 con 0x564e7d5b9d30 2024-03-16T09:17:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.886+0000 7f820c8ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9340 0x564e7d5b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.887+0000 7f820c8ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9340 0x564e7d5b9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50908/0 (socket says 172.21.15.112:50908) 2024-03-16T09:17:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.887+0000 7f8207fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5b9d30 0x564e7d5be830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.887+0000 7f820c8ac640 1 -- 172.21.15.112:0/2642416887 learned_addr learned my addr 172.21.15.112:0/2642416887 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.887+0000 7f820d0ad640 1 --2- 172.21.15.112:0/2642416887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5beea0 0x564e7d5c1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.887+0000 7f820c8ac640 1 -- 172.21.15.112:0/2642416887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5b9d30 msgr2=0x564e7d5be830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.887+0000 7f820c8ac640 1 --2- 172.21.15.112:0/2642416887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5b9d30 0x564e7d5be830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.887+0000 7f820c8ac640 1 -- 172.21.15.112:0/2642416887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5beea0 msgr2=0x564e7d5c1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.887+0000 7f820c8ac640 1 --2- 172.21.15.112:0/2642416887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5beea0 0x564e7d5c1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.887+0000 7f820c8ac640 1 -- 172.21.15.112:0/2642416887 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e7d4df460 con 0x564e7d5b9340 2024-03-16T09:17:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.887+0000 7f8207fff640 1 --2- 172.21.15.112:0/824175393 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f81f80a2940 0x7f81f80a4e00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f81f402c900 tx=0x7f81f402dbc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.887+0000 7f820c8ac640 1 --2- 172.21.15.112:0/2642416887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9340 0x564e7d5b9760 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009ca0 tx=0x7f81fc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f309b657b92188df server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.887+0000 7f81f2ffd640 1 -- 172.21.15.112:0/2642416887 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc009e50 con 0x564e7d5b9340 2024-03-16T09:17:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.887+0000 7f81f2ffd640 1 -- 172.21.15.112:0/2642416887 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f81fc0035f0 con 0x564e7d5b9340 2024-03-16T09:17:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.887+0000 7f81f2ffd640 1 -- 172.21.15.112:0/2642416887 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc019630 con 0x564e7d5b9340 2024-03-16T09:17:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.888+0000 7f82117d8740 1 -- 172.21.15.112:0/2642416887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9340 msgr2=0x564e7d5b9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.888+0000 7f82117d8740 1 --2- 172.21.15.112:0/2642416887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9340 0x564e7d5b9760 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009ca0 tx=0x7f81fc011810 comp rx=0 tx=0).stop 2024-03-16T09:17:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.888+0000 7f82117d8740 1 -- 172.21.15.112:0/2642416887 shutdown_connections 2024-03-16T09:17:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.888+0000 7f82117d8740 1 --2- 172.21.15.112:0/2642416887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5beea0 0x564e7d5c1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.888+0000 7f82117d8740 1 --2- 172.21.15.112:0/2642416887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5b9d30 0x564e7d5be830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.888+0000 7f82117d8740 1 --2- 172.21.15.112:0/2642416887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9340 0x564e7d5b9760 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.888+0000 7f82117d8740 1 -- 172.21.15.112:0/2642416887 >> 172.21.15.112:0/2642416887 conn(0x564e7d5b48b0 msgr2=0x564e7d5b4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.888+0000 7f82117d8740 1 -- 172.21.15.112:0/2642416887 shutdown_connections 2024-03-16T09:17:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.888+0000 7f82117d8740 1 -- 172.21.15.112:0/2642416887 wait complete. 2024-03-16T09:17:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.888+0000 7f82117d8740 1 -- start start 2024-03-16T09:17:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.888+0000 7f82117d8740 1 Processor -- start 2024-03-16T09:17:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.888+0000 7f82117d8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5b9340 0x564e7d5c3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.888+0000 7f82117d8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9d30 0x564e7d5c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f82117d8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5beea0 0x564e7d5cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f82117d8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564e7d5c1e50 con 0x564e7d5b9d30 2024-03-16T09:17:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f82117d8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564e7d4dad70 con 0x564e7d5b9340 2024-03-16T09:17:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f82117d8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564e7d5c1fd0 con 0x564e7d5beea0 2024-03-16T09:17:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f82117d8740 10 client.0 _ll_register_callbacks cb 0x564e7d232a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f8207fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9d30 0x564e7d5c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f8207fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9d30 0x564e7d5c3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50922/0 (socket says 172.21.15.112:50922) 2024-03-16T09:17:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f8207fff640 1 -- 172.21.15.112:0/2117406964 learned_addr learned my addr 172.21.15.112:0/2117406964 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f820c8ac640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5b9340 0x564e7d5c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f820d0ad640 1 --2- 172.21.15.112:0/2117406964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5beea0 0x564e7d5cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f8207fff640 1 -- 172.21.15.112:0/2117406964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5beea0 msgr2=0x564e7d5cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f8207fff640 1 --2- 172.21.15.112:0/2117406964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5beea0 0x564e7d5cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f8207fff640 1 -- 172.21.15.112:0/2117406964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5b9340 msgr2=0x564e7d5c3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f8207fff640 1 --2- 172.21.15.112:0/2117406964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5b9340 0x564e7d5c3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f8207fff640 1 -- 172.21.15.112:0/2117406964 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e7d233210 con 0x564e7d5b9d30 2024-03-16T09:17:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f8207fff640 1 --2- 172.21.15.112:0/2117406964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9d30 0x564e7d5c3b90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f81f40096c0 tx=0x7f81f402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e800f3558ce2d7f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f81e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f81e7fff640 1 -- 172.21.15.112:0/2117406964 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4049070 con 0x564e7d5b9d30 2024-03-16T09:17:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f81e7fff640 1 -- 172.21.15.112:0/2117406964 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f81f40048d0 con 0x564e7d5b9d30 2024-03-16T09:17:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f82117d8740 1 -- 172.21.15.112:0/2117406964 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81fc0035f0 con 0x564e7d5b9d30 2024-03-16T09:17:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f81e7fff640 1 -- 172.21.15.112:0/2117406964 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f81f4004ad0 con 0x564e7d5b9d30 2024-03-16T09:17:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f82117d8740 10 client.15735 fetch_fsmap learned FSMap version 22 2024-03-16T09:17:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.889+0000 7f82117d8740 1 -- 172.21.15.112:0/2117406964 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564e7d2ff620 con 0x564e7d5b9d30 2024-03-16T09:17:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.890+0000 7f81e7fff640 1 -- 172.21.15.112:0/2117406964 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f81fc0035f0 con 0x564e7d5b9d30 2024-03-16T09:17:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.890+0000 7f81e7fff640 1 -- 172.21.15.112:0/2117406964 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f81f4040040 con 0x564e7d5b9d30 2024-03-16T09:17:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.890+0000 7f82117d8740 10 client.15735 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:17:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.890+0000 7f82117d8740 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.890+0000 7f82117d8740 20 client.15735 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.890+0000 7f82117d8740 1 --2- 172.21.15.112:0/2117406964 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x564e7d5cf4b0 0x564e7d5d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.890+0000 7f82117d8740 4 client.15735 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:17:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.890+0000 7f82117d8740 1 -- 172.21.15.112:0/2117406964 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e7d52bf80 con 0x564e7d5cf4b0 2024-03-16T09:17:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.890+0000 7f820c8ac640 1 --2- 172.21.15.112:0/2117406964 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x564e7d5cf4b0 0x564e7d5d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.890+0000 7f820c8ac640 1 --2- 172.21.15.112:0/2117406964 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x564e7d5cf4b0 0x564e7d5d1900 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a4a3bc9160f8ce8 server_cookie=e1136ec0814e3e27 in_seq=0 out_seq=0 2024-03-16T09:17:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.890+0000 7f81e7fff640 10 client.15735 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.891+0000 7f81e7fff640 1 -- 172.21.15.112:0/2117406964 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564e7d52bf80 con 0x564e7d5cf4b0 2024-03-16T09:17:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.891+0000 7f81e7fff640 10 client.15735 handle_command_reply: tid=0 2024-03-16T09:17:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.891+0000 7f82117d8740 1 client.15735 shutdown 2024-03-16T09:17:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.892+0000 7f82117d8740 1 -- 172.21.15.112:0/2117406964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9d30 msgr2=0x564e7d5c3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.892+0000 7f82117d8740 1 --2- 172.21.15.112:0/2117406964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9d30 0x564e7d5c3b90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f81f40096c0 tx=0x7f81f402bc30 comp rx=0 tx=0).stop 2024-03-16T09:17:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.892+0000 7f82117d8740 1 -- 172.21.15.112:0/2117406964 shutdown_connections 2024-03-16T09:17:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.892+0000 7f82117d8740 1 --2- 172.21.15.112:0/2117406964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5beea0 0x564e7d5cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.892+0000 7f82117d8740 1 --2- 172.21.15.112:0/2117406964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9d30 0x564e7d5c3b90 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.892+0000 7f82117d8740 1 --2- 172.21.15.112:0/2117406964 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x564e7d5cf4b0 0x564e7d5d1900 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.892+0000 7f82117d8740 1 --2- 172.21.15.112:0/2117406964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5b9340 0x564e7d5c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.892+0000 7f81e7fff640 0 client.15735 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.892+0000 7f82117d8740 1 -- 172.21.15.112:0/2117406964 >> 172.21.15.112:0/2117406964 conn(0x564e7d5b48b0 msgr2=0x564e7d5b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.892+0000 7f82117d8740 1 -- 172.21.15.112:0/2117406964 shutdown_connections 2024-03-16T09:17:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.892+0000 7f82117d8740 1 -- 172.21.15.112:0/2117406964 wait complete. 2024-03-16T09:17:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.892+0000 7f82117d8740 20 client.15735 trim_cache size 0 max 16384 2024-03-16T09:17:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.907+0000 7f82117d8740 1 Processor -- start 2024-03-16T09:17:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.907+0000 7f82117d8740 1 -- start start 2024-03-16T09:17:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f82117d8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b48b0 0x564e7d5b8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f82117d8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5b9340 0x564e7d5d6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f82117d8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5b9d30 0x564e7d5d6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f82117d8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f82081ab580 con 0x564e7d5b48b0 2024-03-16T09:17:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f82117d8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f82081a8e50 con 0x564e7d5b9d30 2024-03-16T09:17:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f82117d8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f82081a6ca0 con 0x564e7d5b9340 2024-03-16T09:17:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f820c8ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b48b0 0x564e7d5b8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f820c8ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b48b0 0x564e7d5b8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50930/0 (socket says 172.21.15.112:50930) 2024-03-16T09:17:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f820c8ac640 1 -- 172.21.15.112:0/2114006863 learned_addr learned my addr 172.21.15.112:0/2114006863 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f8207fff640 1 --2- 172.21.15.112:0/2114006863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5b9340 0x564e7d5d6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f820d0ad640 1 --2- 172.21.15.112:0/2114006863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5b9d30 0x564e7d5d6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f8207fff640 1 -- 172.21.15.112:0/2114006863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5b9d30 msgr2=0x564e7d5d6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f8207fff640 1 --2- 172.21.15.112:0/2114006863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5b9d30 0x564e7d5d6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f8207fff640 1 -- 172.21.15.112:0/2114006863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b48b0 msgr2=0x564e7d5b8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f8207fff640 1 --2- 172.21.15.112:0/2114006863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b48b0 0x564e7d5b8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f8207fff640 1 -- 172.21.15.112:0/2114006863 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e7d5b9720 con 0x564e7d5b9340 2024-03-16T09:17:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f820c8ac640 1 --2- 172.21.15.112:0/2114006863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b48b0 0x564e7d5b8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f8207fff640 1 --2- 172.21.15.112:0/2114006863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5b9340 0x564e7d5d6200 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f81f402c170 tx=0x7f81f4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8564af4214cbc373 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f81f27fc640 1 -- 172.21.15.112:0/2114006863 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f40048d0 con 0x564e7d5b9340 2024-03-16T09:17:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f81f27fc640 1 -- 172.21.15.112:0/2114006863 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f81f402de50 con 0x564e7d5b9340 2024-03-16T09:17:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.908+0000 7f81f27fc640 1 -- 172.21.15.112:0/2114006863 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4034960 con 0x564e7d5b9340 2024-03-16T09:17:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.909+0000 7f82117d8740 1 -- 172.21.15.112:0/2114006863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5b9340 msgr2=0x564e7d5d6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.909+0000 7f82117d8740 1 --2- 172.21.15.112:0/2114006863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5b9340 0x564e7d5d6200 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f81f402c170 tx=0x7f81f4005fd0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.909+0000 7f82117d8740 1 -- 172.21.15.112:0/2114006863 shutdown_connections 2024-03-16T09:17:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.909+0000 7f82117d8740 1 --2- 172.21.15.112:0/2114006863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5b9d30 0x564e7d5d6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.909+0000 7f82117d8740 1 --2- 172.21.15.112:0/2114006863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5b9340 0x564e7d5d6200 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.909+0000 7f82117d8740 1 --2- 172.21.15.112:0/2114006863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b48b0 0x564e7d5b8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.909+0000 7f82117d8740 1 -- 172.21.15.112:0/2114006863 >> 172.21.15.112:0/2114006863 conn(0x564e7d5ba930 msgr2=0x564e7d5b7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.909+0000 7f82117d8740 1 -- 172.21.15.112:0/2114006863 shutdown_connections 2024-03-16T09:17:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.909+0000 7f82117d8740 1 -- 172.21.15.112:0/2114006863 wait complete. 2024-03-16T09:17:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.909+0000 7f82117d8740 1 -- start start 2024-03-16T09:17:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.909+0000 7f82117d8740 1 Processor -- start 2024-03-16T09:17:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f82117d8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5b48b0 0x564e7d631950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f82117d8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5b9340 0x564e7d631e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f82117d8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9d30 0x564e7d638e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f82117d8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f82081a7310 con 0x564e7d5b9d30 2024-03-16T09:17:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f82117d8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f82081a8780 con 0x564e7d5b48b0 2024-03-16T09:17:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f82117d8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f82081a79e0 con 0x564e7d5b9340 2024-03-16T09:17:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f82117d8740 10 client.0 _ll_register_callbacks cb 0x7f81f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f820d0ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9d30 0x564e7d638e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f820c8ac640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5b48b0 0x564e7d631950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f8207fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5b9340 0x564e7d631e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f820d0ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9d30 0x564e7d638e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50946/0 (socket says 172.21.15.112:50946) 2024-03-16T09:17:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f8207fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5b9340 0x564e7d631e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:38006/0 (socket says 172.21.15.112:38006) 2024-03-16T09:17:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f820d0ad640 1 -- 172.21.15.112:0/657118258 learned_addr learned my addr 172.21.15.112:0/657118258 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f820c8ac640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5b48b0 0x564e7d631950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:39274/0 (socket says 172.21.15.112:39274) 2024-03-16T09:17:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f8207fff640 1 -- 172.21.15.112:0/657118258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5b48b0 msgr2=0x564e7d631950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f8207fff640 1 --2- 172.21.15.112:0/657118258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5b48b0 0x564e7d631950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f8207fff640 1 -- 172.21.15.112:0/657118258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9d30 msgr2=0x564e7d638e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f8207fff640 1 --2- 172.21.15.112:0/657118258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9d30 0x564e7d638e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f8207fff640 1 -- 172.21.15.112:0/657118258 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e7d639450 con 0x564e7d5b9340 2024-03-16T09:17:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f820d0ad640 1 --2- 172.21.15.112:0/657118258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9d30 0x564e7d638e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f8207fff640 1 --2- 172.21.15.112:0/657118258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5b9340 0x564e7d631e90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f81f4034650 tx=0x7f81f40048d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3975dedbed1dbdd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.910+0000 7f81e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T09:17:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.911+0000 7f81e7fff640 1 -- 172.21.15.112:0/657118258 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f406e060 con 0x564e7d5b9340 2024-03-16T09:17:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.911+0000 7f82117d8740 1 -- 172.21.15.112:0/657118258 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81f402de50 con 0x564e7d5b9340 2024-03-16T09:17:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.911+0000 7f81e7fff640 1 -- 172.21.15.112:0/657118258 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f81f403a040 con 0x564e7d5b9340 2024-03-16T09:17:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.911+0000 7f81e7fff640 1 -- 172.21.15.112:0/657118258 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f81f4049070 con 0x564e7d5b9340 2024-03-16T09:17:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.911+0000 7f82117d8740 10 client.25603 fetch_fsmap learned FSMap version 22 2024-03-16T09:17:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.911+0000 7f82117d8740 1 -- 172.21.15.112:0/657118258 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x564e7d63a760 con 0x564e7d5b9340 2024-03-16T09:17:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.911+0000 7f81e7fff640 1 -- 172.21.15.112:0/657118258 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f81f402de50 con 0x564e7d5b9340 2024-03-16T09:17:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.911+0000 7f81e7fff640 1 -- 172.21.15.112:0/657118258 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f81f40038f0 con 0x564e7d5b9340 2024-03-16T09:17:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.911+0000 7f82117d8740 10 client.25603 fetch_fsmap finished waiting for FSMap version 22 2024-03-16T09:17:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.911+0000 7f82117d8740 10 client.25603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.911+0000 7f82117d8740 20 client.25603 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.911+0000 7f82117d8740 1 --2- 172.21.15.112:0/657118258 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x564e7d5beea0 0x564e7d63d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.911+0000 7f82117d8740 4 client.25603 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T09:17:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.911+0000 7f82117d8740 1 -- 172.21.15.112:0/657118258 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e7d5262d0 con 0x564e7d5beea0 2024-03-16T09:17:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.911+0000 7f820c8ac640 1 --2- 172.21.15.112:0/657118258 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x564e7d5beea0 0x564e7d63d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.912+0000 7f81e7fff640 10 client.25603 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.912+0000 7f820c8ac640 1 --2- 172.21.15.112:0/657118258 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x564e7d5beea0 0x564e7d63d050 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2154b4187aa4fc6c server_cookie=eaf1c412213ba655 in_seq=0 out_seq=0 2024-03-16T09:17:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.919+0000 7f81e7fff640 1 -- 172.21.15.112:0/657118258 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564e7d5262d0 con 0x564e7d5beea0 2024-03-16T09:17:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.919+0000 7f81e7fff640 10 client.25603 handle_command_reply: tid=0 2024-03-16T09:17:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.919+0000 7f82117d8740 1 client.25603 shutdown 2024-03-16T09:17:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.919+0000 7f82117d8740 1 -- 172.21.15.112:0/657118258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5b9340 msgr2=0x564e7d631e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.919+0000 7f82117d8740 1 --2- 172.21.15.112:0/657118258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5b9340 0x564e7d631e90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f81f4034650 tx=0x7f81f40048d0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.919+0000 7f82117d8740 1 -- 172.21.15.112:0/657118258 shutdown_connections 2024-03-16T09:17:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.919+0000 7f82117d8740 1 --2- 172.21.15.112:0/657118258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564e7d5b9d30 0x564e7d638e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.919+0000 7f82117d8740 1 --2- 172.21.15.112:0/657118258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564e7d5b9340 0x564e7d631e90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.919+0000 7f82117d8740 1 --2- 172.21.15.112:0/657118258 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x564e7d5beea0 0x564e7d63d050 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.919+0000 7f82117d8740 1 --2- 172.21.15.112:0/657118258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7d5b48b0 0x564e7d631950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.919+0000 7f81e7fff640 0 client.25603 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.919+0000 7f82117d8740 1 -- 172.21.15.112:0/657118258 >> 172.21.15.112:0/657118258 conn(0x564e7d5ba930 msgr2=0x564e7d5b83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.919+0000 7f82117d8740 1 -- 172.21.15.112:0/657118258 shutdown_connections 2024-03-16T09:17:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.919+0000 7f82117d8740 1 -- 172.21.15.112:0/657118258 wait complete. 2024-03-16T09:17:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.919+0000 7f82117d8740 20 client.25603 trim_cache size 0 max 16384 2024-03-16T09:17:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-03-16T09:17:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-03-16T09:17:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-03-16T09:17:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-03-16T09:17:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.920+0000 7f820eb10640 1 -- 172.21.15.112:0/824175393 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f81f80a2940 msgr2=0x7f81f80a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.920+0000 7f820eb10640 1 --2- 172.21.15.112:0/824175393 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f81f80a2940 0x7f81f80a4e00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f81f402c900 tx=0x7f81f402dbc0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.920+0000 7f820eb10640 1 -- 172.21.15.112:0/824175393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82081033c0 msgr2=0x7f820819f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.920+0000 7f820eb10640 1 --2- 172.21.15.112:0/824175393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82081033c0 0x7f820819f130 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f820000eb10 tx=0x7f820000c2d0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.920+0000 7f820eb10640 1 -- 172.21.15.112:0/824175393 shutdown_connections 2024-03-16T09:17:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.920+0000 7f820eb10640 1 --2- 172.21.15.112:0/824175393 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f81f80a2940 0x7f81f80a4e00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.920+0000 7f820eb10640 1 --2- 172.21.15.112:0/824175393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82081033c0 0x7f820819f130 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.920+0000 7f820eb10640 1 --2- 172.21.15.112:0/824175393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8208102a00 0x7f82081980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.920+0000 7f820eb10640 1 --2- 172.21.15.112:0/824175393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8208101810 0x7f8208197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.920+0000 7f820eb10640 1 -- 172.21.15.112:0/824175393 >> 172.21.15.112:0/824175393 conn(0x7f82080fcda0 msgr2=0x7f8208105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.920+0000 7f820eb10640 1 -- 172.21.15.112:0/824175393 shutdown_connections 2024-03-16T09:17:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:00.920+0000 7f820eb10640 1 -- 172.21.15.112:0/824175393 wait complete. 2024-03-16T09:17:02.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:02 smithi112 ceph-mon[32307]: pgmap v392: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:17:02.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:02 smithi194 ceph-mon[36729]: pgmap v392: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:17:02.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:02 smithi159 ceph-mon[36739]: pgmap v392: 97 pgs: 97 active+clean; 669 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-16T09:17:03.795 INFO:tasks.workunit.client.0.smithi112.stderr:+ mkdir untar_tar 2024-03-16T09:17:03.799 INFO:tasks.workunit.client.0.smithi112.stderr:+ cd untar_tar 2024-03-16T09:17:03.800 INFO:tasks.workunit.client.0.smithi112.stderr:+ tar Jxvf ../linux-4.0.5.tar.xz linux-4.0.5/Documentation/ 2024-03-16T09:17:03.824 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ 2024-03-16T09:17:03.826 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/00-INDEX 2024-03-16T09:17:03.828 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/ 2024-03-16T09:17:03.830 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/README 2024-03-16T09:17:03.831 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/obsolete/ 2024-03-16T09:17:03.832 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/obsolete/proc-sys-vm-nr_pdflush_threads 2024-03-16T09:17:03.834 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-bus-usb 2024-03-16T09:17:03.835 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-class-rfkill 2024-03-16T09:17:03.837 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-koneplus 2024-03-16T09:17:03.842 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-kovaplus 2024-03-16T09:17:03.843 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra 2024-03-16T09:17:03.843 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/removed/ 2024-03-16T09:17:03.843 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/removed/devfs 2024-03-16T09:17:03.843 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/removed/dv1394 2024-03-16T09:17:03.843 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/removed/ip_queue 2024-03-16T09:17:03.844 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/removed/net_dma 2024-03-16T09:17:03.845 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/removed/o2cb 2024-03-16T09:17:03.846 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/removed/raw1394 2024-03-16T09:17:03.847 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/removed/video1394 2024-03-16T09:17:03.848 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/ 2024-03-16T09:17:03.849 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/firewire-cdev 2024-03-16T09:17:03.850 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/o2cb 2024-03-16T09:17:03.851 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/syscalls 2024-03-16T09:17:03.852 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-acpi-pmprofile 2024-03-16T09:17:03.853 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-firewire 2024-03-16T09:17:03.854 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-usb 2024-03-16T09:17:03.855 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-xen-backend 2024-03-16T09:17:03.857 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-backlight 2024-03-16T09:17:03.857 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-rfkill 2024-03-16T09:17:03.858 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-tpm 2024-03-16T09:17:03.859 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-ubi 2024-03-16T09:17:03.860 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-udc 2024-03-16T09:17:03.861 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-node 2024-03-16T09:17:03.862 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-cpu 2024-03-16T09:17:03.863 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-xen_memory 2024-03-16T09:17:03.865 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-ib_srp 2024-03-16T09:17:03.866 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-qla2xxx 2024-03-16T09:17:03.867 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-usb-usbtmc 2024-03-16T09:17:03.868 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-w1_ds28e04 2024-03-16T09:17:03.870 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-efi-vars 2024-03-16T09:17:03.871 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-dump 2024-03-16T09:17:03.873 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-elog 2024-03-16T09:17:03.874 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-module 2024-03-16T09:17:03.876 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-transport-srp 2024-03-16T09:17:03.878 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/thermal-notification 2024-03-16T09:17:03.879 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/vdso 2024-03-16T09:17:03.880 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/ 2024-03-16T09:17:03.882 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-spear-pcie-gadget 2024-03-16T09:17:03.883 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget 2024-03-16T09:17:03.885 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-acm 2024-03-16T09:17:03.886 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ecm 2024-03-16T09:17:03.888 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-eem 2024-03-16T09:17:03.889 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ffs 2024-03-16T09:17:03.891 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-hid 2024-03-16T09:17:03.893 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-loopback 2024-03-16T09:17:03.894 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-mass-storage 2024-03-16T09:17:03.896 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-midi 2024-03-16T09:17:03.898 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ncm 2024-03-16T09:17:03.899 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-obex 2024-03-16T09:17:03.900 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-phonet 2024-03-16T09:17:03.901 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-rndis 2024-03-16T09:17:03.902 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-serial 2024-03-16T09:17:03.903 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-sourcesink 2024-03-16T09:17:03.904 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-subset 2024-03-16T09:17:03.905 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac1 2024-03-16T09:17:03.906 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac2 2024-03-16T09:17:03.907 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uvc 2024-03-16T09:17:03.908 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-driver-genwqe 2024-03-16T09:17:03.909 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ec 2024-03-16T09:17:03.910 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ideapad 2024-03-16T09:17:03.911 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-olpc 2024-03-16T09:17:03.912 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pfo-nx-crypto 2024-03-16T09:17:03.913 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pktcdvd 2024-03-16T09:17:03.914 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/dev-kmsg 2024-03-16T09:17:03.915 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/evm 2024-03-16T09:17:03.916 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/ima_policy 2024-03-16T09:17:03.917 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/procfs-diskstats 2024-03-16T09:17:03.918 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/pstore 2024-03-16T09:17:03.919 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ata 2024-03-16T09:17:03.920 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block 2024-03-16T09:17:03.921 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-bcache 2024-03-16T09:17:03.922 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-dm 2024-03-16T09:17:03.923 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-rssd 2024-03-16T09:17:03.924 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-zram 2024-03-16T09:17:03.925 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-acpi 2024-03-16T09:17:03.926 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-amba 2024-03-16T09:17:03.927 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-bcma 2024-03-16T09:17:03.928 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etb10 2024-03-16T09:17:03.929 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etm3x 2024-03-16T09:17:03.930 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-funnel 2024-03-16T09:17:03.931 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-tmc 2024-03-16T09:17:03.932 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-css 2024-03-16T09:17:03.933 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-events 2024-03-16T09:17:03.934 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-format 2024-03-16T09:17:03.935 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_24x7 2024-03-16T09:17:03.936 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_gpci 2024-03-16T09:17:03.937 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-fcoe 2024-03-16T09:17:03.938 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-hsi 2024-03-16T09:17:03.939 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-fsa9480 2024-03-16T09:17:03.940 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352 2024-03-16T09:17:03.942 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-lm3533 2024-03-16T09:17:03.944 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio 2024-03-16T09:17:03.946 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-accel-bmc150 2024-03-16T09:17:03.947 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-ad9523 2024-03-16T09:17:03.948 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-adf4350 2024-03-16T09:17:03.949 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-gyro-bmg160 2024-03-16T09:17:03.950 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-light-lm3533-als 2024-03-16T09:17:03.951 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-mpu6050 2024-03-16T09:17:03.952 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935 2024-03-16T09:17:03.953 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-trigger-sysfs 2024-03-16T09:17:03.954 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mdio 2024-03-16T09:17:03.956 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-media 2024-03-16T09:17:03.956 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mei 2024-03-16T09:17:03.957 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci 2024-03-16T09:17:03.958 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss 2024-03-16T09:17:03.959 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-drivers-ehci_hcd 2024-03-16T09:17:03.960 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-platform 2024-03-16T09:17:03.961 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rbd 2024-03-16T09:17:03.962 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rpmsg 2024-03-16T09:17:03.963 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-umc 2024-03-16T09:17:03.964 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb 2024-03-16T09:17:03.965 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg 2024-03-16T09:17:03.966 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-lvstest 2024-03-16T09:17:03.967 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-c2port 2024-03-16T09:17:03.968 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-cfq-target-latency 2024-03-16T09:17:03.969 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class 2024-03-16T09:17:03.970 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-adp8870 2024-03-16T09:17:03.971 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-lm3533 2024-03-16T09:17:03.972 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-bdi 2024-03-16T09:17:03.973 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-cxl 2024-03-16T09:17:03.974 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-devfreq 2024-03-16T09:17:03.975 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-extcon 2024-03-16T09:17:03.976 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu 2024-03-16T09:17:03.977 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-amd-iommu 2024-03-16T09:17:03.978 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-intel-iommu 2024-03-16T09:17:03.979 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-lcd 2024-03-16T09:17:03.980 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led 2024-03-16T09:17:03.981 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led-driver-lm3533 2024-03-16T09:17:03.982 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-leds-gt683r 2024-03-16T09:17:03.983 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mei 2024-03-16T09:17:03.984 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mic.txt 2024-03-16T09:17:03.985 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mtd 2024-03-16T09:17:03.986 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net 2024-03-16T09:17:03.987 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-batman-adv 2024-03-16T09:17:03.988 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-cdc_ncm 2024-03-16T09:17:03.989 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-grcan 2024-03-16T09:17:03.990 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-mesh 2024-03-16T09:17:03.991 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-queues 2024-03-16T09:17:03.992 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-statistics 2024-03-16T09:17:03.993 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pktcdvd 2024-03-16T09:17:03.994 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-power 2024-03-16T09:17:03.995 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-powercap 2024-03-16T09:17:03.996 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pwm 2024-03-16T09:17:03.997 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rc 2024-03-16T09:17:03.999 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-regulator 2024-03-16T09:17:03.999 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rtc-rtc0-device-rtc_calibration 2024-03-16T09:17:04.000 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-scsi_host 2024-03-16T09:17:04.001 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc 2024-03-16T09:17:04.002 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc-wusbhc 2024-03-16T09:17:04.003 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-dev 2024-03-16T09:17:04.005 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices 2024-03-16T09:17:04.006 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-edac 2024-03-16T09:17:04.007 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-firmware_node 2024-03-16T09:17:04.009 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-lpss_ltr 2024-03-16T09:17:04.010 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-memory 2024-03-16T09:17:04.012 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-mmc 2024-03-16T09:17:04.013 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-online 2024-03-16T09:17:04.015 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-_UDC_-gadget 2024-03-16T09:17:04.017 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-docg3 2024-03-16T09:17:04.018 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-sh_mobile_lcdc_fb 2024-03-16T09:17:04.020 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power 2024-03-16T09:17:04.021 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D0 2024-03-16T09:17:04.023 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D1 2024-03-16T09:17:04.024 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D2 2024-03-16T09:17:04.026 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D3hot 2024-03-16T09:17:04.027 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_wakeup 2024-03-16T09:17:04.029 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_state 2024-03-16T09:17:04.031 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-real_power_state 2024-03-16T09:17:04.032 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-resource_in_use 2024-03-16T09:17:04.034 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-soc 2024-03-16T09:17:04.036 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-sun 2024-03-16T09:17:04.037 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-cpu 2024-03-16T09:17:04.038 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-ibm-rtl 2024-03-16T09:17:04.040 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-xen_cpu 2024-03-16T09:17:04.041 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-genwqe 2024-03-16T09:17:04.043 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid 2024-03-16T09:17:04.045 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-lenovo 2024-03-16T09:17:04.046 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-logitech-lg4ff 2024-03-16T09:17:04.050 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-multitouch 2024-03-16T09:17:04.050 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-picolcd 2024-03-16T09:17:04.051 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-prodikeys 2024-03-16T09:17:04.053 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-arvo 2024-03-16T09:17:04.054 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-isku 2024-03-16T09:17:04.056 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kone 2024-03-16T09:17:04.057 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-koneplus 2024-03-16T09:17:04.059 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-konepure 2024-03-16T09:17:04.060 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kovaplus 2024-03-16T09:17:04.062 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-lua 2024-03-16T09:17:04.063 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-pyra 2024-03-16T09:17:04.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:04 smithi112 ceph-mon[32307]: pgmap v393: 97 pgs: 97 active+clean; 683 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-03-16T09:17:04.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:04 smithi112 ceph-mon[32307]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-16T09:17:04.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:04 smithi194 ceph-mon[36729]: pgmap v393: 97 pgs: 97 active+clean; 683 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-03-16T09:17:04.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:04 smithi194 ceph-mon[36729]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-16T09:17:04.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:04 smithi159 ceph-mon[36739]: pgmap v393: 97 pgs: 97 active+clean; 683 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-03-16T09:17:04.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:04 smithi159 ceph-mon[36739]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-16T09:17:06.322 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-ryos 2024-03-16T09:17:06.325 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-savu 2024-03-16T09:17:06.326 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-srws1 2024-03-16T09:17:06.328 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-wiimote 2024-03-16T09:17:06.330 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-input-axp-pek 2024-03-16T09:17:06.332 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-intel-rapid-start 2024-03-16T09:17:06.333 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-pciback 2024-03-16T09:17:06.335 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-ppi 2024-03-16T09:17:06.337 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-samsung-laptop 2024-03-16T09:17:06.339 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-sunxi-sid 2024-03-16T09:17:06.340 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-tegra-fuse 2024-03-16T09:17:06.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:06 smithi112 ceph-mon[32307]: pgmap v394: 97 pgs: 97 active+clean; 683 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-03-16T09:17:06.342 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-toshiba_acpi 2024-03-16T09:17:06.344 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-wacom 2024-03-16T09:17:06.346 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkback 2024-03-16T09:17:06.347 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkfront 2024-03-16T09:17:06.349 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-acpi 2024-03-16T09:17:06.351 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-dmi 2024-03-16T09:17:06.353 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi 2024-03-16T09:17:06.355 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi-runtime-map 2024-03-16T09:17:06.356 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-gsmi 2024-03-16T09:17:06.358 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-log 2024-03-16T09:17:06.360 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-memmap 2024-03-16T09:17:06.361 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-ofw 2024-03-16T09:17:06.363 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sfi 2024-03-16T09:17:06.364 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sgi_uv 2024-03-16T09:17:06.365 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-ext4 2024-03-16T09:17:06.366 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-f2fs 2024-03-16T09:17:06.367 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-nilfs2 2024-03-16T09:17:06.369 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-xfs 2024-03-16T09:17:06.370 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-gpio 2024-03-16T09:17:06.371 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-i2c-bmp085 2024-03-16T09:17:06.372 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ibft 2024-03-16T09:17:06.373 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-boot_params 2024-03-16T09:17:06.374 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-fscaps 2024-03-16T09:17:06.375 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-iommu_groups 2024-03-16T09:17:06.376 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-livepatch 2024-03-16T09:17:06.377 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm 2024-03-16T09:17:06.378 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-hugepages 2024-03-16T09:17:06.379 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-ksm 2024-03-16T09:17:06.380 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-slab 2024-03-16T09:17:06.381 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-uids 2024-03-16T09:17:06.382 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo 2024-03-16T09:17:06.384 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-memory-page-offline 2024-03-16T09:17:06.385 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-module 2024-03-16T09:17:06.386 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ocfs2 2024-03-16T09:17:06.387 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-laptop 2024-03-16T09:17:06.388 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-wmi 2024-03-16T09:17:06.389 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-at91 2024-03-16T09:17:06.390 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-brcmstb-gisb-arb 2024-03-16T09:17:06.391 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-chipidea-usb-otg 2024-03-16T09:17:06.392 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-eeepc-laptop 2024-03-16T09:17:06.393 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ideapad-laptop 2024-03-16T09:17:06.394 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-kim 2024-03-16T09:17:06.395 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-msi-laptop 2024-03-16T09:17:06.396 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-tahvo-usb 2024-03-16T09:17:06.397 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ts5500 2024-03-16T09:17:06.399 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-power 2024-03-16T09:17:06.400 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-pps 2024-03-16T09:17:06.401 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-profiling 2024-03-16T09:17:06.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:06 smithi194 ceph-mon[36729]: pgmap v394: 97 pgs: 97 active+clean; 683 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-03-16T09:17:06.402 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ptp 2024-03-16T09:17:06.403 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-tty 2024-03-16T09:17:06.404 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-wusb_cbaf 2024-03-16T09:17:06.406 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/BUG-HUNTING 2024-03-16T09:17:06.406 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/Changes 2024-03-16T09:17:06.407 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/CodeOfConflict 2024-03-16T09:17:06.408 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/CodingStyle 2024-03-16T09:17:06.409 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DMA-API-HOWTO.txt 2024-03-16T09:17:06.411 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DMA-API.txt 2024-03-16T09:17:06.412 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DMA-ISA-LPC.txt 2024-03-16T09:17:06.413 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DMA-attributes.txt 2024-03-16T09:17:06.414 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/ 2024-03-16T09:17:06.415 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/.gitignore 2024-03-16T09:17:06.416 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/80211.tmpl 2024-03-16T09:17:06.417 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/Makefile 2024-03-16T09:17:06.418 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/alsa-driver-api.tmpl 2024-03-16T09:17:06.419 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/crypto-API.tmpl 2024-03-16T09:17:06.420 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/debugobjects.tmpl 2024-03-16T09:17:06.421 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/device-drivers.tmpl 2024-03-16T09:17:06.422 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/deviceiobook.tmpl 2024-03-16T09:17:06.422 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/drm.tmpl 2024-03-16T09:17:06.424 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/filesystems.tmpl 2024-03-16T09:17:06.425 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/gadget.tmpl 2024-03-16T09:17:06.426 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/genericirq.tmpl 2024-03-16T09:17:06.427 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/kernel-api.tmpl 2024-03-16T09:17:06.428 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/kernel-hacking.tmpl 2024-03-16T09:17:06.429 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/kernel-locking.tmpl 2024-03-16T09:17:06.431 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/kgdb.tmpl 2024-03-16T09:17:06.432 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/libata.tmpl 2024-03-16T09:17:06.433 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/librs.tmpl 2024-03-16T09:17:06.433 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/lsm.tmpl 2024-03-16T09:17:06.434 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/ 2024-03-16T09:17:06.435 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/Makefile 2024-03-16T09:17:06.436 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/bayer.png.b64 2024-03-16T09:17:06.437 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/constraints.png.b64 2024-03-16T09:17:06.438 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/crop.gif.b64 2024-03-16T09:17:06.439 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ 2024-03-16T09:17:06.440 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/.gitignore 2024-03-16T09:17:06.441 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/audio.xml 2024-03-16T09:17:06.442 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ca.xml 2024-03-16T09:17:06.443 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/demux.xml 2024-03-16T09:17:06.446 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbapi.xml 2024-03-16T09:17:06.447 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbproperty.xml 2024-03-16T09:17:06.449 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbstb.pdf 2024-03-16T09:17:06.451 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/examples.xml 2024-03-16T09:17:06.453 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/frontend.xml 2024-03-16T09:17:06.455 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/intro.xml 2024-03-16T09:17:06.458 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/kdapi.xml 2024-03-16T09:17:06.459 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/net.xml 2024-03-16T09:17:06.461 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/video.xml 2024-03-16T09:17:06.463 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvbstb.png.b64 2024-03-16T09:17:06.465 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_bt.gif.b64 2024-03-16T09:17:06.467 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_tb.gif.b64 2024-03-16T09:17:06.469 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt.gif.b64 2024-03-16T09:17:06.471 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt_example.gif.b64 2024-03-16T09:17:06.472 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/pipeline.png.b64 2024-03-16T09:17:06.474 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/selection.png.b64 2024-03-16T09:17:06.476 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/ 2024-03-16T09:17:06.477 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/.gitignore 2024-03-16T09:17:06.479 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/biblio.xml 2024-03-16T09:17:06.481 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/capture.c.xml 2024-03-16T09:17:06.483 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/common.xml 2024-03-16T09:17:06.484 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/compat.xml 2024-03-16T09:17:06.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:06 smithi159 ceph-mon[36739]: pgmap v394: 97 pgs: 97 active+clean; 683 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-03-16T09:17:06.487 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/controls.xml 2024-03-16T09:17:06.490 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/crop.pdf 2024-03-16T09:17:06.491 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-capture.xml 2024-03-16T09:17:06.493 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-codec.xml 2024-03-16T09:17:06.494 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-effect.xml 2024-03-16T09:17:06.495 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-event.xml 2024-03-16T09:17:06.497 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-osd.xml 2024-03-16T09:17:06.499 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-output.xml 2024-03-16T09:17:06.501 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-overlay.xml 2024-03-16T09:17:06.503 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-radio.xml 2024-03-16T09:17:06.504 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-raw-vbi.xml 2024-03-16T09:17:06.506 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-rds.xml 2024-03-16T09:17:06.507 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sdr.xml 2024-03-16T09:17:06.508 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sliced-vbi.xml 2024-03-16T09:17:06.509 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-subdev.xml 2024-03-16T09:17:06.510 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-teletext.xml 2024-03-16T09:17:06.514 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/driver.xml 2024-03-16T09:17:06.514 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fdl-appendix.xml 2024-03-16T09:17:06.514 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_bt.pdf 2024-03-16T09:17:06.515 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_tb.pdf 2024-03-16T09:17:06.516 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-close.xml 2024-03-16T09:17:06.518 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-ioctl.xml 2024-03-16T09:17:06.519 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-mmap.xml 2024-03-16T09:17:06.520 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-munmap.xml 2024-03-16T09:17:06.521 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-open.xml 2024-03-16T09:17:06.522 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-poll.xml 2024-03-16T09:17:06.523 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-read.xml 2024-03-16T09:17:06.525 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-select.xml 2024-03-16T09:17:06.526 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-write.xml 2024-03-16T09:17:06.527 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/gen-errors.xml 2024-03-16T09:17:06.528 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/io.xml 2024-03-16T09:17:06.530 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/keytable.c.xml 2024-03-16T09:17:06.531 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/libv4l.xml 2024-03-16T09:17:06.532 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/lirc_device_interface.xml 2024-03-16T09:17:06.533 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-controller.xml 2024-03-16T09:17:06.534 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-close.xml 2024-03-16T09:17:06.536 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-ioctl.xml 2024-03-16T09:17:06.537 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-open.xml 2024-03-16T09:17:06.538 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-device-info.xml 2024-03-16T09:17:06.539 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-entities.xml 2024-03-16T09:17:06.541 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-links.xml 2024-03-16T09:17:06.542 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-setup-link.xml 2024-03-16T09:17:06.543 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pipeline.pdf 2024-03-16T09:17:06.545 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-grey.xml 2024-03-16T09:17:06.546 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-m420.xml 2024-03-16T09:17:06.548 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12.xml 2024-03-16T09:17:06.549 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12m.xml 2024-03-16T09:17:06.551 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12mt.xml 2024-03-16T09:17:06.552 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16.xml 2024-03-16T09:17:06.554 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16m.xml 2024-03-16T09:17:06.555 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv24.xml 2024-03-16T09:17:06.557 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-rgb.xml 2024-03-16T09:17:06.558 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-yuv.xml 2024-03-16T09:17:06.560 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr16.xml 2024-03-16T09:17:06.561 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr8.xml 2024-03-16T09:17:06.563 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs08.xml 2024-03-16T09:17:06.564 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs14le.xml 2024-03-16T09:17:06.566 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu08.xml 2024-03-16T09:17:06.568 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu16le.xml 2024-03-16T09:17:06.569 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-ru12le.xml 2024-03-16T09:17:06.571 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgbrg8.xml 2024-03-16T09:17:06.572 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgrbg8.xml 2024-03-16T09:17:06.574 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10.xml 2024-03-16T09:17:06.575 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10alaw8.xml 2024-03-16T09:17:06.577 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10dpcm8.xml 2024-03-16T09:17:06.578 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10p.xml 2024-03-16T09:17:06.580 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb12.xml 2024-03-16T09:17:06.583 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb8.xml 2024-03-16T09:17:06.584 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uv8.xml 2024-03-16T09:17:06.586 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uyvy.xml 2024-03-16T09:17:06.587 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-vyuy.xml 2024-03-16T09:17:06.589 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10.xml 2024-03-16T09:17:06.590 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10b.xml 2024-03-16T09:17:06.592 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y12.xml 2024-03-16T09:17:06.593 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y16.xml 2024-03-16T09:17:06.595 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y41p.xml 2024-03-16T09:17:06.596 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv410.xml 2024-03-16T09:17:06.598 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv411p.xml 2024-03-16T09:17:06.600 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420.xml 2024-03-16T09:17:06.602 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420m.xml 2024-03-16T09:17:06.603 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv422p.xml 2024-03-16T09:17:06.605 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuyv.xml 2024-03-16T09:17:06.606 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvu420m.xml 2024-03-16T09:17:06.608 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvyu.xml 2024-03-16T09:17:06.610 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt.xml 2024-03-16T09:17:06.612 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/planar-apis.xml 2024-03-16T09:17:06.613 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/remote_controllers.xml 2024-03-16T09:17:06.615 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selection-api.xml 2024-03-16T09:17:06.616 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selections-common.xml 2024-03-16T09:17:06.617 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-formats.xml 2024-03-16T09:17:06.619 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.dia 2024-03-16T09:17:06.621 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.svg 2024-03-16T09:17:06.622 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.dia 2024-03-16T09:17:06.624 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.svg 2024-03-16T09:17:06.625 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.dia 2024-03-16T09:17:06.627 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.svg 2024-03-16T09:17:06.629 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2.xml 2024-03-16T09:17:06.630 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2grab.c.xml 2024-03-16T09:17:06.632 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_525.pdf 2024-03-16T09:17:06.634 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_625.pdf 2024-03-16T09:17:06.636 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_hsync.pdf 2024-03-16T09:17:06.637 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml 2024-03-16T09:17:06.639 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-cropcap.xml 2024-03-16T09:17:06.640 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-chip-info.xml 2024-03-16T09:17:06.642 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-register.xml 2024-03-16T09:17:06.644 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-decoder-cmd.xml 2024-03-16T09:17:06.646 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dqevent.xml 2024-03-16T09:17:06.647 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dv-timings-cap.xml 2024-03-16T09:17:06.649 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-encoder-cmd.xml 2024-03-16T09:17:06.651 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-dv-timings.xml 2024-03-16T09:17:06.652 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-fmt.xml 2024-03-16T09:17:06.654 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-frameintervals.xml 2024-03-16T09:17:06.656 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-framesizes.xml 2024-03-16T09:17:06.657 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-freq-bands.xml 2024-03-16T09:17:06.659 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudio.xml 2024-03-16T09:17:06.661 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudioout.xml 2024-03-16T09:17:06.663 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enuminput.xml 2024-03-16T09:17:06.664 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumoutput.xml 2024-03-16T09:17:06.666 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumstd.xml 2024-03-16T09:17:06.668 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-expbuf.xml 2024-03-16T09:17:06.670 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audio.xml 2024-03-16T09:17:06.671 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audioout.xml 2024-03-16T09:17:06.672 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-crop.xml 2024-03-16T09:17:06.674 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ctrl.xml 2024-03-16T09:17:06.675 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-dv-timings.xml 2024-03-16T09:17:06.676 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-edid.xml 2024-03-16T09:17:06.678 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-enc-index.xml 2024-03-16T09:17:06.680 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ext-ctrls.xml 2024-03-16T09:17:06.683 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fbuf.xml 2024-03-16T09:17:06.685 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fmt.xml 2024-03-16T09:17:06.687 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-frequency.xml 2024-03-16T09:17:06.689 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-input.xml 2024-03-16T09:17:06.691 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-jpegcomp.xml 2024-03-16T09:17:06.693 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-modulator.xml 2024-03-16T09:17:06.695 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-output.xml 2024-03-16T09:17:06.697 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-parm.xml 2024-03-16T09:17:06.698 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-priority.xml 2024-03-16T09:17:06.701 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-selection.xml 2024-03-16T09:17:06.703 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-sliced-vbi-cap.xml 2024-03-16T09:17:06.705 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-std.xml 2024-03-16T09:17:06.708 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-tuner.xml 2024-03-16T09:17:06.710 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-log-status.xml 2024-03-16T09:17:06.712 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-overlay.xml 2024-03-16T09:17:06.715 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-prepare-buf.xml 2024-03-16T09:17:06.717 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-qbuf.xml 2024-03-16T09:17:06.719 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-query-dv-timings.xml 2024-03-16T09:17:06.722 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querybuf.xml 2024-03-16T09:17:06.810 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querycap.xml 2024-03-16T09:17:06.811 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-queryctrl.xml 2024-03-16T09:17:06.812 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querystd.xml 2024-03-16T09:17:06.813 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-reqbufs.xml 2024-03-16T09:17:06.815 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-s-hw-freq-seek.xml 2024-03-16T09:17:06.817 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-streamon.xml 2024-03-16T09:17:06.818 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-interval.xml 2024-03-16T09:17:06.820 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-size.xml 2024-03-16T09:17:06.822 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-mbus-code.xml 2024-03-16T09:17:06.823 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-crop.xml 2024-03-16T09:17:06.825 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-fmt.xml 2024-03-16T09:17:06.826 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-frame-interval.xml 2024-03-16T09:17:06.828 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-selection.xml 2024-03-16T09:17:06.830 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml 2024-03-16T09:17:06.832 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_525.gif.b64 2024-03-16T09:17:06.833 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_625.gif.b64 2024-03-16T09:17:06.835 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_hsync.gif.b64 2024-03-16T09:17:06.836 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media_api.tmpl 2024-03-16T09:17:06.838 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/mtdnand.tmpl 2024-03-16T09:17:06.840 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/networking.tmpl 2024-03-16T09:17:06.841 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/rapidio.tmpl 2024-03-16T09:17:06.843 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/regulator.tmpl 2024-03-16T09:17:06.845 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/s390-drivers.tmpl 2024-03-16T09:17:06.847 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/scsi.tmpl 2024-03-16T09:17:06.848 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/sh.tmpl 2024-03-16T09:17:06.850 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/stylesheet.xsl 2024-03-16T09:17:06.851 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/tracepoint.tmpl 2024-03-16T09:17:06.853 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/uio-howto.tmpl 2024-03-16T09:17:06.855 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/usb.tmpl 2024-03-16T09:17:06.857 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/w1.tmpl 2024-03-16T09:17:06.858 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/writing-an-alsa-driver.tmpl 2024-03-16T09:17:06.861 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/writing_musb_glue_layer.tmpl 2024-03-16T09:17:06.863 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/writing_usb_driver.tmpl 2024-03-16T09:17:06.865 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/z8530book.tmpl 2024-03-16T09:17:06.867 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/ 2024-03-16T09:17:06.868 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/1024x768.S 2024-03-16T09:17:06.873 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/1280x1024.S 2024-03-16T09:17:06.873 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/1600x1200.S 2024-03-16T09:17:06.873 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/1680x1050.S 2024-03-16T09:17:06.874 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/1920x1080.S 2024-03-16T09:17:06.876 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/800x600.S 2024-03-16T09:17:06.877 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/HOWTO.txt 2024-03-16T09:17:06.879 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/Makefile 2024-03-16T09:17:06.880 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/edid.S 2024-03-16T09:17:06.882 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/hex 2024-03-16T09:17:06.883 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/HOWTO 2024-03-16T09:17:06.885 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/IPMI.txt 2024-03-16T09:17:06.887 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/IRQ-affinity.txt 2024-03-16T09:17:06.889 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/IRQ-domain.txt 2024-03-16T09:17:06.890 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/IRQ.txt 2024-03-16T09:17:06.892 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/Intel-IOMMU.txt 2024-03-16T09:17:06.895 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/Makefile 2024-03-16T09:17:06.898 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ManagementStyle 2024-03-16T09:17:06.898 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/PCI/ 2024-03-16T09:17:06.899 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/PCI/00-INDEX 2024-03-16T09:17:06.900 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/PCI/MSI-HOWTO.txt 2024-03-16T09:17:06.902 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/PCI/PCIEBUS-HOWTO.txt 2024-03-16T09:17:06.903 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/PCI/pci-error-recovery.txt 2024-03-16T09:17:06.905 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/PCI/pci-iov-howto.txt 2024-03-16T09:17:06.906 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/PCI/pci.txt 2024-03-16T09:17:06.908 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/PCI/pcieaer-howto.txt 2024-03-16T09:17:06.910 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/ 2024-03-16T09:17:06.912 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/00-INDEX 2024-03-16T09:17:06.913 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/NMI-RCU.txt 2024-03-16T09:17:06.914 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/RTFP.txt 2024-03-16T09:17:06.916 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/UP.txt 2024-03-16T09:17:06.918 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/arrayRCU.txt 2024-03-16T09:17:06.922 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/checklist.txt 2024-03-16T09:17:06.922 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/listRCU.txt 2024-03-16T09:17:06.922 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/lockdep-splat.txt 2024-03-16T09:17:06.924 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/lockdep.txt 2024-03-16T09:17:06.926 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/rcu.txt 2024-03-16T09:17:06.927 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/rcu_dereference.txt 2024-03-16T09:17:06.929 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/rcubarrier.txt 2024-03-16T09:17:06.930 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/rculist_nulls.txt 2024-03-16T09:17:06.932 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/rcuref.txt 2024-03-16T09:17:06.933 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/stallwarn.txt 2024-03-16T09:17:06.935 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/torture.txt 2024-03-16T09:17:06.936 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/trace.txt 2024-03-16T09:17:06.938 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/whatisRCU.txt 2024-03-16T09:17:06.940 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/SAK.txt 2024-03-16T09:17:06.941 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/SM501.txt 2024-03-16T09:17:06.943 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/SecurityBugs 2024-03-16T09:17:06.945 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/SubmitChecklist 2024-03-16T09:17:06.946 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/SubmittingDrivers 2024-03-16T09:17:06.947 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/SubmittingPatches 2024-03-16T09:17:06.948 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/VGA-softcursor.txt 2024-03-16T09:17:06.949 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/accounting/ 2024-03-16T09:17:06.950 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/accounting/.gitignore 2024-03-16T09:17:06.951 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/accounting/Makefile 2024-03-16T09:17:06.952 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/accounting/cgroupstats.txt 2024-03-16T09:17:06.953 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/accounting/delay-accounting.txt 2024-03-16T09:17:06.954 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/accounting/getdelays.c 2024-03-16T09:17:06.955 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/accounting/taskstats-struct.txt 2024-03-16T09:17:06.956 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/accounting/taskstats.txt 2024-03-16T09:17:06.956 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/ 2024-03-16T09:17:06.957 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/apei/ 2024-03-16T09:17:06.958 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/apei/einj.txt 2024-03-16T09:17:06.959 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/apei/output_format.txt 2024-03-16T09:17:06.960 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/debug.txt 2024-03-16T09:17:06.961 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/dsdt-override.txt 2024-03-16T09:17:06.962 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/enumeration.txt 2024-03-16T09:17:06.963 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/gpio-properties.txt 2024-03-16T09:17:06.964 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/initrd_table_override.txt 2024-03-16T09:17:06.965 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/method-customizing.txt 2024-03-16T09:17:06.966 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/method-tracing.txt 2024-03-16T09:17:06.967 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/namespace.txt 2024-03-16T09:17:06.968 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/scan_handlers.txt 2024-03-16T09:17:06.969 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/video_extension.txt 2024-03-16T09:17:06.970 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/aoe/ 2024-03-16T09:17:06.971 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/aoe/aoe.txt 2024-03-16T09:17:06.972 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/aoe/autoload.sh 2024-03-16T09:17:06.973 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/aoe/status.sh 2024-03-16T09:17:06.974 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/aoe/todo.txt 2024-03-16T09:17:06.975 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/aoe/udev-install.sh 2024-03-16T09:17:06.976 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/aoe/udev.txt 2024-03-16T09:17:06.977 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/applying-patches.txt 2024-03-16T09:17:06.978 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/ 2024-03-16T09:17:06.979 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/00-INDEX 2024-03-16T09:17:06.980 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Atmel/ 2024-03-16T09:17:06.981 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Atmel/README 2024-03-16T09:17:06.982 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Booting 2024-03-16T09:17:06.983 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/CCN.txt 2024-03-16T09:17:06.984 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/IXP4xx 2024-03-16T09:17:06.985 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Interrupts 2024-03-16T09:17:06.986 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Makefile 2024-03-16T09:17:06.987 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Marvell/ 2024-03-16T09:17:06.988 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Marvell/README 2024-03-16T09:17:06.988 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Netwinder 2024-03-16T09:17:06.989 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/OMAP/ 2024-03-16T09:17:06.990 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/OMAP/DSS 2024-03-16T09:17:06.991 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/OMAP/omap_pm 2024-03-16T09:17:06.992 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Porting 2024-03-16T09:17:06.993 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/README 2024-03-16T09:17:06.994 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/ 2024-03-16T09:17:06.995 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/ADSBitsy 2024-03-16T09:17:06.996 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/Assabet 2024-03-16T09:17:06.996 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/Brutus 2024-03-16T09:17:06.997 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/CERF 2024-03-16T09:17:06.998 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/FreeBird 2024-03-16T09:17:06.999 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsClient 2024-03-16T09:17:07.000 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsMaster 2024-03-16T09:17:07.001 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/HUW_WEBPANEL 2024-03-16T09:17:07.002 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/Itsy 2024-03-16T09:17:07.003 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/LART 2024-03-16T09:17:07.004 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/PLEB 2024-03-16T09:17:07.005 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/Pangolin 2024-03-16T09:17:07.006 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/Tifon 2024-03-16T09:17:07.007 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/Victor 2024-03-16T09:17:07.008 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/Yopy 2024-03-16T09:17:07.009 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/empeg 2024-03-16T09:17:07.009 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/nanoEngine 2024-03-16T09:17:07.010 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/serial_UART 2024-03-16T09:17:07.011 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/ 2024-03-16T09:17:07.012 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/.gitignore 2024-03-16T09:17:07.013 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/Makefile 2024-03-16T09:17:07.014 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/vrl4.c 2024-03-16T09:17:07.015 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-mmcif.txt 2024-03-16T09:17:07.016 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-sdhi.txt 2024-03-16T09:17:07.017 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SPEAr/ 2024-03-16T09:17:07.018 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SPEAr/overview.txt 2024-03-16T09:17:07.018 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/ 2024-03-16T09:17:07.019 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/CPUfreq.txt 2024-03-16T09:17:07.020 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/EB2410ITX.txt 2024-03-16T09:17:07.021 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/GPIO.txt 2024-03-16T09:17:07.022 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/H1940.txt 2024-03-16T09:17:07.023 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/NAND.txt 2024-03-16T09:17:07.024 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Overview.txt 2024-03-16T09:17:07.025 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2412.txt 2024-03-16T09:17:07.025 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2413.txt 2024-03-16T09:17:07.026 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/SMDK2440.txt 2024-03-16T09:17:07.027 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Suspend.txt 2024-03-16T09:17:07.028 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/USB-Host.txt 2024-03-16T09:17:07.029 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung/ 2024-03-16T09:17:07.030 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung/GPIO.txt 2024-03-16T09:17:07.031 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung/Overview.txt 2024-03-16T09:17:07.032 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung/clksrc-change-registers.awk 2024-03-16T09:17:07.033 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Setup 2024-03-16T09:17:07.034 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/VFP/ 2024-03-16T09:17:07.035 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/VFP/release-notes.txt 2024-03-16T09:17:07.036 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/cluster-pm-race-avoidance.txt 2024-03-16T09:17:07.037 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/firmware.txt 2024-03-16T09:17:07.038 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/kernel_mode_neon.txt 2024-03-16T09:17:07.039 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/kernel_user_helpers.txt 2024-03-16T09:17:07.040 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/mem_alignment 2024-03-16T09:17:07.041 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/memory.txt 2024-03-16T09:17:07.041 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/msm/ 2024-03-16T09:17:07.042 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/msm/gpiomux.txt 2024-03-16T09:17:07.043 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/nwfpe/ 2024-03-16T09:17:07.044 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/nwfpe/NOTES 2024-03-16T09:17:07.045 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/nwfpe/README 2024-03-16T09:17:07.046 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/nwfpe/README.FPE 2024-03-16T09:17:07.047 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/nwfpe/TODO 2024-03-16T09:17:07.048 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/pxa/ 2024-03-16T09:17:07.049 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/pxa/mfp.txt 2024-03-16T09:17:07.050 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/sti/ 2024-03-16T09:17:07.051 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/sti/overview.txt 2024-03-16T09:17:07.052 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/sti/stih407-overview.txt 2024-03-16T09:17:07.053 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/sti/stih415-overview.txt 2024-03-16T09:17:07.054 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/sti/stih416-overview.txt 2024-03-16T09:17:07.055 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/sti/stih418-overview.txt 2024-03-16T09:17:07.056 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/sunxi/ 2024-03-16T09:17:07.057 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/sunxi/README 2024-03-16T09:17:07.058 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/sunxi/clocks.txt 2024-03-16T09:17:07.059 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/swp_emulation 2024-03-16T09:17:07.060 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/tcm.txt 2024-03-16T09:17:07.061 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/uefi.txt 2024-03-16T09:17:07.062 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/vlocks.txt 2024-03-16T09:17:07.063 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm64/ 2024-03-16T09:17:07.064 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm64/booting.txt 2024-03-16T09:17:07.065 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm64/legacy_instructions.txt 2024-03-16T09:17:07.066 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm64/memory.txt 2024-03-16T09:17:07.067 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm64/tagged-pointers.txt 2024-03-16T09:17:07.067 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/assoc_array.txt 2024-03-16T09:17:07.068 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/atomic_ops.txt 2024-03-16T09:17:07.070 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/auxdisplay/ 2024-03-16T09:17:07.070 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/auxdisplay/.gitignore 2024-03-16T09:17:07.071 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/auxdisplay/Makefile 2024-03-16T09:17:07.072 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b 2024-03-16T09:17:07.073 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b-example.c 2024-03-16T09:17:07.074 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/auxdisplay/ks0108 2024-03-16T09:17:07.075 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/backlight/ 2024-03-16T09:17:07.076 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/backlight/lp855x-driver.txt 2024-03-16T09:17:07.077 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/bad_memory.txt 2024-03-16T09:17:07.078 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/basic_profiling.txt 2024-03-16T09:17:07.079 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/bcache.txt 2024-03-16T09:17:07.080 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/binfmt_misc.txt 2024-03-16T09:17:07.081 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blackfin/ 2024-03-16T09:17:07.083 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blackfin/00-INDEX 2024-03-16T09:17:07.083 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blackfin/Makefile 2024-03-16T09:17:07.084 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blackfin/bfin-gpio-notes.txt 2024-03-16T09:17:07.085 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blackfin/bfin-spi-notes.txt 2024-03-16T09:17:07.086 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blackfin/gptimers-example.c 2024-03-16T09:17:07.087 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/ 2024-03-16T09:17:07.088 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/00-INDEX 2024-03-16T09:17:07.089 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/biodoc.txt 2024-03-16T09:17:07.090 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/biovecs.txt 2024-03-16T09:17:07.091 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/capability.txt 2024-03-16T09:17:07.092 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/cfq-iosched.txt 2024-03-16T09:17:07.093 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/cmdline-partition.txt 2024-03-16T09:17:07.094 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/data-integrity.txt 2024-03-16T09:17:07.095 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/deadline-iosched.txt 2024-03-16T09:17:07.096 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/ioprio.txt 2024-03-16T09:17:07.097 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/null_blk.txt 2024-03-16T09:17:07.098 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/queue-sysfs.txt 2024-03-16T09:17:07.099 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/request.txt 2024-03-16T09:17:07.100 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/stat.txt 2024-03-16T09:17:07.101 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/switching-sched.txt 2024-03-16T09:17:07.103 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/writeback_cache_control.txt 2024-03-16T09:17:07.104 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/ 2024-03-16T09:17:07.105 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/00-INDEX 2024-03-16T09:17:07.105 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/README.DAC960 2024-03-16T09:17:07.106 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/cciss.txt 2024-03-16T09:17:07.107 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/cpqarray.txt 2024-03-16T09:17:07.108 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/drbd/ 2024-03-16T09:17:07.109 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-8.3-data-packets.svg 2024-03-16T09:17:07.110 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-data-packets.svg 2024-03-16T09:17:07.111 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/drbd/README.txt 2024-03-16T09:17:07.113 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/drbd/conn-states-8.dot 2024-03-16T09:17:07.113 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/drbd/data-structure-v9.txt 2024-03-16T09:17:07.114 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/drbd/disk-states-8.dot 2024-03-16T09:17:07.115 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/drbd/drbd-connection-state-overview.dot 2024-03-16T09:17:07.116 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/drbd/node-states-8.dot 2024-03-16T09:17:07.117 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/floppy.txt 2024-03-16T09:17:07.118 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/mflash.txt 2024-03-16T09:17:07.119 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/nbd.txt 2024-03-16T09:17:07.120 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/paride.txt 2024-03-16T09:17:07.121 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/ramdisk.txt 2024-03-16T09:17:07.122 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/zram.txt 2024-03-16T09:17:07.123 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/braille-console.txt 2024-03-16T09:17:07.124 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/bt8xxgpio.txt 2024-03-16T09:17:07.126 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/btmrvl.txt 2024-03-16T09:17:07.127 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/bus-devices/ 2024-03-16T09:17:07.128 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/bus-devices/ti-gpmc.txt 2024-03-16T09:17:07.129 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/bus-virt-phys-mapping.txt 2024-03-16T09:17:07.130 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cachetlb.txt 2024-03-16T09:17:07.131 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cdrom/ 2024-03-16T09:17:07.132 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cdrom/00-INDEX 2024-03-16T09:17:07.133 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cdrom/Makefile 2024-03-16T09:17:07.134 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cdrom/cdrom-standard.tex 2024-03-16T09:17:07.135 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cdrom/ide-cd 2024-03-16T09:17:07.136 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cdrom/packet-writing.txt 2024-03-16T09:17:07.137 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/ 2024-03-16T09:17:07.138 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/00-INDEX 2024-03-16T09:17:07.139 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/blkio-controller.txt 2024-03-16T09:17:07.141 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/cgroups.txt 2024-03-16T09:17:07.142 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/cpuacct.txt 2024-03-16T09:17:07.143 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/cpusets.txt 2024-03-16T09:17:07.144 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/devices.txt 2024-03-16T09:17:07.145 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/freezer-subsystem.txt 2024-03-16T09:17:07.146 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/hugetlb.txt 2024-03-16T09:17:07.147 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/memcg_test.txt 2024-03-16T09:17:07.148 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/memory.txt 2024-03-16T09:17:07.149 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/net_cls.txt 2024-03-16T09:17:07.150 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/net_prio.txt 2024-03-16T09:17:07.151 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/unified-hierarchy.txt 2024-03-16T09:17:07.152 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/circular-buffers.txt 2024-03-16T09:17:07.153 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/clk.txt 2024-03-16T09:17:07.154 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/coccinelle.txt 2024-03-16T09:17:07.155 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/connector/ 2024-03-16T09:17:07.156 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/connector/.gitignore 2024-03-16T09:17:07.157 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/connector/Makefile 2024-03-16T09:17:07.158 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/connector/cn_test.c 2024-03-16T09:17:07.159 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/connector/connector.txt 2024-03-16T09:17:07.160 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/connector/ucon.c 2024-03-16T09:17:07.161 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/console/ 2024-03-16T09:17:07.162 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/console/console.txt 2024-03-16T09:17:07.163 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/ 2024-03-16T09:17:07.164 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/amd-powernow.txt 2024-03-16T09:17:07.165 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/boost.txt 2024-03-16T09:17:07.166 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/core.txt 2024-03-16T09:17:07.167 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/cpu-drivers.txt 2024-03-16T09:17:07.168 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-nforce2.txt 2024-03-16T09:17:07.169 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-stats.txt 2024-03-16T09:17:07.170 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/governors.txt 2024-03-16T09:17:07.171 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/index.txt 2024-03-16T09:17:07.172 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/intel-pstate.txt 2024-03-16T09:17:07.173 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/pcc-cpufreq.txt 2024-03-16T09:17:07.174 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/user-guide.txt 2024-03-16T09:17:07.175 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-hotplug.txt 2024-03-16T09:17:07.176 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-load.txt 2024-03-16T09:17:07.177 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpuidle/ 2024-03-16T09:17:07.178 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpuidle/core.txt 2024-03-16T09:17:07.179 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpuidle/driver.txt 2024-03-16T09:17:07.180 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpuidle/governor.txt 2024-03-16T09:17:07.181 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpuidle/sysfs.txt 2024-03-16T09:17:07.182 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cputopology.txt 2024-03-16T09:17:07.183 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/crc32.txt 2024-03-16T09:17:07.184 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cris/ 2024-03-16T09:17:07.185 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cris/README 2024-03-16T09:17:07.186 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/crypto/ 2024-03-16T09:17:07.187 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/crypto/api-intro.txt 2024-03-16T09:17:07.188 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/crypto/asymmetric-keys.txt 2024-03-16T09:17:07.189 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/crypto/async-tx-api.txt 2024-03-16T09:17:07.190 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/crypto/crypto-API-userspace.txt 2024-03-16T09:17:07.191 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/crypto/descore-readme.txt 2024-03-16T09:17:07.192 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dcdbas.txt 2024-03-16T09:17:07.193 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/debugging-modules.txt 2024-03-16T09:17:07.194 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/debugging-via-ohci1394.txt 2024-03-16T09:17:07.195 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dell_rbu.txt 2024-03-16T09:17:07.196 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/development-process/ 2024-03-16T09:17:07.197 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/development-process/1.Intro 2024-03-16T09:17:07.198 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/development-process/2.Process 2024-03-16T09:17:07.199 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/development-process/3.Early-stage 2024-03-16T09:17:07.200 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/development-process/4.Coding 2024-03-16T09:17:07.201 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/development-process/5.Posting 2024-03-16T09:17:07.202 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/development-process/6.Followthrough 2024-03-16T09:17:07.203 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/development-process/7.AdvancedTopics 2024-03-16T09:17:07.204 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/development-process/8.Conclusion 2024-03-16T09:17:07.205 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/ 2024-03-16T09:17:07.206 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/cache-policies.txt 2024-03-16T09:17:07.207 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/cache.txt 2024-03-16T09:17:07.208 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/delay.txt 2024-03-16T09:17:07.210 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/dm-crypt.txt 2024-03-16T09:17:07.211 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/dm-flakey.txt 2024-03-16T09:17:07.211 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/dm-io.txt 2024-03-16T09:17:07.212 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/dm-log.txt 2024-03-16T09:17:07.213 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/dm-queue-length.txt 2024-03-16T09:17:07.214 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/dm-raid.txt 2024-03-16T09:17:07.215 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/dm-service-time.txt 2024-03-16T09:17:07.216 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/dm-uevent.txt 2024-03-16T09:17:07.218 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/era.txt 2024-03-16T09:17:07.220 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/kcopyd.txt 2024-03-16T09:17:07.220 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/linear.txt 2024-03-16T09:17:07.221 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/persistent-data.txt 2024-03-16T09:17:07.222 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/snapshot.txt 2024-03-16T09:17:07.223 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/statistics.txt 2024-03-16T09:17:07.224 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/striped.txt 2024-03-16T09:17:07.225 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/switch.txt 2024-03-16T09:17:07.226 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/thin-provisioning.txt 2024-03-16T09:17:07.229 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/verity.txt 2024-03-16T09:17:07.229 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/zero.txt 2024-03-16T09:17:07.229 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devices.txt 2024-03-16T09:17:07.231 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/ 2024-03-16T09:17:07.232 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/00-INDEX 2024-03-16T09:17:07.233 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ 2024-03-16T09:17:07.234 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ABI.txt 2024-03-16T09:17:07.235 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/ 2024-03-16T09:17:07.236 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/interrupts.txt 2024-03-16T09:17:07.237 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/pmu.txt 2024-03-16T09:17:07.238 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ 2024-03-16T09:17:07.239 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/adapteva.txt 2024-03-16T09:17:07.240 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/ 2024-03-16T09:17:07.241 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-clk-manager.txt 2024-03-16T09:17:07.242 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-sdram-edac.txt 2024-03-16T09:17:07.243 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-system.txt 2024-03-16T09:17:07.244 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/amlogic.txt 2024-03-16T09:17:07.245 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arch_timer.txt 2024-03-16T09:17:07.246 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arm-boards 2024-03-16T09:17:07.247 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt 2024-03-16T09:17:07.248 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp.txt 2024-03-16T09:17:07.249 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-375.txt 2024-03-16T09:17:07.250 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-380-mpcore-soc-ctrl.txt 2024-03-16T09:17:07.251 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-38x.txt 2024-03-16T09:17:07.253 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-cpu-reset.txt 2024-03-16T09:17:07.253 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armadeus.txt 2024-03-16T09:17:07.254 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-at91.txt 2024-03-16T09:17:07.255 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-pmc.txt 2024-03-16T09:17:07.256 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/axxia.txt 2024-03-16T09:17:07.257 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/ 2024-03-16T09:17:07.258 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm11351.txt 2024-03-16T09:17:07.259 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm21664.txt 2024-03-16T09:17:07.260 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm63138.txt 2024-03-16T09:17:07.261 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/brcm,bcm11351-cpu-method 2024-03-16T09:17:07.262 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/cygnus.txt 2024-03-16T09:17:07.263 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-resetmgr.txt 2024-03-16T09:17:07.264 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-timer.txt 2024-03-16T09:17:07.265 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-wdt.txt 2024-03-16T09:17:07.267 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm2835.txt 2024-03-16T09:17:07.267 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm4708.txt 2024-03-16T09:17:07.268 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/brcm-brcmstb.txt 2024-03-16T09:17:07.269 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda.txt 2024-03-16T09:17:07.270 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/ 2024-03-16T09:17:07.271 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/combophy.txt 2024-03-16T09:17:07.272 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/l2ecc.txt 2024-03-16T09:17:07.273 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/mem-ctrlr.txt 2024-03-16T09:17:07.274 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cavium-thunder.txt 2024-03-16T09:17:07.275 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cci.txt 2024-03-16T09:17:07.277 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ccn.txt 2024-03-16T09:17:07.278 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coherency-fabric.txt 2024-03-16T09:17:07.279 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coresight.txt 2024-03-16T09:17:07.280 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/ 2024-03-16T09:17:07.282 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/marvell,berlin-smp 2024-03-16T09:17:07.283 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpus.txt 2024-03-16T09:17:07.285 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci.txt 2024-03-16T09:17:07.287 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/ 2024-03-16T09:17:07.288 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/cp-intc.txt 2024-03-16T09:17:07.290 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/digicolor.txt 2024-03-16T09:17:07.291 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/ 2024-03-16T09:17:07.293 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/power_domain.txt 2024-03-16T09:17:07.294 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/smp-sysram.txt 2024-03-16T09:17:07.296 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/ 2024-03-16T09:17:07.297 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/tlm,trusted-foundations.txt 2024-03-16T09:17:07.299 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fsl.txt 2024-03-16T09:17:07.301 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fw-cfg.txt 2024-03-16T09:17:07.302 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/geniatech.txt 2024-03-16T09:17:07.304 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic-v3.txt 2024-03-16T09:17:07.306 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic.txt 2024-03-16T09:17:07.306 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/global_timer.txt 2024-03-16T09:17:07.308 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/ 2024-03-16T09:17:07.310 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/hisilicon.txt 2024-03-16T09:17:07.311 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/idle-states.txt 2024-03-16T09:17:07.313 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/insignal-boards.txt 2024-03-16T09:17:07.314 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/ 2024-03-16T09:17:07.315 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/keystone.txt 2024-03-16T09:17:07.316 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/kirkwood.txt 2024-03-16T09:17:07.317 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/l2cc.txt 2024-03-16T09:17:07.318 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx-mic.txt 2024-03-16T09:17:07.319 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx.txt 2024-03-16T09:17:07.320 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,berlin.txt 2024-03-16T09:17:07.321 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,dove.txt 2024-03-16T09:17:07.322 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,kirkwood.txt 2024-03-16T09:17:07.323 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek.txt 2024-03-16T09:17:07.324 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/ 2024-03-16T09:17:07.325 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/mediatek,sysirq.txt 2024-03-16T09:17:07.326 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/moxart.txt 2024-03-16T09:17:07.329 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/ 2024-03-16T09:17:07.330 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/feroceon.txt 2024-03-16T09:17:07.330 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/intc.txt 2024-03-16T09:17:07.330 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/mrvl.txt 2024-03-16T09:17:07.330 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/tauros2.txt 2024-03-16T09:17:07.331 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/timer.txt 2024-03-16T09:17:07.332 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ 2024-03-16T09:17:07.333 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,kpss-acc.txt 2024-03-16T09:17:07.334 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,saw2.txt 2024-03-16T09:17:07.335 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ssbi.txt 2024-03-16T09:17:07.336 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/timer.txt 2024-03-16T09:17:07.337 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mvebu-system-controller.txt 2024-03-16T09:17:07.338 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/nspire.txt 2024-03-16T09:17:07.339 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/olimex.txt 2024-03-16T09:17:07.340 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/ 2024-03-16T09:17:07.341 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/counter.txt 2024-03-16T09:17:07.342 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/crossbar.txt 2024-03-16T09:17:07.343 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dmm.txt 2024-03-16T09:17:07.344 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dsp.txt 2024-03-16T09:17:07.345 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/intc.txt 2024-03-16T09:17:07.346 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/iva.txt 2024-03-16T09:17:07.347 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/l3-noc.txt 2024-03-16T09:17:07.348 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/mpu.txt 2024-03-16T09:17:07.349 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/omap.txt 2024-03-16T09:17:07.350 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/prcm.txt 2024-03-16T09:17:07.351 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/timer.txt 2024-03-16T09:17:07.352 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/picoxcell.txt 2024-03-16T09:17:07.353 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/pmu.txt 2024-03-16T09:17:07.354 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/primecell.txt 2024-03-16T09:17:07.356 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/psci.txt 2024-03-16T09:17:07.357 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip.txt 2024-03-16T09:17:07.359 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/ 2024-03-16T09:17:07.360 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu-sram.txt 2024-03-16T09:17:07.362 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu.txt 2024-03-16T09:17:07.363 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/smp-sram.txt 2024-03-16T09:17:07.365 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rtsm-dcscb.txt 2024-03-16T09:17:07.367 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung-boards.txt 2024-03-16T09:17:07.368 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/ 2024-03-16T09:17:07.370 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-adc.txt 2024-03-16T09:17:07.371 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-chipid.txt 2024-03-16T09:17:07.373 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/interrupt-combiner.txt 2024-03-16T09:17:07.375 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/pmu.txt 2024-03-16T09:17:07.376 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/sysreg.txt 2024-03-16T09:17:07.378 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/shmobile.txt 2024-03-16T09:17:07.379 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sirf.txt 2024-03-16T09:17:07.380 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-misc.txt 2024-03-16T09:17:07.381 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-timer.txt 2024-03-16T09:17:07.382 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear.txt 2024-03-16T09:17:07.383 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/ 2024-03-16T09:17:07.384 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/shirq.txt 2024-03-16T09:17:07.385 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sprd.txt 2024-03-16T09:17:07.386 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-nomadik.txt 2024-03-16T09:17:07.387 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-u300.txt 2024-03-16T09:17:07.388 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sti.txt 2024-03-16T09:17:07.389 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sunxi.txt 2024-03-16T09:17:07.390 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra.txt 2024-03-16T09:17:07.391 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/ 2024-03-16T09:17:07.392 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-ahb.txt 2024-03-16T09:17:07.393 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-emc.txt 2024-03-16T09:17:07.394 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-flowctrl.txt 2024-03-16T09:17:07.395 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-mc.txt 2024-03-16T09:17:07.396 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-pmc.txt 2024-03-16T09:17:07.397 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra30-mc.txt 2024-03-16T09:17:07.398 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/topology.txt 2024-03-16T09:17:07.399 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/twd.txt 2024-03-16T09:17:07.400 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/ 2024-03-16T09:17:07.401 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/power_domain.txt 2024-03-16T09:17:07.402 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-fpga-irq.txt 2024-03-16T09:17:07.403 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-sysreg.txt 2024-03-16T09:17:07.404 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-scc.txt 2024-03-16T09:17:07.405 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-sysreg.txt 2024-03-16T09:17:07.406 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress.txt 2024-03-16T09:17:07.407 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vic.txt 2024-03-16T09:17:07.408 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500.txt 2024-03-16T09:17:07.410 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/ 2024-03-16T09:17:07.411 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-intc.txt 2024-03-16T09:17:07.710 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-pmc.txt 2024-03-16T09:17:07.711 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-timer.txt 2024-03-16T09:17:07.713 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xen.txt 2024-03-16T09:17:07.713 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xilinx.txt 2024-03-16T09:17:07.714 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ 2024-03-16T09:17:07.715 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-platform.txt 2024-03-16T09:17:07.716 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-st.txt 2024-03-16T09:17:07.717 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/apm-xgene.txt 2024-03-16T09:17:07.718 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/atmel-at91_cf.txt 2024-03-16T09:17:07.719 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/cavium-compact-flash.txt 2024-03-16T09:17:07.720 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/exynos-sata.txt 2024-03-16T09:17:07.721 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/fsl-sata.txt 2024-03-16T09:17:07.722 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-pata.txt 2024-03-16T09:17:07.723 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-sata.txt 2024-03-16T09:17:07.725 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/marvell.txt 2024-03-16T09:17:07.726 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/pata-arasan.txt 2024-03-16T09:17:07.728 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/qcom-sata.txt 2024-03-16T09:17:07.729 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_highbank.txt 2024-03-16T09:17:07.730 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_rcar.txt 2024-03-16T09:17:07.731 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/tegra-sata.txt 2024-03-16T09:17:07.732 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/btmrvl.txt 2024-03-16T09:17:07.733 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ 2024-03-16T09:17:07.734 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/bcma.txt 2024-03-16T09:17:07.738 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/brcm,gisb-arb.txt 2024-03-16T09:17:07.738 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/imx-weim.txt 2024-03-16T09:17:07.738 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/mvebu-mbus.txt 2024-03-16T09:17:07.739 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/omap-ocp2scp.txt 2024-03-16T09:17:07.739 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ti-gpmc.txt 2024-03-16T09:17:07.740 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/ 2024-03-16T09:17:07.741 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/clocks.txt 2024-03-16T09:17:07.742 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/dscr.txt 2024-03-16T09:17:07.745 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/emifa.txt 2024-03-16T09:17:07.745 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/interrupt.txt 2024-03-16T09:17:07.745 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/soc.txt 2024-03-16T09:17:07.746 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/timer64.txt 2024-03-16T09:17:07.748 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/chosen.txt 2024-03-16T09:17:07.748 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ 2024-03-16T09:17:07.749 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/alphascale,acc.txt 2024-03-16T09:17:07.750 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/altr_socfpga.txt 2024-03-16T09:17:07.751 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/arm-integrator.txt 2024-03-16T09:17:07.752 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/at91-clock.txt 2024-03-16T09:17:07.753 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/axi-clkgen.txt 2024-03-16T09:17:07.754 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-cygnus-clock.txt 2024-03-16T09:17:07.755 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-kona-clock.txt 2024-03-16T09:17:07.756 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/calxeda.txt 2024-03-16T09:17:07.757 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-exynos-audss.txt 2024-03-16T09:17:07.758 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-palmas-clk32kg-clocks.txt 2024-03-16T09:17:07.758 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-s5pv210-audss.txt 2024-03-16T09:17:07.759 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clock-bindings.txt 2024-03-16T09:17:07.760 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clps711x-clock.txt 2024-03-16T09:17:07.761 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/efm32-clock.txt 2024-03-16T09:17:07.762 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/emev2-clock.txt 2024-03-16T09:17:07.763 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos3250-clock.txt 2024-03-16T09:17:07.764 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4-clock.txt 2024-03-16T09:17:07.765 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4415-clock.txt 2024-03-16T09:17:07.766 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5250-clock.txt 2024-03-16T09:17:07.767 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5260-clock.txt 2024-03-16T09:17:07.768 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5410-clock.txt 2024-03-16T09:17:07.769 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5420-clock.txt 2024-03-16T09:17:07.770 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5440-clock.txt 2024-03-16T09:17:07.771 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos7-clock.txt 2024-03-16T09:17:07.772 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-clock.txt 2024-03-16T09:17:07.773 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-factor-clock.txt 2024-03-16T09:17:07.774 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/gpio-gate-clock.txt 2024-03-16T09:17:07.775 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hi3620-clock.txt 2024-03-16T09:17:07.776 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hix5hd2-clock.txt 2024-03-16T09:17:07.777 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx1-clock.txt 2024-03-16T09:17:07.778 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx21-clock.txt 2024-03-16T09:17:07.779 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx23-clock.txt 2024-03-16T09:17:07.780 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx25-clock.txt 2024-03-16T09:17:07.781 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx27-clock.txt 2024-03-16T09:17:07.782 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx28-clock.txt 2024-03-16T09:17:07.783 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx31-clock.txt 2024-03-16T09:17:07.783 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx35-clock.txt 2024-03-16T09:17:07.784 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx5-clock.txt 2024-03-16T09:17:07.785 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6q-clock.txt 2024-03-16T09:17:07.786 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sl-clock.txt 2024-03-16T09:17:07.787 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sx-clock.txt 2024-03-16T09:17:07.788 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-gate.txt 2024-03-16T09:17:07.789 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-pll.txt 2024-03-16T09:17:07.790 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/lsi,axm5516-clks.txt 2024-03-16T09:17:07.791 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,mmp2.txt 2024-03-16T09:17:07.792 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa168.txt 2024-03-16T09:17:07.793 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa910.txt 2024-03-16T09:17:07.794 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77686.txt 2024-03-16T09:17:07.795 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77802.txt 2024-03-16T09:17:07.796 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/moxa,moxart-clock.txt 2024-03-16T09:17:07.797 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-core-clock.txt 2024-03-16T09:17:07.798 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-corediv-clock.txt 2024-03-16T09:17:07.799 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-cpu-clock.txt 2024-03-16T09:17:07.800 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-gated-clock.txt 2024-03-16T09:17:07.801 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nspire-clock.txt 2024-03-16T09:17:07.802 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra114-car.txt 2024-03-16T09:17:07.803 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra124-car.txt 2024-03-16T09:17:07.804 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra20-car.txt 2024-03-16T09:17:07.805 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra30-car.txt 2024-03-16T09:17:07.806 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/prima2-clock.txt 2024-03-16T09:17:07.807 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/pxa-clock.txt 2024-03-16T09:17:07.809 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,gcc.txt 2024-03-16T09:17:07.810 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,lcc.txt 2024-03-16T09:17:07.811 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,mmcc.txt 2024-03-16T09:17:07.812 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qoriq-clock.txt 2024-03-16T09:17:07.813 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-div6-clocks.txt 2024-03-16T09:17:07.814 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-mstp-clocks.txt 2024-03-16T09:17:07.815 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a73a4-cpg-clocks.txt 2024-03-16T09:17:07.816 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7740-cpg-clocks.txt 2024-03-16T09:17:07.817 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7779-cpg-clocks.txt 2024-03-16T09:17:07.818 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rcar-gen2-cpg-clocks.txt 2024-03-16T09:17:07.819 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rz-cpg-clocks.txt 2024-03-16T09:17:07.820 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,sh73a0-cpg-clocks.txt 2024-03-16T09:17:07.821 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3188-cru.txt 2024-03-16T09:17:07.822 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3288-cru.txt 2024-03-16T09:17:07.823 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip.txt 2024-03-16T09:17:07.824 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2410-clock.txt 2024-03-16T09:17:07.825 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2412-clock.txt 2024-03-16T09:17:07.826 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2443-clock.txt 2024-03-16T09:17:07.827 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c64xx-clock.txt 2024-03-16T09:17:07.828 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s5pv210-clock.txt 2024-03-16T09:17:07.829 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si5351.txt 2024-03-16T09:17:07.830 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si570.txt 2024-03-16T09:17:07.831 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st,nomadik.txt 2024-03-16T09:17:07.832 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/ 2024-03-16T09:17:07.833 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-divmux.txt 2024-03-16T09:17:07.834 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-mux.txt 2024-03-16T09:17:07.835 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-pll.txt 2024-03-16T09:17:07.836 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-prediv.txt 2024-03-16T09:17:07.837 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-vcc.txt 2024-03-16T09:17:07.838 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen.txt 2024-03-16T09:17:07.839 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,flexgen.txt 2024-03-16T09:17:07.840 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,quadfs.txt 2024-03-16T09:17:07.841 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ste-u300-syscon-clock.txt 2024-03-16T09:17:07.842 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/sunxi.txt 2024-03-16T09:17:07.843 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti,cdce706.txt 2024-03-16T09:17:07.844 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti-keystone-pllctrl.txt 2024-03-16T09:17:07.845 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/ 2024-03-16T09:17:07.847 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/apll.txt 2024-03-16T09:17:07.847 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/autoidle.txt 2024-03-16T09:17:07.848 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/clockdomain.txt 2024-03-16T09:17:07.849 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/composite.txt 2024-03-16T09:17:07.850 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/divider.txt 2024-03-16T09:17:07.851 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dpll.txt 2024-03-16T09:17:07.852 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dra7-atl.txt 2024-03-16T09:17:07.853 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fapll.txt 2024-03-16T09:17:07.854 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fixed-factor-clock.txt 2024-03-16T09:17:07.855 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/gate.txt 2024-03-16T09:17:07.856 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/interface.txt 2024-03-16T09:17:07.857 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/mux.txt 2024-03-16T09:17:07.858 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vf610-clock.txt 2024-03-16T09:17:07.858 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vt8500.txt 2024-03-16T09:17:07.859 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/xgene.txt 2024-03-16T09:17:07.860 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/zynq-7000.txt 2024-03-16T09:17:07.861 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/ 2024-03-16T09:17:07.862 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/arm_big_little_dt.txt 2024-03-16T09:17:07.863 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-dt.txt 2024-03-16T09:17:07.864 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-exynos5440.txt 2024-03-16T09:17:07.865 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-spear.txt 2024-03-16T09:17:07.866 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/ 2024-03-16T09:17:07.867 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/amd-ccp.txt 2024-03-16T09:17:07.868 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/atmel-crypto.txt 2024-03-16T09:17:07.869 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-dcp.txt 2024-03-16T09:17:07.870 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-imx-sahara.txt 2024-03-16T09:17:07.871 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec2.txt 2024-03-16T09:17:07.872 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec4.txt 2024-03-16T09:17:07.873 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec6.txt 2024-03-16T09:17:07.874 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/mv_cesa.txt 2024-03-16T09:17:07.875 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-aes.txt 2024-03-16T09:17:07.876 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-des.txt 2024-03-16T09:17:07.876 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-sham.txt 2024-03-16T09:17:07.877 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/picochip-spacc.txt 2024-03-16T09:17:07.878 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/qcom-qce.txt 2024-03-16T09:17:07.879 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/samsung-sss.txt 2024-03-16T09:17:07.880 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/ 2024-03-16T09:17:07.881 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/ 2024-03-16T09:17:07.882 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/exynos-ppmu.txt 2024-03-16T09:17:07.883 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ 2024-03-16T09:17:07.884 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/arm-pl330.txt 2024-03-16T09:17:07.885 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-dma.txt 2024-03-16T09:17:07.886 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-xdma.txt 2024-03-16T09:17:07.887 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/bcm2835-dma.txt 2024-03-16T09:17:07.888 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/dma.txt 2024-03-16T09:17:07.889 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-edma.txt 2024-03-16T09:17:07.889 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-dma.txt 2024-03-16T09:17:07.890 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-sdma.txt 2024-03-16T09:17:07.891 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-mxs-dma.txt 2024-03-16T09:17:07.892 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/img-mdc-dma.txt 2024-03-16T09:17:07.893 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/k3dma.txt 2024-03-16T09:17:07.894 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mmp-dma.txt 2024-03-16T09:17:07.895 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/moxa,moxart-dma.txt 2024-03-16T09:17:07.896 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mpc512x-dma.txt 2024-03-16T09:17:07.897 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mv-xor.txt 2024-03-16T09:17:07.898 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/nbpfaxi.txt 2024-03-16T09:17:07.899 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_adm.txt 2024-03-16T09:17:07.900 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_bam_dma.txt 2024-03-16T09:17:07.901 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/rcar-audmapp.txt 2024-03-16T09:17:07.903 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/renesas,rcar-dmac.txt 2024-03-16T09:17:07.904 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/shdma.txt 2024-03-16T09:17:07.906 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sirfsoc-dma.txt 2024-03-16T09:17:07.908 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/snps-dma.txt 2024-03-16T09:17:07.909 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-coh901318.txt 2024-03-16T09:17:07.911 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-dma40.txt 2024-03-16T09:17:07.912 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sun6i-dma.txt 2024-03-16T09:17:07.914 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/tegra20-apbdma.txt 2024-03-16T09:17:07.916 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ti-edma.txt 2024-03-16T09:17:07.918 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/ 2024-03-16T09:17:07.919 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt 2024-03-16T09:17:07.920 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_vdma.txt 2024-03-16T09:17:07.920 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/ 2024-03-16T09:17:07.921 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/ 2024-03-16T09:17:07.922 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/marvell,dove-lcd.txt 2024-03-16T09:17:07.923 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/ 2024-03-16T09:17:07.924 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/hlcdc-dc.txt 2024-03-16T09:17:07.925 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/ 2024-03-16T09:17:07.925 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/dw_hdmi.txt 2024-03-16T09:17:07.927 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/ 2024-03-16T09:17:07.927 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/tda998x.txt 2024-03-16T09:17:07.928 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ 2024-03-16T09:17:07.929 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/fsl-imx-drm.txt 2024-03-16T09:17:07.930 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/hdmi.txt 2024-03-16T09:17:07.931 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ldb.txt 2024-03-16T09:17:07.932 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/ 2024-03-16T09:17:07.933 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/gpu.txt 2024-03-16T09:17:07.933 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/hdmi.txt 2024-03-16T09:17:07.934 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/mdp.txt 2024-03-16T09:17:07.935 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/ 2024-03-16T09:17:07.936 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/panel.txt 2024-03-16T09:17:07.937 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/slave.txt 2024-03-16T09:17:07.938 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tfp410.txt 2024-03-16T09:17:07.939 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tilcdc.txt 2024-03-16T09:17:07.942 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/eeprom.txt 2024-03-16T09:17:07.942 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/ 2024-03-16T09:17:07.942 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-palmas.txt 2024-03-16T09:17:07.943 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-rt8973a.txt 2024-03-16T09:17:07.943 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-sm5502.txt 2024-03-16T09:17:07.944 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/ 2024-03-16T09:17:07.945 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/mxsfb.txt 2024-03-16T09:17:07.946 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/sm501fb.txt 2024-03-16T09:17:07.947 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/ 2024-03-16T09:17:07.948 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/altera-socfpga-fpga-mgr.txt 2024-03-16T09:17:07.949 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/ 2024-03-16T09:17:07.950 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/nvidia,tegra20-fuse.txt 2024-03-16T09:17:07.951 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/ 2024-03-16T09:17:07.952 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/8xxx_gpio.txt 2024-03-16T09:17:07.953 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/abilis,tb10x-gpio.txt 2024-03-16T09:17:07.954 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cavium-octeon-gpio.txt 2024-03-16T09:17:07.955 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cirrus,clps711x-mctrl-gpio.txt 2024-03-16T09:17:07.955 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fsl-imx-gpio.txt 2024-03-16T09:17:07.956 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fujitsu,mb86s70-gpio.txt 2024-03-16T09:17:07.957 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74x164.txt 2024-03-16T09:17:07.959 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74xx-mmio.txt 2024-03-16T09:17:07.960 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-adnp.txt 2024-03-16T09:17:07.960 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-bcm-kona.txt 2024-03-16T09:17:07.961 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-clps711x.txt 2024-03-16T09:17:07.962 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-davinci.txt 2024-03-16T09:17:07.963 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-dsp-keystone.txt 2024-03-16T09:17:07.965 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-fan.txt 2024-03-16T09:17:07.966 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-grgpio.txt 2024-03-16T09:17:07.968 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-lp3943.txt 2024-03-16T09:17:07.970 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-max732x.txt 2024-03-16T09:17:07.971 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mcp23s08.txt 2024-03-16T09:17:07.973 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mm-lantiq.txt 2024-03-16T09:17:07.974 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-msm.txt 2024-03-16T09:17:07.976 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mvebu.txt 2024-03-16T09:17:07.978 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mxs.txt 2024-03-16T09:17:07.979 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-nmk.txt 2024-03-16T09:17:07.981 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-omap.txt 2024-03-16T09:17:07.983 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-palmas.txt 2024-03-16T09:17:07.985 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pca953x.txt 2024-03-16T09:17:07.986 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pcf857x.txt 2024-03-16T09:17:07.988 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-poweroff.txt 2024-03-16T09:17:07.989 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-restart.txt 2024-03-16T09:17:07.991 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-samsung.txt 2024-03-16T09:17:07.993 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stericsson-coh901.txt 2024-03-16T09:17:07.994 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stmpe.txt 2024-03-16T09:17:07.996 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stp-xway.txt 2024-03-16T09:17:07.996 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-sx150x.txt 2024-03-16T09:17:07.997 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-twl4030.txt 2024-03-16T09:17:07.998 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090-pdc.txt 2024-03-16T09:17:07.999 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090.txt 2024-03-16T09:17:08.000 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-vf610.txt 2024-03-16T09:17:08.001 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene-sb.txt 2024-03-16T09:17:08.002 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene.txt 2024-03-16T09:17:08.003 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xilinx.txt 2024-03-16T09:17:08.004 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zevio.txt 2024-03-16T09:17:08.005 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zynq.txt 2024-03-16T09:17:08.006 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio.txt 2024-03-16T09:17:08.007 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_atmel.txt 2024-03-16T09:17:08.008 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_lpc32xx.txt 2024-03-16T09:17:08.009 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/moxa,moxart-gpio.txt 2024-03-16T09:17:08.011 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/mrvl-gpio.txt 2024-03-16T09:17:08.012 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/nvidia,tegra20-gpio.txt 2024-03-16T09:17:08.013 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/pl061-gpio.txt 2024-03-16T09:17:08.015 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/renesas,gpio-rcar.txt 2024-03-16T09:17:08.015 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/snps-dwapb-gpio.txt 2024-03-16T09:17:08.016 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/sodaville.txt 2024-03-16T09:17:08.017 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/spear_spics.txt 2024-03-16T09:17:08.018 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/ 2024-03-16T09:17:08.019 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,gk20a.txt 2024-03-16T09:17:08.020 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,tegra20-host1x.txt 2024-03-16T09:17:08.021 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-g2d.txt 2024-03-16T09:17:08.022 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-rotator.txt 2024-03-16T09:17:08.023 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/st,stih4xx.txt 2024-03-16T09:17:08.024 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/graph.txt 2024-03-16T09:17:08.025 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/ 2024-03-16T09:17:08.026 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/hid-over-i2c.txt 2024-03-16T09:17:08.027 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/ 2024-03-16T09:17:08.028 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/client-devices.txt 2024-03-16T09:17:08.029 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/nokia-modem.txt 2024-03-16T09:17:08.030 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/omap-ssi.txt 2024-03-16T09:17:08.031 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ 2024-03-16T09:17:08.032 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ads1015.txt 2024-03-16T09:17:08.033 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/g762.txt 2024-03-16T09:17:08.034 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ibmpowernv.txt 2024-03-16T09:17:08.034 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/lm90.txt 2024-03-16T09:17:08.035 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ltc2978.txt 2024-03-16T09:17:08.036 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ntc_thermistor.txt 2024-03-16T09:17:08.037 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/pwm-fan.txt 2024-03-16T09:17:08.038 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/vexpress.txt 2024-03-16T09:17:08.039 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/ 2024-03-16T09:17:08.040 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/atmel-trng.txt 2024-03-16T09:17:08.041 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/omap_rng.txt 2024-03-16T09:17:08.042 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/timeriomem_rng.txt 2024-03-16T09:17:08.043 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ 2024-03-16T09:17:08.044 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,bcm2835-i2c.txt 2024-03-16T09:17:08.045 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,iproc-i2c.txt 2024-03-16T09:17:08.046 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-arb-gpio-challenge.txt 2024-03-16T09:17:08.048 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-at91.txt 2024-03-16T09:17:08.049 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-axxia.txt 2024-03-16T09:17:08.050 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-bcm-kona.txt 2024-03-16T09:17:08.051 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cadence.txt 2024-03-16T09:17:08.052 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cbus-gpio.txt 2024-03-16T09:17:08.053 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cros-ec-tunnel.txt 2024-03-16T09:17:08.054 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-davinci.txt 2024-03-16T09:17:08.055 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-designware.txt 2024-03-16T09:17:08.056 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-efm32.txt 2024-03-16T09:17:08.056 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-exynos5.txt 2024-03-16T09:17:08.058 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-gpio.txt 2024-03-16T09:17:08.058 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-hix5hd2.txt 2024-03-16T09:17:08.059 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-img-scb.txt 2024-03-16T09:17:08.060 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-imx.txt 2024-03-16T09:17:08.061 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-meson.txt 2024-03-16T09:17:08.062 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mpc.txt 2024-03-16T09:17:08.063 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-gpio.txt 2024-03-16T09:17:08.064 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pca954x.txt 2024-03-16T09:17:08.065 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pinctrl.txt 2024-03-16T09:17:08.066 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux.txt 2024-03-16T09:17:08.067 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mv64xxx.txt 2024-03-16T09:17:08.068 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mxs.txt 2024-03-16T09:17:08.069 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-nomadik.txt 2024-03-16T09:17:08.070 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-ocores.txt 2024-03-16T09:17:08.071 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-octeon.txt 2024-03-16T09:17:08.072 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-omap.txt 2024-03-16T09:17:08.073 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-opal.txt 2024-03-16T09:17:08.074 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pnx.txt 2024-03-16T09:17:08.075 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa-pci-ce4100.txt 2024-03-16T09:17:08.076 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa.txt 2024-03-16T09:17:08.077 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rcar.txt 2024-03-16T09:17:08.078 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-riic.txt 2024-03-16T09:17:08.079 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rk3x.txt 2024-03-16T09:17:08.080 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-s3c2410.txt 2024-03-16T09:17:08.081 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sh_mobile.txt 2024-03-16T09:17:08.082 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sirf.txt 2024-03-16T09:17:08.083 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st-ddci2c.txt 2024-03-16T09:17:08.084 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st.txt 2024-03-16T09:17:08.085 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sunxi-p2wi.txt 2024-03-16T09:17:08.086 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-versatile.txt 2024-03-16T09:17:08.087 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-vt8500.txt 2024-03-16T09:17:08.089 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-xiic.txt 2024-03-16T09:17:08.090 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina209.txt 2024-03-16T09:17:08.092 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina2xx.txt 2024-03-16T09:17:08.093 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/max6697.txt 2024-03-16T09:17:08.095 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/nvidia,tegra20-i2c.txt 2024-03-16T09:17:08.097 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/qcom,i2c-qup.txt 2024-03-16T09:17:08.099 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ti,bq32k.txt 2024-03-16T09:17:08.099 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/trivial-devices.txt 2024-03-16T09:17:08.101 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/ 2024-03-16T09:17:08.103 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/ 2024-03-16T09:17:08.104 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/bma180.txt 2024-03-16T09:17:08.106 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/ 2024-03-16T09:17:08.107 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/at91_adc.txt 2024-03-16T09:17:08.108 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/cc10001_adc.txt 2024-03-16T09:17:08.110 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/max1027-adc.txt 2024-03-16T09:17:08.112 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/nuvoton-nau7802.txt 2024-03-16T09:17:08.113 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-iadc.txt 2024-03-16T09:17:08.115 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-vadc.txt 2024-03-16T09:17:08.116 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/rockchip-saradc.txt 2024-03-16T09:17:08.118 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/twl4030-madc.txt 2024-03-16T09:17:08.120 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/vf610-adc.txt 2024-03-16T09:17:08.121 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/xilinx-xadc.txt 2024-03-16T09:17:08.123 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ 2024-03-16T09:17:08.124 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ad7303.txt 2024-03-16T09:17:08.126 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/max5821.txt 2024-03-16T09:17:08.128 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/ 2024-03-16T09:17:08.129 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/adf4350.txt 2024-03-16T09:17:08.130 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/ 2024-03-16T09:17:08.131 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/dht11.txt 2024-03-16T09:17:08.132 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/iio-bindings.txt 2024-03-16T09:17:08.133 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/ 2024-03-16T09:17:08.134 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/apds9300.txt 2024-03-16T09:17:08.135 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/cm36651.txt 2024-03-16T09:17:08.136 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/gp2ap020a00f.txt 2024-03-16T09:17:08.137 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/tsl2563.txt 2024-03-16T09:17:08.138 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ 2024-03-16T09:17:08.139 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ak8975.txt 2024-03-16T09:17:08.140 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/hmc5843.txt 2024-03-16T09:17:08.141 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/ 2024-03-16T09:17:08.144 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/as3935.txt 2024-03-16T09:17:08.144 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/sensorhub.txt 2024-03-16T09:17:08.145 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/st-sensors.txt 2024-03-16T09:17:08.146 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ 2024-03-16T09:17:08.147 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ads7846.txt 2024-03-16T09:17:08.148 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/atmel,maxtouch.txt 2024-03-16T09:17:08.149 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cap11xx.txt 2024-03-16T09:17:08.150 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/clps711x-keypad.txt 2024-03-16T09:17:08.152 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cros-ec-keyb.txt 2024-03-16T09:17:08.154 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/e3x0-button.txt 2024-03-16T09:17:08.155 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elan_i2c.txt 2024-03-16T09:17:08.157 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elants_i2c.txt 2024-03-16T09:17:08.158 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/fsl-mma8450.txt 2024-03-16T09:17:08.160 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-beeper.txt 2024-03-16T09:17:08.162 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys-polled.txt 2024-03-16T09:17:08.163 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys.txt 2024-03-16T09:17:08.165 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-matrix-keypad.txt 2024-03-16T09:17:08.166 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/imx-keypad.txt 2024-03-16T09:17:08.168 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/input-reset.txt 2024-03-16T09:17:08.169 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/lpc32xx-key.txt 2024-03-16T09:17:08.170 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/matrix-keymap.txt 2024-03-16T09:17:08.171 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/nvidia,tegra20-kbc.txt 2024-03-16T09:17:08.172 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/omap-keypad.txt 2024-03-16T09:17:08.173 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ps2keyb-mouse-apbps2.txt 2024-03-16T09:17:08.174 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pwm-beeper.txt 2024-03-16T09:17:08.175 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pxa27x-keypad.txt 2024-03-16T09:17:08.176 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-keypad.txt 2024-03-16T09:17:08.177 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-pwrkey.txt 2024-03-16T09:17:08.178 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-vib.txt 2024-03-16T09:17:08.179 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/regulator-haptic.txt 2024-03-16T09:17:08.180 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/rotary-encoder.txt 2024-03-16T09:17:08.181 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/samsung-keypad.txt 2024-03-16T09:17:08.182 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/spear-keyboard.txt 2024-03-16T09:17:08.183 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/st-keyscan.txt 2024-03-16T09:17:08.184 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/stmpe-keypad.txt 2024-03-16T09:17:08.185 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/sun4i-lradc-keys.txt 2024-03-16T09:17:08.186 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tca8418_keypad.txt 2024-03-16T09:17:08.187 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv260x.txt 2024-03-16T09:17:08.188 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv2667.txt 2024-03-16T09:17:08.189 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,nspire-keypad.txt 2024-03-16T09:17:08.190 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,palmas-pwrbutton.txt 2024-03-16T09:17:08.191 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ 2024-03-16T09:17:08.192 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/auo_pixcir_ts.txt 2024-03-16T09:17:08.193 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/bu21013.txt 2024-03-16T09:17:08.194 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/edt-ft5x06.txt 2024-03-16T09:17:08.195 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/egalax-ts.txt 2024-03-16T09:17:08.196 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/lpc32xx-tsc.txt 2024-03-16T09:17:08.197 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/mms114.txt 2024-03-16T09:17:08.198 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/pixcir_i2c_ts.txt 2024-03-16T09:17:08.199 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sitronix-st1232.txt 2024-03-16T09:17:08.200 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/stmpe.txt 2024-03-16T09:17:08.201 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sun4i.txt 2024-03-16T09:17:08.202 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ti-tsc-adc.txt 2024-03-16T09:17:08.203 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/touchscreen.txt 2024-03-16T09:17:08.204 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2005.txt 2024-03-16T09:17:08.205 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2007.txt 2024-03-16T09:17:08.206 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/zforce_ts.txt 2024-03-16T09:17:08.206 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tps65218-pwrbutton.txt 2024-03-16T09:17:08.207 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-keypad.txt 2024-03-16T09:17:08.208 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-pwrbutton.txt 2024-03-16T09:17:08.209 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ 2024-03-16T09:17:08.211 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/abilis,tb10x-ictl.txt 2024-03-16T09:17:08.211 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun4i-ic.txt 2024-03-16T09:17:08.212 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun67i-sc-nmi.txt 2024-03-16T09:17:08.213 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/atmel,aic.txt 2024-03-16T09:17:08.214 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm2835-armctrl-ic.txt 2024-03-16T09:17:08.215 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm7120-l2-intc.txt 2024-03-16T09:17:08.216 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,l2-intc.txt 2024-03-16T09:17:08.217 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/cirrus,clps711x-intc.txt 2024-03-16T09:17:08.218 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/digicolor-ic.txt 2024-03-16T09:17:08.219 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/interrupts.txt 2024-03-16T09:17:08.220 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/lsi,zevio-intc.txt 2024-03-16T09:17:08.221 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,armada-370-xp-mpic.txt 2024-03-16T09:17:08.222 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,orion-intc.txt 2024-03-16T09:17:08.223 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/mips-gic.txt 2024-03-16T09:17:08.224 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/opencores,or1k-pic.txt 2024-03-16T09:17:08.226 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,intc-irqpin.txt 2024-03-16T09:17:08.227 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,irqc.txt 2024-03-16T09:17:08.229 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/samsung,s3c24xx-irq.txt 2024-03-16T09:17:08.230 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/snps,dw-apb-ictl.txt 2024-03-16T09:17:08.231 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,keystone-irq.txt 2024-03-16T09:17:08.232 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,omap-intc-irq.txt 2024-03-16T09:17:08.233 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ 2024-03-16T09:17:08.234 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/arm,smmu.txt 2024-03-16T09:17:08.235 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/iommu.txt 2024-03-16T09:17:08.236 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra20-gart.txt 2024-03-16T09:17:08.237 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra30-smmu.txt 2024-03-16T09:17:08.238 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/renesas,ipmmu-vmsa.txt 2024-03-16T09:17:08.239 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/rockchip,iommu.txt 2024-03-16T09:17:08.240 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/samsung,sysmmu.txt 2024-03-16T09:17:08.241 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ti,omap-iommu.txt 2024-03-16T09:17:08.242 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/ 2024-03-16T09:17:08.244 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/common.txt 2024-03-16T09:17:08.245 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-gpio.txt 2024-03-16T09:17:08.246 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp55xx.txt 2024-03-16T09:17:08.248 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp8860.txt 2024-03-16T09:17:08.249 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-ns2.txt 2024-03-16T09:17:08.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:08 smithi112 ceph-mon[32307]: pgmap v395: 97 pgs: 97 active+clean; 703 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-03-16T09:17:08.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:08 smithi194 ceph-mon[36729]: pgmap v395: 97 pgs: 97 active+clean; 703 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-03-16T09:17:08.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:08 smithi159 ceph-mon[36739]: pgmap v395: 97 pgs: 97 active+clean; 703 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-03-16T09:17:08.505 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-pwm.txt 2024-03-16T09:17:08.507 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/pca963x.txt 2024-03-16T09:17:08.509 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/register-bit-led.txt 2024-03-16T09:17:08.511 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/tca6507.txt 2024-03-16T09:17:08.512 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/ 2024-03-16T09:17:08.514 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2-timings.txt 2024-03-16T09:17:08.515 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2.txt 2024-03-16T09:17:08.517 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/ 2024-03-16T09:17:08.519 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/altera-mailbox.txt 2024-03-16T09:17:08.520 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/mailbox.txt 2024-03-16T09:17:08.522 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/omap-mailbox.txt 2024-03-16T09:17:08.524 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/marvell.txt 2024-03-16T09:17:08.525 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ 2024-03-16T09:17:08.526 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/atmel-isi.txt 2024-03-16T09:17:08.527 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/coda.txt 2024-03-16T09:17:08.528 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-fimc-lite.txt 2024-03-16T09:17:08.529 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-jpeg-codec.txt 2024-03-16T09:17:08.530 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos4-fimc-is.txt 2024-03-16T09:17:08.531 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos5-gsc.txt 2024-03-16T09:17:08.531 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/gpio-ir-receiver.txt 2024-03-16T09:17:08.532 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/hix5hd2-ir.txt 2024-03-16T09:17:08.534 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ 2024-03-16T09:17:08.535 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7343.txt 2024-03-16T09:17:08.537 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7604.txt 2024-03-16T09:17:08.538 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9m111.txt 2024-03-16T09:17:08.540 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9p031.txt 2024-03-16T09:17:08.542 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/nokia,smia.txt 2024-03-16T09:17:08.543 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ths8200.txt 2024-03-16T09:17:08.545 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp514x.txt 2024-03-16T09:17:08.546 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp7002.txt 2024-03-16T09:17:08.548 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/img-ir-rev1.txt 2024-03-16T09:17:08.549 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/meson-ir.txt 2024-03-16T09:17:08.551 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/pxa-camera.txt 2024-03-16T09:17:08.552 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/rcar_vin.txt 2024-03-16T09:17:08.554 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/renesas,vsp1.txt 2024-03-16T09:17:08.555 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/s5p-mfc.txt 2024-03-16T09:17:08.557 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-fimc.txt 2024-03-16T09:17:08.558 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-mipi-csis.txt 2024-03-16T09:17:08.560 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5c73m3.txt 2024-03-16T09:17:08.561 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k5baf.txt 2024-03-16T09:17:08.563 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k6a3.txt 2024-03-16T09:17:08.565 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sh_mobile_ceu.txt 2024-03-16T09:17:08.566 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/si4713.txt 2024-03-16T09:17:08.568 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/st-rc.txt 2024-03-16T09:17:08.569 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sunxi-ir.txt 2024-03-16T09:17:08.570 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ti-am437x-vpfe.txt 2024-03-16T09:17:08.572 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/video-interfaces.txt 2024-03-16T09:17:08.574 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ 2024-03-16T09:17:08.575 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ 2024-03-16T09:17:08.576 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ifc.txt 2024-03-16T09:17:08.578 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-devbus.txt 2024-03-16T09:17:08.579 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-sdram-controller.txt 2024-03-16T09:17:08.581 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra-mc.txt 2024-03-16T09:17:08.582 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/renesas-memory-controllers.txt 2024-03-16T09:17:08.584 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/synopsys.txt 2024-03-16T09:17:08.585 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti-aemif.txt 2024-03-16T09:17:08.587 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/ 2024-03-16T09:17:08.588 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/emif.txt 2024-03-16T09:17:08.590 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/ 2024-03-16T09:17:08.591 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta-intc.txt 2024-03-16T09:17:08.593 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta.txt 2024-03-16T09:17:08.594 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/pdc-intc.txt 2024-03-16T09:17:08.596 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ 2024-03-16T09:17:08.597 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/88pm860x.txt 2024-03-16T09:17:08.599 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ab8500.txt 2024-03-16T09:17:08.600 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/arizona.txt 2024-03-16T09:17:08.602 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3711.txt 2024-03-16T09:17:08.604 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3722.txt 2024-03-16T09:17:08.605 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-gpbr.txt 2024-03-16T09:17:08.606 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-hlcdc.txt 2024-03-16T09:17:08.608 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-matrix.txt 2024-03-16T09:17:08.609 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-smc.txt 2024-03-16T09:17:08.611 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bcm590xx.txt 2024-03-16T09:17:08.612 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bfticu.txt 2024-03-16T09:17:08.614 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/cros-ec.txt 2024-03-16T09:17:08.615 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9052-i2c.txt 2024-03-16T09:17:08.617 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9055.txt 2024-03-16T09:17:08.618 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9063.txt 2024-03-16T09:17:08.620 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/hi6421.txt 2024-03-16T09:17:08.621 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/lp3943.txt 2024-03-16T09:17:08.623 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max14577.txt 2024-03-16T09:17:08.624 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77686.txt 2024-03-16T09:17:08.626 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77693.txt 2024-03-16T09:17:08.628 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8925.txt 2024-03-16T09:17:08.629 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8998.txt 2024-03-16T09:17:08.631 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/mc13xxx.txt 2024-03-16T09:17:08.632 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-host.txt 2024-03-16T09:17:08.634 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-tll.txt 2024-03-16T09:17:08.635 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/palmas.txt 2024-03-16T09:17:08.637 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom,spmi-pmic.txt 2024-03-16T09:17:08.638 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-pm8xxx.txt 2024-03-16T09:17:08.640 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-rpm.txt 2024-03-16T09:17:08.642 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qriox.txt 2024-03-16T09:17:08.643 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rk808.txt 2024-03-16T09:17:08.645 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rn5t618.txt 2024-03-16T09:17:08.646 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mpa01.txt 2024-03-16T09:17:08.648 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mps11.txt 2024-03-16T09:17:08.649 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/stmpe.txt 2024-03-16T09:17:08.651 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/sun6i-prcm.txt 2024-03-16T09:17:08.653 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/syscon.txt 2024-03-16T09:17:08.654 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tc3589x.txt 2024-03-16T09:17:08.656 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ti-keystone-devctrl.txt 2024-03-16T09:17:08.657 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps6507x.txt 2024-03-16T09:17:08.659 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps65910.txt 2024-03-16T09:17:08.661 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl-familly.txt 2024-03-16T09:17:08.662 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-audio.txt 2024-03-16T09:17:08.664 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-power.txt 2024-03-16T09:17:08.666 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl6040.txt 2024-03-16T09:17:08.667 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/ 2024-03-16T09:17:08.669 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/ 2024-03-16T09:17:08.670 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/mipi-dsi-bus.txt 2024-03-16T09:17:08.672 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/nvidia,tegra114-mipi.txt 2024-03-16T09:17:08.674 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ 2024-03-16T09:17:08.675 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/ 2024-03-16T09:17:08.677 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bcm3384-intc.txt 2024-03-16T09:17:08.678 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bmips.txt 2024-03-16T09:17:08.679 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/cm-dsl.txt 2024-03-16T09:17:08.681 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/usb.txt 2024-03-16T09:17:08.683 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ 2024-03-16T09:17:08.684 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/bootbus.txt 2024-03-16T09:17:08.685 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/cib.txt 2024-03-16T09:17:08.687 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu.txt 2024-03-16T09:17:08.688 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu2.txt 2024-03-16T09:17:08.688 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/dma-engine.txt 2024-03-16T09:17:08.689 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/uctl.txt 2024-03-16T09:17:08.690 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cpu_irq.txt 2024-03-16T09:17:08.691 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ralink.txt 2024-03-16T09:17:08.692 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ 2024-03-16T09:17:08.693 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/allwinner,sunxi-sid.txt 2024-03-16T09:17:08.694 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/arm-charlcd.txt 2024-03-16T09:17:08.695 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/at25.txt 2024-03-16T09:17:08.696 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/atmel-ssc.txt 2024-03-16T09:17:08.697 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/bmp085.txt 2024-03-16T09:17:08.698 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/fsl,qoriq-mc.txt 2024-03-16T09:17:08.699 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ifm-csi.txt 2024-03-16T09:17:08.700 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/lis302.txt 2024-03-16T09:17:08.701 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/nvidia,tegra20-apbmisc.txt 2024-03-16T09:17:08.702 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/smc.txt 2024-03-16T09:17:08.704 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/sram.txt 2024-03-16T09:17:08.705 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ti,dac7512.txt 2024-03-16T09:17:08.705 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ 2024-03-16T09:17:08.706 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/arasan,sdhci.txt 2024-03-16T09:17:08.707 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/atmel-hsmci.txt 2024-03-16T09:17:08.708 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/brcm,bcm2835-sdhci.txt 2024-03-16T09:17:08.709 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/davinci_mmc.txt 2024-03-16T09:17:08.710 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/exynos-dw-mshc.txt 2024-03-16T09:17:08.711 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-esdhc.txt 2024-03-16T09:17:08.712 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-esdhc.txt 2024-03-16T09:17:08.713 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-mmc.txt 2024-03-16T09:17:08.714 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/img-dw-mshc.txt 2024-03-16T09:17:08.715 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/k3-dw-mshc.txt 2024-03-16T09:17:08.716 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/kona-sdhci.txt 2024-03-16T09:17:08.717 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-emmc.txt 2024-03-16T09:17:08.718 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-simple.txt 2024-03-16T09:17:08.719 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-spi-slot.txt 2024-03-16T09:17:08.720 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc.txt 2024-03-16T09:17:08.721 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmci.txt 2024-03-16T09:17:08.722 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/moxa,moxart-mmc.txt 2024-03-16T09:17:08.723 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mxs-mmc.txt 2024-03-16T09:17:08.724 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/nvidia,tegra20-sdhci.txt 2024-03-16T09:17:08.725 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/orion-sdio.txt 2024-03-16T09:17:08.726 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/pxa-mmc.txt 2024-03-16T09:17:08.727 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/renesas,mmcif.txt 2024-03-16T09:17:08.728 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.txt 2024-03-16T09:17:08.729 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/samsung-sdhci.txt 2024-03-16T09:17:08.730 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-dove.txt 2024-03-16T09:17:08.731 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-fujitsu.txt 2024-03-16T09:17:08.732 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-msm.txt 2024-03-16T09:17:08.733 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-pxa.txt 2024-03-16T09:17:08.734 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-sirf.txt 2024-03-16T09:17:08.735 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-spear.txt 2024-03-16T09:17:08.736 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-st.txt 2024-03-16T09:17:08.737 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/socfpga-dw-mshc.txt 2024-03-16T09:17:08.738 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sunxi-mmc.txt 2024-03-16T09:17:08.739 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/synopsys-dw-mshc.txt 2024-03-16T09:17:08.740 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap-hsmmc.txt 2024-03-16T09:17:08.741 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap.txt 2024-03-16T09:17:08.742 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/tmio_mmc.txt 2024-03-16T09:17:08.743 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/usdhi6rol0.txt 2024-03-16T09:17:08.744 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/vt8500-sdmmc.txt 2024-03-16T09:17:08.745 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/ 2024-03-16T09:17:08.746 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/arm-versatile.txt 2024-03-16T09:17:08.747 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-dataflash.txt 2024-03-16T09:17:08.748 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-nand.txt 2024-03-16T09:17:08.749 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/davinci-nand.txt 2024-03-16T09:17:08.750 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/denali-nand.txt 2024-03-16T09:17:08.751 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/diskonchip.txt 2024-03-16T09:17:08.752 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/elm.txt 2024-03-16T09:17:08.753 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/flctl-nand.txt 2024-03-16T09:17:08.754 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-quadspi.txt 2024-03-16T09:17:08.755 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-upm-nand.txt 2024-03-16T09:17:08.756 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsmc-nand.txt 2024-03-16T09:17:08.757 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpio-control-nand.txt 2024-03-16T09:17:08.758 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nand.txt 2024-03-16T09:17:08.760 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nor.txt 2024-03-16T09:17:08.760 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-onenand.txt 2024-03-16T09:17:08.761 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmi-nand.txt 2024-03-16T09:17:08.762 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/hisi504-nand.txt 2024-03-16T09:17:08.766 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-mlc.txt 2024-03-16T09:17:08.766 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-slc.txt 2024-03-16T09:17:08.766 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/m25p80.txt 2024-03-16T09:17:08.766 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mtd-physmap.txt 2024-03-16T09:17:08.767 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mxc-nand.txt 2024-03-16T09:17:08.768 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/nand.txt 2024-03-16T09:17:08.769 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/orion-nand.txt 2024-03-16T09:17:08.770 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/partition.txt 2024-03-16T09:17:08.771 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/pxa3xx-nand.txt 2024-03-16T09:17:08.772 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/spear_smi.txt 2024-03-16T09:17:08.773 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/st-fsm.txt 2024-03-16T09:17:08.774 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/sunxi-nand.txt 2024-03-16T09:17:08.776 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ 2024-03-16T09:17:08.777 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-emac.txt 2024-03-16T09:17:08.778 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-mdio.txt 2024-03-16T09:17:08.782 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun7i-a20-gmac.txt 2024-03-16T09:17:08.782 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/altera_tse.txt 2024-03-16T09:17:08.783 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe-phy.txt 2024-03-16T09:17:08.784 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe.txt 2024-03-16T09:17:08.785 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/apm-xgene-enet.txt 2024-03-16T09:17:08.786 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/arc_emac.txt 2024-03-16T09:17:08.788 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcm87xx.txt 2024-03-16T09:17:08.789 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcmgenet.txt 2024-03-16T09:17:08.791 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-mdio-unimac.txt 2024-03-16T09:17:08.792 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-sf2.txt 2024-03-16T09:17:08.794 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-systemport.txt 2024-03-16T09:17:08.795 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/calxeda-xgmac.txt 2024-03-16T09:17:08.797 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/ 2024-03-16T09:17:08.798 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/atmel-can.txt 2024-03-16T09:17:08.799 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/c_can.txt 2024-03-16T09:17:08.801 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/cc770.txt 2024-03-16T09:17:08.803 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/fsl-flexcan.txt 2024-03-16T09:17:08.804 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/grcan.txt 2024-03-16T09:17:08.805 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/m_can.txt 2024-03-16T09:17:08.807 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/microchip,mcp251x.txt 2024-03-16T09:17:08.808 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/mpc5xxx-mscan.txt 2024-03-16T09:17:08.810 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/rcar_can.txt 2024-03-16T09:17:08.812 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/sja1000.txt 2024-03-16T09:17:08.813 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/xilinx_can.txt 2024-03-16T09:17:08.815 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mdio.txt 2024-03-16T09:17:08.816 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mix.txt 2024-03-16T09:17:08.817 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-pip.txt 2024-03-16T09:17:08.818 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cdns-emac.txt 2024-03-16T09:17:08.819 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw-phy-sel.txt 2024-03-16T09:17:08.821 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw.txt 2024-03-16T09:17:08.822 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davicom-dm9000.txt 2024-03-16T09:17:08.823 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci-mdio.txt 2024-03-16T09:17:08.824 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci_emac.txt 2024-03-16T09:17:08.825 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/ 2024-03-16T09:17:08.827 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/dsa.txt 2024-03-16T09:17:08.828 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/emac_rockchip.txt 2024-03-16T09:17:08.829 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ethernet.txt 2024-03-16T09:17:08.830 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fixed-link.txt 2024-03-16T09:17:08.831 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-fec.txt 2024-03-16T09:17:08.833 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-tsec-phy.txt 2024-03-16T09:17:08.834 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/gpmc-eth.txt 2024-03-16T09:17:08.835 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hip04-net.txt 2024-03-16T09:17:08.836 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hix5hd2-gmac.txt 2024-03-16T09:17:08.837 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/ 2024-03-16T09:17:08.838 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/at86rf230.txt 2024-03-16T09:17:08.840 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/cc2520.txt 2024-03-16T09:17:08.841 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/keystone-netcp.txt 2024-03-16T09:17:08.842 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/lpc-eth.txt 2024-03-16T09:17:08.843 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/macb.txt 2024-03-16T09:17:08.844 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-armada-370-neta.txt 2024-03-16T09:17:08.846 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-mdio.txt 2024-03-16T09:17:08.847 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-net.txt 2024-03-16T09:17:08.848 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pp2.txt 2024-03-16T09:17:08.849 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pxa168.txt 2024-03-16T09:17:08.850 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-gpio.txt 2024-03-16T09:17:08.852 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-gpio.txt 2024-03-16T09:17:08.853 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-mmioreg.txt 2024-03-16T09:17:08.854 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux.txt 2024-03-16T09:17:08.856 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/meson-dwmac.txt 2024-03-16T09:17:08.857 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ks8851.txt 2024-03-16T09:17:08.859 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ksz90x1.txt 2024-03-16T09:17:08.860 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel.txt 2024-03-16T09:17:08.862 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/moxa,moxart-mac.txt 2024-03-16T09:17:08.863 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/ 2024-03-16T09:17:08.865 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/pn544.txt 2024-03-16T09:17:08.866 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfca.txt 2024-03-16T09:17:08.868 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfcb.txt 2024-03-16T09:17:08.869 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/trf7970a.txt 2024-03-16T09:17:08.871 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/opencores-ethoc.txt 2024-03-16T09:17:08.872 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/phy.txt 2024-03-16T09:17:08.874 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/qca-qca7000-spi.txt 2024-03-16T09:17:08.876 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/rockchip-dwmac.txt 2024-03-16T09:17:08.877 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/samsung-sxgbe.txt 2024-03-16T09:17:08.879 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sh_eth.txt 2024-03-16T09:17:08.880 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc-lan91c111.txt 2024-03-16T09:17:08.881 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc911x.txt 2024-03-16T09:17:08.882 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/socfpga-dwmac.txt 2024-03-16T09:17:08.883 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sti-dwmac.txt 2024-03-16T09:17:08.885 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/stmmac.txt 2024-03-16T09:17:08.886 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-rhine.txt 2024-03-16T09:17:08.887 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-velocity.txt 2024-03-16T09:17:08.888 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ 2024-03-16T09:17:08.890 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/brcm,bcm43xx-fmac.txt 2024-03-16T09:17:08.891 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.txt 2024-03-16T09:17:08.892 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ti,wl1251.txt 2024-03-16T09:17:08.893 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/ 2024-03-16T09:17:08.894 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/nios2.txt 2024-03-16T09:17:08.896 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/timer.txt 2024-03-16T09:17:08.897 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/ 2024-03-16T09:17:08.898 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/nvidia,nvec.txt 2024-03-16T09:17:08.899 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/open-pic.txt 2024-03-16T09:17:08.900 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/ 2024-03-16T09:17:08.902 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101aw03.txt 2024-03-16T09:17:08.903 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101xtn01.txt 2024-03-16T09:17:08.904 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b116xw03.txt 2024-03-16T09:17:08.905 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133htn01.txt 2024-03-16T09:17:08.907 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133xtn01.txt 2024-03-16T09:17:08.908 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/avic,tm070ddh03.txt 2024-03-16T09:17:08.909 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wa01a.txt 2024-03-16T09:17:08.910 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wb03.txt 2024-03-16T09:17:08.911 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et057090dhu.txt 2024-03-16T09:17:08.912 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et070080dh6.txt 2024-03-16T09:17:08.914 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,etm0700g0dh6.txt 2024-03-16T09:17:08.915 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/foxlink,fl500wvr00-a0t.txt 2024-03-16T09:17:08.916 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/giantplus,gpg482739qs5.txt 2024-03-16T09:17:08.917 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hannstar,hsd070pww1.txt 2024-03-16T09:17:08.918 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hit,tx23d38vm0caa.txt 2024-03-16T09:17:08.920 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,g121i1-l01.txt 2024-03-16T09:17:08.922 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n116bge.txt 2024-03-16T09:17:08.924 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n156bge-l21.txt 2024-03-16T09:17:08.926 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,ld070wx3-sl01.txt 2024-03-16T09:17:08.928 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lh500wx1-sd03.txt 2024-03-16T09:17:08.929 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lp129qe.txt 2024-03-16T09:17:08.931 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/panasonic,vvx10f004b00.txt 2024-03-16T09:17:08.933 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ld9040.txt 2024-03-16T09:17:08.935 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ltn101nt05.txt 2024-03-16T09:17:08.937 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,s6e8aa0.txt 2024-03-16T09:17:08.939 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/sharp,lq101r1sx01.txt 2024-03-16T09:17:08.940 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/simple-panel.txt 2024-03-16T09:17:08.942 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ 2024-03-16T09:17:08.944 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/83xx-512x-pci.txt 2024-03-16T09:17:08.946 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/designware-pcie.txt 2024-03-16T09:17:08.948 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.txt 2024-03-16T09:17:08.950 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,pci.txt 2024-03-16T09:17:08.951 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/host-generic-pci.txt 2024-03-16T09:17:08.953 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/layerscape-pci.txt 2024-03-16T09:17:08.955 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/mvebu-pci.txt 2024-03-16T09:17:08.958 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/nvidia,tegra20-pcie.txt 2024-03-16T09:17:08.960 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-keystone.txt 2024-03-16T09:17:08.962 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-rcar-gen2.txt 2024-03-16T09:17:08.963 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci.txt 2024-03-16T09:17:08.965 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ralink,rt3883-pci.txt 2024-03-16T09:17:08.969 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/rcar-pci.txt 2024-03-16T09:17:08.969 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/samsung,exynos5440-pcie.txt 2024-03-16T09:17:08.970 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/spear13xx-pcie.txt 2024-03-16T09:17:08.971 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ti-pci.txt 2024-03-16T09:17:08.972 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/v3-v360epc-pci.txt 2024-03-16T09:17:08.973 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/versatile.txt 2024-03-16T09:17:08.975 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xgene-pci.txt 2024-03-16T09:17:08.976 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xilinx-pcie.txt 2024-03-16T09:17:08.978 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ 2024-03-16T09:17:08.980 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/apm-xgene-phy.txt 2024-03-16T09:17:08.982 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/bcm-phy.txt 2024-03-16T09:17:08.983 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-sata-phy.txt 2024-03-16T09:17:08.985 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-usb-phy.txt 2024-03-16T09:17:08.987 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/hix5hd2-phy.txt 2024-03-16T09:17:08.989 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-bindings.txt 2024-03-16T09:17:08.991 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy28lp.txt 2024-03-16T09:17:08.993 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy365x.txt 2024-03-16T09:17:08.995 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-mvebu.txt 2024-03-16T09:17:08.997 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih407-usb.txt 2024-03-16T09:17:08.998 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih41x-usb.txt 2024-03-16T09:17:09.000 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-apq8064-sata-phy.txt 2024-03-16T09:17:09.002 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-dwc3-usb-phy.txt 2024-03-16T09:17:09.004 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-ipq806x-sata-phy.txt 2024-03-16T09:17:09.006 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rcar-gen2-phy.txt 2024-03-16T09:17:09.009 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rockchip-usb-phy.txt 2024-03-16T09:17:09.010 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/samsung-phy.txt 2024-03-16T09:17:09.011 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/st-spear-miphy.txt 2024-03-16T09:17:09.013 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/sun4i-usb-phy.txt 2024-03-16T09:17:09.015 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ti-phy.txt 2024-03-16T09:17:09.017 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ 2024-03-16T09:17:09.019 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/abilis,tb10x-iomux.txt 2024-03-16T09:17:09.021 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/allwinner,sunxi-pinctrl.txt 2024-03-16T09:17:09.023 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/atmel,at91-pinctrl.txt 2024-03-16T09:17:09.024 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm11351-pinctrl.txt 2024-03-16T09:17:09.026 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm2835-gpio.txt 2024-03-16T09:17:09.028 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx-pinctrl.txt 2024-03-16T09:17:09.029 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx25-pinctrl.txt 2024-03-16T09:17:09.031 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx27-pinctrl.txt 2024-03-16T09:17:09.033 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx35-pinctrl.txt 2024-03-16T09:17:09.034 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx51-pinctrl.txt 2024-03-16T09:17:09.036 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx53-pinctrl.txt 2024-03-16T09:17:09.037 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6dl-pinctrl.txt 2024-03-16T09:17:09.039 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6q-pinctrl.txt 2024-03-16T09:17:09.040 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sl-pinctrl.txt 2024-03-16T09:17:09.042 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sx-pinctrl.txt 2024-03-16T09:17:09.043 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,mxs-pinctrl.txt 2024-03-16T09:17:09.045 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,vf610-pinctrl.txt 2024-03-16T09:17:09.046 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pdc-pinctrl.txt 2024-03-16T09:17:09.048 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pinctrl.txt 2024-03-16T09:17:09.050 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,falcon-pinumx.txt 2024-03-16T09:17:09.051 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,xway-pinumx.txt 2024-03-16T09:17:09.053 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-370-pinctrl.txt 2024-03-16T09:17:09.054 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-375-pinctrl.txt 2024-03-16T09:17:09.056 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-38x-pinctrl.txt 2024-03-16T09:17:09.057 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-xp-pinctrl.txt 2024-03-16T09:17:09.059 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,dove-pinctrl.txt 2024-03-16T09:17:09.060 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,kirkwood-pinctrl.txt 2024-03-16T09:17:09.062 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,mvebu-pinctrl.txt 2024-03-16T09:17:09.063 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,orion-pinctrl.txt 2024-03-16T09:17:09.065 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/meson,pinctrl.txt 2024-03-16T09:17:09.067 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra114-pinmux.txt 2024-03-16T09:17:09.068 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-pinmux.txt 2024-03-16T09:17:09.069 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-xusb-padctl.txt 2024-03-16T09:17:09.071 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra20-pinmux.txt 2024-03-16T09:17:09.072 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra30-pinmux.txt 2024-03-16T09:17:09.074 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-bindings.txt 2024-03-16T09:17:09.076 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-palmas.txt 2024-03-16T09:17:09.078 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-single.txt 2024-03-16T09:17:09.080 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-sirf.txt 2024-03-16T09:17:09.083 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-st.txt 2024-03-16T09:17:09.085 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-vt8500.txt 2024-03-16T09:17:09.087 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl_spear.txt 2024-03-16T09:17:09.089 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8064-pinctrl.txt 2024-03-16T09:17:09.090 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8084-pinctrl.txt 2024-03-16T09:17:09.091 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,ipq8064-pinctrl.txt 2024-03-16T09:17:09.092 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8916-pinctrl.txt 2024-03-16T09:17:09.093 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8960-pinctrl.txt 2024-03-16T09:17:09.094 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8974-pinctrl.txt 2024-03-16T09:17:09.095 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-gpio.txt 2024-03-16T09:17:09.096 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-mpp.txt 2024-03-16T09:17:09.097 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/renesas,pfc-pinctrl.txt 2024-03-16T09:17:09.099 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/rockchip,pinctrl.txt 2024-03-16T09:17:09.100 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/samsung-pinctrl.txt 2024-03-16T09:17:09.102 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,abx500.txt 2024-03-16T09:17:09.103 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,nomadik.txt 2024-03-16T09:17:09.105 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ti,omap-pinctrl.txt 2024-03-16T09:17:09.106 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/xlnx,zynq-pinctrl.txt 2024-03-16T09:17:09.108 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ 2024-03-16T09:17:09.109 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/bq2415x.txt 2024-03-16T09:17:09.111 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/isp1704.txt 2024-03-16T09:17:09.113 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ltc2941.txt 2024-03-16T09:17:09.114 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/opp.txt 2024-03-16T09:17:09.116 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power-controller.txt 2024-03-16T09:17:09.117 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power_domain.txt 2024-03-16T09:17:09.119 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/renesas,sysc-rmobile.txt 2024-03-16T09:17:09.121 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ 2024-03-16T09:17:09.122 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/keystone-reset.txt 2024-03-16T09:17:09.123 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ltc2952-poweroff.txt 2024-03-16T09:17:09.124 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/st-reset.txt 2024-03-16T09:17:09.126 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/syscon-reboot.txt 2024-03-16T09:17:09.127 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rockchip-io-domain.txt 2024-03-16T09:17:09.129 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rx51-battery.txt 2024-03-16T09:17:09.130 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/twl-charger.txt 2024-03-16T09:17:09.132 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ 2024-03-16T09:17:09.133 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/ 2024-03-16T09:17:09.135 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/btemp.txt 2024-03-16T09:17:09.136 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/chargalg.txt 2024-03-16T09:17:09.138 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/charger.txt 2024-03-16T09:17:09.139 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/fg.txt 2024-03-16T09:17:09.141 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/axxia-reset.txt 2024-03-16T09:17:09.142 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/charger-manager.txt 2024-03-16T09:17:09.144 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/gpio-charger.txt 2024-03-16T09:17:09.148 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/imx-snvs-poweroff.txt 2024-03-16T09:17:09.150 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/lp8727_charger.txt 2024-03-16T09:17:09.152 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max17042_battery.txt 2024-03-16T09:17:09.153 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max8925_batter.txt 2024-03-16T09:17:09.155 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/msm-poweroff.txt 2024-03-16T09:17:09.157 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/olpc_battery.txt 2024-03-16T09:17:09.158 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/power_supply.txt 2024-03-16T09:17:09.159 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/qnap-poweroff.txt 2024-03-16T09:17:09.161 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/restart-poweroff.txt 2024-03-16T09:17:09.162 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/sbs_sbs-battery.txt 2024-03-16T09:17:09.164 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ti,bq24735.txt 2024-03-16T09:17:09.166 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/tps65090.txt 2024-03-16T09:17:09.167 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/ 2024-03-16T09:17:09.169 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ 2024-03-16T09:17:09.170 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/akebono.txt 2024-03-16T09:17:09.173 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/cpm.txt 2024-03-16T09:17:09.173 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/emac.txt 2024-03-16T09:17:09.175 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/hsta.txt 2024-03-16T09:17:09.176 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ndfc.txt 2024-03-16T09:17:09.177 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ppc440spe-adma.txt 2024-03-16T09:17:09.179 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/reboot.txt 2024-03-16T09:17:09.180 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ 2024-03-16T09:17:09.182 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/board.txt 2024-03-16T09:17:09.183 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cache_sram.txt 2024-03-16T09:17:09.184 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ccf.txt 2024-03-16T09:17:09.185 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/ 2024-03-16T09:17:09.186 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm.txt 2024-03-16T09:17:09.187 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/ 2024-03-16T09:17:09.188 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/brg.txt 2024-03-16T09:17:09.189 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/i2c.txt 2024-03-16T09:17:09.190 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/pic.txt 2024-03-16T09:17:09.192 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/usb.txt 2024-03-16T09:17:09.194 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/gpio.txt 2024-03-16T09:17:09.195 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/network.txt 2024-03-16T09:17:09.197 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe.txt 2024-03-16T09:17:09.198 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ 2024-03-16T09:17:09.199 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/firmware.txt 2024-03-16T09:17:09.200 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/par_io.txt 2024-03-16T09:17:09.201 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/pincfg.txt 2024-03-16T09:17:09.202 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ucc.txt 2024-03-16T09:17:09.203 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/usb.txt 2024-03-16T09:17:09.204 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/serial.txt 2024-03-16T09:17:09.205 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpus.txt 2024-03-16T09:17:09.206 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dcsr.txt 2024-03-16T09:17:09.206 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/diu.txt 2024-03-16T09:17:09.207 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dma.txt 2024-03-16T09:17:09.208 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ecm.txt 2024-03-16T09:17:09.209 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/fman.txt 2024-03-16T09:17:09.210 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/gtm.txt 2024-03-16T09:17:09.211 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/guts.txt 2024-03-16T09:17:09.212 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/interlaken-lac.txt 2024-03-16T09:17:09.213 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/l2cache.txt 2024-03-16T09:17:09.213 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/lbc.txt 2024-03-16T09:17:09.214 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcm.txt 2024-03-16T09:17:09.215 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcu-mpc8349emitx.txt 2024-03-16T09:17:09.216 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mem-ctrlr.txt 2024-03-16T09:17:09.217 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5121-psc.txt 2024-03-16T09:17:09.218 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5200.txt 2024-03-16T09:17:09.219 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-msgr.txt 2024-03-16T09:17:09.220 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-timer.txt 2024-03-16T09:17:09.221 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic.txt 2024-03-16T09:17:09.222 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/msi-pic.txt 2024-03-16T09:17:09.222 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pamu.txt 2024-03-16T09:17:09.223 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pmc.txt 2024-03-16T09:17:09.224 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/raideng.txt 2024-03-16T09:17:09.227 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio-rmu.txt 2024-03-16T09:17:09.227 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio.txt 2024-03-16T09:17:09.227 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/ 2024-03-16T09:17:09.228 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/gamecube.txt 2024-03-16T09:17:09.229 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/wii.txt 2024-03-16T09:17:09.230 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/ 2024-03-16T09:17:09.231 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/pps-gpio.txt 2024-03-16T09:17:09.232 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ 2024-03-16T09:17:09.233 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-hlcdc-pwm.txt 2024-03-16T09:17:09.233 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-pwm.txt 2024-03-16T09:17:09.234 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-tcb-pwm.txt 2024-03-16T09:17:09.235 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/bcm-kona-pwm.txt 2024-03-16T09:17:09.236 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/cirrus,clps711x-pwm.txt 2024-03-16T09:17:09.237 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/img-pwm.txt 2024-03-16T09:17:09.238 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/imx-pwm.txt 2024-03-16T09:17:09.239 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/lpc32xx-pwm.txt 2024-03-16T09:17:09.240 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/mxs-pwm.txt 2024-03-16T09:17:09.241 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nvidia,tegra20-pwm.txt 2024-03-16T09:17:09.242 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nxp,pca9685-pwm.txt 2024-03-16T09:17:09.243 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-bcm2835.txt 2024-03-16T09:17:09.244 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-fsl-ftm.txt 2024-03-16T09:17:09.245 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-lp3943.txt 2024-03-16T09:17:09.246 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-rockchip.txt 2024-03-16T09:17:09.247 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-samsung.txt 2024-03-16T09:17:09.248 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-st.txt 2024-03-16T09:17:09.249 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-sun4i.txt 2024-03-16T09:17:09.249 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiecap.txt 2024-03-16T09:17:09.250 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiehrpwm.txt 2024-03-16T09:17:09.251 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tipwmss.txt 2024-03-16T09:17:09.252 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm.txt 2024-03-16T09:17:09.253 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pxa-pwm.txt 2024-03-16T09:17:09.254 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/renesas,tpu-pwm.txt 2024-03-16T09:17:09.255 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/spear-pwm.txt 2024-03-16T09:17:09.256 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwm.txt 2024-03-16T09:17:09.257 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwmled.txt 2024-03-16T09:17:09.258 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/vt8500-pwm.txt 2024-03-16T09:17:09.259 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/ 2024-03-16T09:17:09.261 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/regmap.txt 2024-03-16T09:17:09.262 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ 2024-03-16T09:17:09.263 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm800.txt 2024-03-16T09:17:09.264 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm860x.txt 2024-03-16T09:17:09.265 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/act8865-regulator.txt 2024-03-16T09:17:09.266 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/anatop-regulator.txt 2024-03-16T09:17:09.267 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/as3722-regulator.txt 2024-03-16T09:17:09.268 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9210.txt 2024-03-16T09:17:09.269 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9211.txt 2024-03-16T09:17:09.270 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fan53555.txt 2024-03-16T09:17:09.271 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fixed-regulator.txt 2024-03-16T09:17:09.272 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/gpio-regulator.txt 2024-03-16T09:17:09.273 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/isl9305.txt 2024-03-16T09:17:09.274 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/lp872x.txt 2024-03-16T09:17:09.275 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ltc3589.txt 2024-03-16T09:17:09.276 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max1586-regulator.txt 2024-03-16T09:17:09.277 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max77802.txt 2024-03-16T09:17:09.278 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8660.txt 2024-03-16T09:17:09.279 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8907.txt 2024-03-16T09:17:09.280 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8925-regulator.txt 2024-03-16T09:17:09.281 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8952.txt 2024-03-16T09:17:09.282 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8973-regulator.txt 2024-03-16T09:17:09.283 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8997-regulator.txt 2024-03-16T09:17:09.284 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/mt6397-regulator.txt 2024-03-16T09:17:09.285 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/palmas-pmic.txt 2024-03-16T09:17:09.286 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pbias-regulator.txt 2024-03-16T09:17:09.287 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pfuze100.txt 2024-03-16T09:17:09.288 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pwm-regulator.txt 2024-03-16T09:17:09.289 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/regulator.txt 2024-03-16T09:17:09.290 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/s5m8767-regulator.txt 2024-03-16T09:17:09.291 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/sky81452-regulator.txt 2024-03-16T09:17:09.292 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ti-abb-regulator.txt 2024-03-16T09:17:09.293 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps51632-regulator.txt 2024-03-16T09:17:09.294 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps62360-regulator.txt 2024-03-16T09:17:09.295 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65090.txt 2024-03-16T09:17:09.296 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65217.txt 2024-03-16T09:17:09.297 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65218.txt 2024-03-16T09:17:09.298 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps6586x.txt 2024-03-16T09:17:09.299 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/twl-regulator.txt 2024-03-16T09:17:09.300 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/vexpress.txt 2024-03-16T09:17:09.301 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/ 2024-03-16T09:17:09.302 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt 2024-03-16T09:17:09.303 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/ 2024-03-16T09:17:09.304 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/allwinner,sunxi-clock-reset.txt 2024-03-16T09:17:09.305 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/fsl,imx-src.txt 2024-03-16T09:17:09.306 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/reset.txt 2024-03-16T09:17:09.307 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/sirf,rstc.txt 2024-03-16T09:17:09.308 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/socfpga-reset.txt 2024-03-16T09:17:09.309 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-picophyreset.txt 2024-03-16T09:17:09.310 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-powerdown.txt 2024-03-16T09:17:09.312 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-softreset.txt 2024-03-16T09:17:09.313 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/resource-names.txt 2024-03-16T09:17:09.314 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/ 2024-03-16T09:17:09.315 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/apm,rng.txt 2024-03-16T09:17:09.316 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/brcm,bcm2835.txt 2024-03-16T09:17:09.317 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/qcom,prng.txt 2024-03-16T09:17:09.318 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/ 2024-03-16T09:17:09.319 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/armada-380-rtc.txt 2024-03-16T09:17:09.321 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91rm9200-rtc.txt 2024-03-16T09:17:09.322 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91sam9-rtc.txt 2024-03-16T09:17:09.323 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dallas,ds1339.txt 2024-03-16T09:17:09.324 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dw-apb.txt 2024-03-16T09:17:09.325 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/haoyu,hym8563.txt 2024-03-16T09:17:09.326 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/imxdi-rtc.txt 2024-03-16T09:17:09.327 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/isil,isl12057.txt 2024-03-16T09:17:09.328 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/lpc32xx-rtc.txt 2024-03-16T09:17:09.329 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/maxim,ds1742.txt 2024-03-16T09:17:09.331 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/moxa,moxart-rtc.txt 2024-03-16T09:17:09.332 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nvidia,tegra20-rtc.txt 2024-03-16T09:17:09.332 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nxp,rtc-2123.txt 2024-03-16T09:17:09.333 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/olpc-xo1-rtc.txt 2024-03-16T09:17:09.334 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/orion-rtc.txt 2024-03-16T09:17:09.335 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/pxa-rtc.txt 2024-03-16T09:17:09.336 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-cmos.txt 2024-03-16T09:17:09.337 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-omap.txt 2024-03-16T09:17:09.338 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-opal.txt 2024-03-16T09:17:09.340 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-palmas.txt 2024-03-16T09:17:09.341 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/s3c-rtc.txt 2024-03-16T09:17:09.342 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sa1100-rtc.txt 2024-03-16T09:17:09.343 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/snvs-rtc.txt 2024-03-16T09:17:09.344 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/spear-rtc.txt 2024-03-16T09:17:09.345 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/stmp3xxx-rtc.txt 2024-03-16T09:17:09.346 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sun6i-rtc.txt 2024-03-16T09:17:09.347 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sunxi-rtc.txt 2024-03-16T09:17:09.348 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/twl-rtc.txt 2024-03-16T09:17:09.349 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/via,vt8500-rtc.txt 2024-03-16T09:17:09.350 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/xgene-rtc.txt 2024-03-16T09:17:09.351 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/ 2024-03-16T09:17:09.353 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/ 2024-03-16T09:17:09.353 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/st33zp24-i2c.txt 2024-03-16T09:17:09.355 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/ 2024-03-16T09:17:09.356 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/8250.txt 2024-03-16T09:17:09.356 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_jtaguart.txt 2024-03-16T09:17:09.357 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_uart.txt 2024-03-16T09:17:09.358 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/arc-uart.txt 2024-03-16T09:17:09.359 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/atmel-usart.txt 2024-03-16T09:17:09.360 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/axis,etraxfs-uart.txt 2024-03-16T09:17:09.361 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/bcm63xx-uart.txt 2024-03-16T09:17:09.363 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cavium-uart.txt 2024-03-16T09:17:09.364 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cdns,uart.txt 2024-03-16T09:17:09.365 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cirrus,clps711x-uart.txt 2024-03-16T09:17:09.366 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/digicolor-usart.txt 2024-03-16T09:17:09.367 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/efm32-uart.txt 2024-03-16T09:17:09.368 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-imx-uart.txt 2024-03-16T09:17:09.369 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-lpuart.txt 2024-03-16T09:17:09.370 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-mxs-auart.txt 2024-03-16T09:17:09.371 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/lantiq_asc.txt 2024-03-16T09:17:09.372 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/maxim,max310x.txt 2024-03-16T09:17:09.373 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl,pxa-ssp.txt 2024-03-16T09:17:09.374 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl-serial.txt 2024-03-16T09:17:09.377 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mtk-uart.txt 2024-03-16T09:17:09.378 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nvidia,tegra20-hsuart.txt 2024-03-16T09:17:09.378 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp,sc16is7xx.txt 2024-03-16T09:17:09.378 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp-lpc32xx-hsuart.txt 2024-03-16T09:17:09.379 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/omap_serial.txt 2024-03-16T09:17:09.380 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/pl011.txt 2024-03-16T09:17:09.381 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qca,ar9330-uart.txt 2024-03-16T09:17:09.382 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uart.txt 2024-03-16T09:17:09.383 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uartdm.txt 2024-03-16T09:17:09.385 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/renesas,sci-serial.txt 2024-03-16T09:17:09.386 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/rs485.txt 2024-03-16T09:17:09.387 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/samsung_uart.txt 2024-03-16T09:17:09.388 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sirf-uart.txt 2024-03-16T09:17:09.389 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/snps-dw-apb-uart.txt 2024-03-16T09:17:09.390 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sprd-uart.txt 2024-03-16T09:17:09.391 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/st-asc.txt 2024-03-16T09:17:09.392 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/vt8500-uart.txt 2024-03-16T09:17:09.393 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/ 2024-03-16T09:17:09.394 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/allwinner,sun4i-ps2.txt 2024-03-16T09:17:09.395 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/altera_ps2.txt 2024-03-16T09:17:09.396 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/olpc,ap-sp.txt 2024-03-16T09:17:09.397 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/snps-arc_ps2.txt 2024-03-16T09:17:09.398 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ 2024-03-16T09:17:09.399 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/ 2024-03-16T09:17:09.400 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman-portals.txt 2024-03-16T09:17:09.401 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman.txt 2024-03-16T09:17:09.402 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman-portals.txt 2024-03-16T09:17:09.403 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman.txt 2024-03-16T09:17:09.404 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/ 2024-03-16T09:17:09.405 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/qcom,gsbi.txt 2024-03-16T09:17:09.406 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/ 2024-03-16T09:17:09.407 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-dma.txt 2024-03-16T09:17:09.408 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-qmss.txt 2024-03-16T09:17:09.409 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ 2024-03-16T09:17:09.411 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,adau1701.txt 2024-03-16T09:17:09.411 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-i2s.txt 2024-03-16T09:17:09.412 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-spdif-tx.txt 2024-03-16T09:17:09.413 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,ssm2602.txt 2024-03-16T09:17:09.414 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4104.txt 2024-03-16T09:17:09.415 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4554.c 2024-03-16T09:17:09.416 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4642.txt 2024-03-16T09:17:09.417 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak5386.txt 2024-03-16T09:17:09.418 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5623.txt 2024-03-16T09:17:09.419 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5632.txt 2024-03-16T09:17:09.421 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/armada-370db-audio.txt 2024-03-16T09:17:09.421 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/arndale.txt 2024-03-16T09:17:09.423 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-at91sam9g20ek-wm8731-audio.txt 2024-03-16T09:17:09.423 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-sam9x5-wm8731-audio.txt 2024-03-16T09:17:09.424 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-wm8904.txt 2024-03-16T09:17:09.425 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel_ac97c.txt 2024-03-16T09:17:09.426 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/bcm2835-i2s.txt 2024-03-16T09:17:09.427 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cdns,xtfpga-i2s.txt 2024-03-16T09:17:09.428 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs35l32.txt 2024-03-16T09:17:09.429 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4265.txt 2024-03-16T09:17:09.430 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4270.txt 2024-03-16T09:17:09.431 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4271.txt 2024-03-16T09:17:09.432 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l52.txt 2024-03-16T09:17:09.433 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l56.txt 2024-03-16T09:17:09.434 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l73.txt 2024-03-16T09:17:09.435 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42xx8.txt 2024-03-16T09:17:09.436 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/da9055.txt 2024-03-16T09:17:09.437 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-evm-audio.txt 2024-03-16T09:17:09.438 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-mcasp-audio.txt 2024-03-16T09:17:09.439 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/designware-i2s.txt 2024-03-16T09:17:09.441 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/es8328.txt 2024-03-16T09:17:09.442 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/eukrea-tlv320.txt 2024-03-16T09:17:09.443 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,asrc.txt 2024-03-16T09:17:09.444 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,esai.txt 2024-03-16T09:17:09.445 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,spdif.txt 2024-03-16T09:17:09.446 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,ssi.txt 2024-03-16T09:17:09.448 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-asoc-card.txt 2024-03-16T09:17:09.449 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-sai.txt 2024-03-16T09:17:09.450 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/hdmi.txt 2024-03-16T09:17:09.452 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-es8328.txt 2024-03-16T09:17:09.454 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-sgtl5000.txt 2024-03-16T09:17:09.455 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-spdif.txt 2024-03-16T09:17:09.457 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-wm8962.txt 2024-03-16T09:17:09.459 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audmux.txt 2024-03-16T09:17:09.460 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ingenic,jz4740-i2s.txt 2024-03-16T09:17:09.462 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98090.txt 2024-03-16T09:17:09.464 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98095.txt 2024-03-16T09:17:09.465 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98357a.txt 2024-03-16T09:17:09.467 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa-ssp.txt 2024-03-16T09:17:09.468 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa2xx-pcm.txt 2024-03-16T09:17:09.470 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mvebu-audio.txt 2024-03-16T09:17:09.472 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-audio-sgtl5000.txt 2024-03-16T09:17:09.473 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-saif.txt 2024-03-16T09:17:09.475 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nokia,rx51.txt 2024-03-16T09:17:09.477 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-alc5632.txt 2024-03-16T09:17:09.478 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-max98090.txt 2024-03-16T09:17:09.479 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5640.txt 2024-03-16T09:17:09.480 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5677.txt 2024-03-16T09:17:09.481 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-trimslice.txt 2024-03-16T09:17:09.482 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8753.txt 2024-03-16T09:17:09.483 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8903.txt 2024-03-16T09:17:09.484 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm9712.txt 2024-03-16T09:17:09.485 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-ac97.txt 2024-03-16T09:17:09.486 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-das.txt 2024-03-16T09:17:09.488 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-i2s.txt 2024-03-16T09:17:09.488 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-ahub.txt 2024-03-16T09:17:09.489 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-hda.txt 2024-03-16T09:17:09.491 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-i2s.txt 2024-03-16T09:17:09.492 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-abe-twl6040.txt 2024-03-16T09:17:09.493 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-dmic.txt 2024-03-16T09:17:09.494 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcbsp.txt 2024-03-16T09:17:09.495 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcpdm.txt 2024-03-16T09:17:09.496 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-twl4030.txt 2024-03-16T09:17:09.497 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm1792a.txt 2024-03-16T09:17:09.498 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm512x.txt 2024-03-16T09:17:09.499 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,fsi.txt 2024-03-16T09:17:09.501 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,rsnd.txt 2024-03-16T09:17:09.502 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rockchip-i2s.txt 2024-03-16T09:17:09.502 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5631.txt 2024-03-16T09:17:09.505 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5640.txt 2024-03-16T09:17:09.505 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5677.txt 2024-03-16T09:17:09.506 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,odroidx2-max98090.txt 2024-03-16T09:17:09.506 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,smdk-wm8994.txt 2024-03-16T09:17:09.507 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung-i2s.txt 2024-03-16T09:17:09.508 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sgtl5000.txt 2024-03-16T09:17:09.510 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/simple-card.txt 2024-03-16T09:17:09.510 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-codec.txt 2024-03-16T09:17:09.511 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-port.txt 2024-03-16T09:17:09.512 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio.txt 2024-03-16T09:17:09.513 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-usp.txt 2024-03-16T09:17:09.514 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/snow.txt 2024-03-16T09:17:09.516 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/soc-ac97link.txt 2024-03-16T09:17:09.516 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-receiver.txt 2024-03-16T09:17:09.517 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-transmitter.txt 2024-03-16T09:17:09.518 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm2518.txt 2024-03-16T09:17:09.519 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm4567.txt 2024-03-16T09:17:09.520 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta32x.txt 2024-03-16T09:17:09.521 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta350.txt 2024-03-16T09:17:09.522 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tas2552.txt 2024-03-16T09:17:09.523 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tdm-slot.txt 2024-03-16T09:17:09.524 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,pcm1681.txt 2024-03-16T09:17:09.525 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,tas5086.txt 2024-03-16T09:17:09.526 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic31xx.txt 2024-03-16T09:17:09.527 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic32x4.txt 2024-03-16T09:17:09.528 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic3x.txt 2024-03-16T09:17:09.529 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tpa6130a2.txt 2024-03-16T09:17:09.530 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ts3a227e.txt 2024-03-16T09:17:09.531 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-mop500.txt 2024-03-16T09:17:09.532 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-msp.txt 2024-03-16T09:17:09.533 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/widgets.txt 2024-03-16T09:17:09.535 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8510.txt 2024-03-16T09:17:09.536 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8523.txt 2024-03-16T09:17:09.537 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8580.txt 2024-03-16T09:17:09.538 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8711.txt 2024-03-16T09:17:09.539 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8728.txt 2024-03-16T09:17:09.540 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8731.txt 2024-03-16T09:17:09.541 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8737.txt 2024-03-16T09:17:09.542 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8741.txt 2024-03-16T09:17:09.543 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8750.txt 2024-03-16T09:17:09.545 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8753.txt 2024-03-16T09:17:09.545 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8770.txt 2024-03-16T09:17:09.546 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8776.txt 2024-03-16T09:17:09.547 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8804.txt 2024-03-16T09:17:09.548 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8903.txt 2024-03-16T09:17:09.549 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8904.txt 2024-03-16T09:17:09.551 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8960.txt 2024-03-16T09:17:09.552 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8962.txt 2024-03-16T09:17:09.553 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8994.txt 2024-03-16T09:17:09.554 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ 2024-03-16T09:17:09.555 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/brcm,bcm2835-spi.txt 2024-03-16T09:17:09.556 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/efm32-spi.txt 2024-03-16T09:17:09.557 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-imx-cspi.txt 2024-03-16T09:17:09.558 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-spi.txt 2024-03-16T09:17:09.559 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/mxs-spi.txt 2024-03-16T09:17:09.560 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra114-spi.txt 2024-03-16T09:17:09.561 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-sflash.txt 2024-03-16T09:17:09.562 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-slink.txt 2024-03-16T09:17:09.563 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/omap-spi.txt 2024-03-16T09:17:09.564 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/qcom,spi-qup.txt 2024-03-16T09:17:09.565 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-hspi.txt 2024-03-16T09:17:09.567 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-msiof.txt 2024-03-16T09:17:09.568 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/snps,dw-apb-ssi.txt 2024-03-16T09:17:09.569 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-bus.txt 2024-03-16T09:17:09.570 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-cadence.txt 2024-03-16T09:17:09.572 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-davinci.txt 2024-03-16T09:17:09.573 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-dw.txt 2024-03-16T09:17:09.575 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-fsl-dspi.txt 2024-03-16T09:17:09.577 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-gpio.txt 2024-03-16T09:17:09.578 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-img-spfi.txt 2024-03-16T09:17:09.581 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-meson.txt 2024-03-16T09:17:09.582 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-octeon.txt 2024-03-16T09:17:09.584 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-orion.txt 2024-03-16T09:17:09.585 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rockchip.txt 2024-03-16T09:17:09.586 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rspi.txt 2024-03-16T09:17:09.588 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-samsung.txt 2024-03-16T09:17:09.588 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sc18is602.txt 2024-03-16T09:17:09.589 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sirf.txt 2024-03-16T09:17:09.590 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-st-ssc.txt 2024-03-16T09:17:09.592 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun4i.txt 2024-03-16T09:17:09.593 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun6i.txt 2024-03-16T09:17:09.593 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-xtensa-xtfpga.txt 2024-03-16T09:17:09.595 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_altera.txt 2024-03-16T09:17:09.596 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_atmel.txt 2024-03-16T09:17:09.597 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_oc_tiny.txt 2024-03-16T09:17:09.598 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_pl022.txt 2024-03-16T09:17:09.599 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ti_qspi.txt 2024-03-16T09:17:09.600 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/ 2024-03-16T09:17:09.602 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/qcom,spmi-pmic-arb.txt 2024-03-16T09:17:09.602 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/spmi.txt 2024-03-16T09:17:09.604 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/ 2024-03-16T09:17:09.605 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/ 2024-03-16T09:17:09.606 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/ 2024-03-16T09:17:09.607 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/lpc32xx-adc.txt 2024-03-16T09:17:09.608 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/mxs-lradc.txt 2024-03-16T09:17:09.610 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/spear-adc.txt 2024-03-16T09:17:09.612 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/submitting-patches.txt 2024-03-16T09:17:09.614 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ 2024-03-16T09:17:09.616 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/armada-thermal.txt 2024-03-16T09:17:09.618 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/db8500-thermal.txt 2024-03-16T09:17:09.619 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/dove-thermal.txt 2024-03-16T09:17:09.621 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/exynos-thermal.txt 2024-03-16T09:17:09.622 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/imx-thermal.txt 2024-03-16T09:17:09.624 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/kirkwood-thermal.txt 2024-03-16T09:17:09.626 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rcar-thermal.txt 2024-03-16T09:17:09.627 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rockchip-thermal.txt 2024-03-16T09:17:09.629 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/spear-thermal.txt 2024-03-16T09:17:09.631 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/st-thermal.txt 2024-03-16T09:17:09.633 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/tegra-soctherm.txt 2024-03-16T09:17:09.633 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/thermal.txt 2024-03-16T09:17:09.634 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ti_soc_thermal.txt 2024-03-16T09:17:09.635 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ 2024-03-16T09:17:09.636 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun4i-timer.txt 2024-03-16T09:17:09.637 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun5i-a13-hstimer.txt 2024-03-16T09:17:09.638 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/amlogic,meson6-timer.txt 2024-03-16T09:17:09.640 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/arm,sp804.txt 2024-03-16T09:17:09.640 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/brcm,bcm2835-system-timer.txt 2024-03-16T09:17:09.641 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cadence,ttc-timer.txt 2024-03-16T09:17:09.642 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cirrus,clps711x-timer.txt 2024-03-16T09:17:09.643 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/digicolor-timer.txt 2024-03-16T09:17:09.644 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/energymicro,efm32-timer.txt 2024-03-16T09:17:09.645 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,ftm-timer.txt 2024-03-16T09:17:09.646 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,imxgpt.txt 2024-03-16T09:17:09.647 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/lsi,zevio-timer.txt 2024-03-16T09:17:09.649 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,armada-370-xp-timer.txt 2024-03-16T09:17:09.650 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,orion-timer.txt 2024-03-16T09:17:09.651 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/mediatek,mtk-timer.txt 2024-03-16T09:17:09.652 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/moxa,moxart-timer.txt 2024-03-16T09:17:09.653 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra20-timer.txt 2024-03-16T09:17:09.654 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra30-timer.txt 2024-03-16T09:17:09.655 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,cmt.txt 2024-03-16T09:17:09.656 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,mtu2.txt 2024-03-16T09:17:09.657 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,tmu.txt 2024-03-16T09:17:09.659 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/rockchip,rk3288-timer.txt 2024-03-16T09:17:09.660 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/samsung,exynos4210-mct.txt 2024-03-16T09:17:09.661 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/stericsson-u300-apptimer.txt 2024-03-16T09:17:09.662 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ti,keystone-timer.txt 2024-03-16T09:17:09.663 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ 2024-03-16T09:17:09.664 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ufshcd-pltfrm.txt 2024-03-16T09:17:09.666 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/unittest.txt 2024-03-16T09:17:09.668 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ 2024-03-16T09:17:09.669 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/am33xx-usb.txt 2024-03-16T09:17:09.671 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/atmel-usb.txt 2024-03-16T09:17:09.673 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-imx.txt 2024-03-16T09:17:09.675 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-qcom.txt 2024-03-16T09:17:09.676 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-usb2.txt 2024-03-16T09:17:09.678 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-zevio.txt 2024-03-16T09:17:09.680 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc2.txt 2024-03-16T09:17:09.681 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3-st.txt 2024-03-16T09:17:09.683 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3.txt 2024-03-16T09:17:09.685 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-omap.txt 2024-03-16T09:17:09.687 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-orion.txt 2024-03-16T09:17:09.688 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-st.txt 2024-03-16T09:17:09.690 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/exynos-usb.txt 2024-03-16T09:17:09.692 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/fsl-usb.txt 2024-03-16T09:17:09.693 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/generic.txt 2024-03-16T09:17:09.695 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/gr-udc.txt 2024-03-16T09:17:09.697 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/isp1301.txt 2024-03-16T09:17:09.699 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-phy.txt 2024-03-16T09:17:09.700 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-usb.txt 2024-03-16T09:17:09.702 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/lpc32xx-udc.txt 2024-03-16T09:17:09.704 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/msm-hsusb.txt 2024-03-16T09:17:09.705 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/mxs-phy.txt 2024-03-16T09:17:09.707 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-ehci.txt 2024-03-16T09:17:09.709 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-usb-phy.txt 2024-03-16T09:17:09.711 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-nxp.txt 2024-03-16T09:17:09.712 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-omap3.txt 2024-03-16T09:17:09.715 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-st.txt 2024-03-16T09:17:09.716 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/omap-usb.txt 2024-03-16T09:17:09.718 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/pxa-usb.txt 2024-03-16T09:17:09.719 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/qcom,dwc3.txt 2024-03-16T09:17:09.721 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/renesas_usbhs.txt 2024-03-16T09:17:09.723 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-hsotg.txt 2024-03-16T09:17:09.725 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-usbphy.txt 2024-03-16T09:17:09.727 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/spear-usb.txt 2024-03-16T09:17:09.728 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/twlxxxx-usb.txt 2024-03-16T09:17:09.729 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/udc-xilinx.txt 2024-03-16T09:17:09.730 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ehci.txt 2024-03-16T09:17:09.731 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-nop-xceiv.txt 2024-03-16T09:17:09.732 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ohci.txt 2024-03-16T09:17:09.734 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-uhci.txt 2024-03-16T09:17:09.734 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-xhci.txt 2024-03-16T09:17:09.735 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb3503.txt 2024-03-16T09:17:09.736 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usbmisc-imx.txt 2024-03-16T09:17:09.737 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ux500-usb.txt 2024-03-16T09:17:09.738 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/vendor-prefixes.txt 2024-03-16T09:17:09.739 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ 2024-03-16T09:17:09.740 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7123.txt 2024-03-16T09:17:09.741 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7511.txt 2024-03-16T09:17:09.742 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/analog-tv-connector.txt 2024-03-16T09:17:09.743 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/arm,pl11x.txt 2024-03-16T09:17:09.744 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/atmel,lcdc.txt 2024-03-16T09:17:09.745 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/ 2024-03-16T09:17:09.746 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/88pm860x.txt 2024-03-16T09:17:09.747 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/gpio-backlight.txt 2024-03-16T09:17:09.748 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/lp855x.txt 2024-03-16T09:17:09.749 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/max8925-backlight.txt 2024-03-16T09:17:09.750 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/pwm-backlight.txt 2024-03-16T09:17:09.751 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/tps65217-backlight.txt 2024-03-16T09:17:09.752 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ 2024-03-16T09:17:09.753 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ps8622.txt 2024-03-16T09:17:09.754 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ptn3460.txt 2024-03-16T09:17:09.755 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/cirrus,clps711x-fb.txt 2024-03-16T09:17:09.756 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/display-timing.txt 2024-03-16T09:17:09.757 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dvi-connector.txt 2024-03-16T09:17:09.758 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dw_hdmi-rockchip.txt 2024-03-16T09:17:09.759 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos7-decon.txt 2024-03-16T09:17:09.760 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dp.txt 2024-03-16T09:17:09.762 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dsim.txt 2024-03-16T09:17:09.764 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmi.txt 2024-03-16T09:17:09.765 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiddc.txt 2024-03-16T09:17:09.767 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiphy.txt 2024-03-16T09:17:09.769 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_mixer.txt 2024-03-16T09:17:09.770 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/fsl,imx-fb.txt 2024-03-16T09:17:09.772 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/hdmi-connector.txt 2024-03-16T09:17:09.773 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/lgphilips,lb035q02.txt 2024-03-16T09:17:09.775 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dpi.txt 2024-03-16T09:17:09.776 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dsi-cm.txt 2024-03-16T09:17:09.777 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/renesas,du.txt 2024-03-16T09:17:09.778 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-drm.txt 2024-03-16T09:17:09.779 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-vop.txt 2024-03-16T09:17:10.217 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/samsung-fimd.txt 2024-03-16T09:17:10.218 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sharp,ls037v7dw01.txt 2024-03-16T09:17:10.220 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer-sunxi.txt 2024-03-16T09:17:10.222 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer.txt 2024-03-16T09:17:10.223 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sony,acx565akm.txt 2024-03-16T09:17:10.225 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1289fb.txt 2024-03-16T09:17:10.227 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1307fb.txt 2024-03-16T09:17:10.228 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/thine,thc63lvdm83d 2024-03-16T09:17:10.230 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,dra7-dss.txt 2024-03-16T09:17:10.232 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap-dss.txt 2024-03-16T09:17:10.233 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap2-dss.txt 2024-03-16T09:17:10.235 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap3-dss.txt 2024-03-16T09:17:10.237 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap4-dss.txt 2024-03-16T09:17:10.239 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap5-dss.txt 2024-03-16T09:17:10.240 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,opa362.txt 2024-03-16T09:17:10.242 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tfp410.txt 2024-03-16T09:17:10.243 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tpd12s015.txt 2024-03-16T09:17:10.245 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/toppoly,td028ttec1.txt 2024-03-16T09:17:10.247 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/tpo,td043mtea1.txt 2024-03-16T09:17:10.248 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/vga-connector.txt 2024-03-16T09:17:10.250 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/via,vt8500-fb.txt 2024-03-16T09:17:10.251 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,prizm-ge-rops.txt 2024-03-16T09:17:10.253 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,wm8505-fb.txt 2024-03-16T09:17:10.255 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/ 2024-03-16T09:17:10.256 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/mmio.txt 2024-03-16T09:17:10.258 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/ 2024-03-16T09:17:10.259 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/fsl-imx-owire.txt 2024-03-16T09:17:10.261 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/omap-hdq.txt 2024-03-16T09:17:10.262 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/w1-gpio.txt 2024-03-16T09:17:10.264 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ 2024-03-16T09:17:10.268 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-at91rm9200-wdt.txt 2024-03-16T09:17:10.269 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-wdt.txt 2024-03-16T09:17:10.271 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/brcm,bcm2835-pm-wdog.txt 2024-03-16T09:17:10.272 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/cadence-wdt.txt 2024-03-16T09:17:10.274 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/davinci-wdt.txt 2024-03-16T09:17:10.275 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/dw_wdt.txt 2024-03-16T09:17:10.277 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/fsl-imx-wdt.txt 2024-03-16T09:17:10.279 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/gpio-wdt.txt 2024-03-16T09:17:10.280 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/imgpdc-wdt.txt 2024-03-16T09:17:10.282 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ingenic,jz4740-wdt.txt 2024-03-16T09:17:10.284 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/marvel.txt 2024-03-16T09:17:10.285 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/men-a021-wdt.txt 2024-03-16T09:17:10.287 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/meson6-wdt.txt 2024-03-16T09:17:10.289 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/moxa,moxart-watchdog.txt 2024-03-16T09:17:10.290 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/mtk-wdt.txt 2024-03-16T09:17:10.292 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/of-xilinx-wdt.txt 2024-03-16T09:17:10.293 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/omap-wdt.txt 2024-03-16T09:17:10.295 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/pnx4008-wdt.txt 2024-03-16T09:17:10.297 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qca-ar7130-wdt.txt 2024-03-16T09:17:10.299 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qcom-wdt.txt 2024-03-16T09:17:10.300 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/rt2880-wdt.txt 2024-03-16T09:17:10.302 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/samsung-wdt.txt 2024-03-16T09:17:10.304 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sirfsoc_wdt.txt 2024-03-16T09:17:10.305 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/stericsson-coh901327.txt 2024-03-16T09:17:10.307 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sunxi-wdt.txt 2024-03-16T09:17:10.308 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/twl4030-wdt.txt 2024-03-16T09:17:10.310 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ 2024-03-16T09:17:10.312 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ce4100.txt 2024-03-16T09:17:10.313 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/interrupt.txt 2024-03-16T09:17:10.315 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/timer.txt 2024-03-16T09:17:10.316 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/xilinx.txt 2024-03-16T09:17:10.318 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/ 2024-03-16T09:17:10.320 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/xillybus.txt 2024-03-16T09:17:10.321 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/booting-without-of.txt 2024-03-16T09:17:10.323 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/changesets.txt 2024-03-16T09:17:10.325 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/dynamic-resolution-notes.txt 2024-03-16T09:17:10.327 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/of_selftest.txt 2024-03-16T09:17:10.328 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/overlay-notes.txt 2024-03-16T09:17:10.330 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/todo.txt 2024-03-16T09:17:10.331 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/usage-model.txt 2024-03-16T09:17:10.333 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/digsig.txt 2024-03-16T09:17:10.335 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dma-buf-sharing.txt 2024-03-16T09:17:10.337 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dmaengine/ 2024-03-16T09:17:10.338 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dmaengine/00-INDEX 2024-03-16T09:17:10.340 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dmaengine/client.txt 2024-03-16T09:17:10.341 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dmaengine/dmatest.txt 2024-03-16T09:17:10.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:10 smithi112 ceph-mon[32307]: pgmap v396: 97 pgs: 97 active+clean; 35 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.9 MiB/s wr, 21 op/s 2024-03-16T09:17:10.343 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dmaengine/provider.txt 2024-03-16T09:17:10.344 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dontdiff 2024-03-16T09:17:10.346 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/ 2024-03-16T09:17:10.348 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/binding.txt 2024-03-16T09:17:10.349 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/bus.txt 2024-03-16T09:17:10.350 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/class.txt 2024-03-16T09:17:10.352 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/design-patterns.txt 2024-03-16T09:17:10.354 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/device.txt 2024-03-16T09:17:10.355 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/devres.txt 2024-03-16T09:17:10.357 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/driver.txt 2024-03-16T09:17:10.359 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/overview.txt 2024-03-16T09:17:10.360 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/platform.txt 2024-03-16T09:17:10.362 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/porting.txt 2024-03-16T09:17:10.363 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/ 2024-03-16T09:17:10.365 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/README.dvb-usb 2024-03-16T09:17:10.367 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/avermedia.txt 2024-03-16T09:17:10.368 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/bt8xx.txt 2024-03-16T09:17:10.370 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/cards.txt 2024-03-16T09:17:10.371 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/ci.txt 2024-03-16T09:17:10.373 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/contributors.txt 2024-03-16T09:17:10.374 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/faq.txt 2024-03-16T09:17:10.376 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/get_dvb_firmware 2024-03-16T09:17:10.377 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/lmedm04.txt 2024-03-16T09:17:10.379 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/opera-firmware.txt 2024-03-16T09:17:10.380 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/readme.txt 2024-03-16T09:17:10.382 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/technisat.txt 2024-03-16T09:17:10.383 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/ttusb-dec.txt 2024-03-16T09:17:10.385 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/udev.txt 2024-03-16T09:17:10.386 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dynamic-debug-howto.txt 2024-03-16T09:17:10.388 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/early-userspace/ 2024-03-16T09:17:10.390 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/early-userspace/README 2024-03-16T09:17:10.391 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/early-userspace/buffer-format.txt 2024-03-16T09:17:10.393 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/edac.txt 2024-03-16T09:17:10.398 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/efi-stub.txt 2024-03-16T09:17:10.398 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/eisa.txt 2024-03-16T09:17:10.398 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/email-clients.txt 2024-03-16T09:17:10.400 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/extcon/ 2024-03-16T09:17:10.401 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/extcon/porting-android-switch-class 2024-03-16T09:17:10.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:10 smithi194 ceph-mon[36729]: pgmap v396: 97 pgs: 97 active+clean; 35 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.9 MiB/s wr, 21 op/s 2024-03-16T09:17:10.402 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fault-injection/ 2024-03-16T09:17:10.404 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fault-injection/fault-injection.txt 2024-03-16T09:17:10.406 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fault-injection/notifier-error-inject.txt 2024-03-16T09:17:10.407 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fault-injection/provoke-crashes.txt 2024-03-16T09:17:10.409 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/ 2024-03-16T09:17:10.410 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/00-INDEX 2024-03-16T09:17:10.412 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/api.txt 2024-03-16T09:17:10.413 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/arkfb.txt 2024-03-16T09:17:10.415 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/aty128fb.txt 2024-03-16T09:17:10.417 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/cirrusfb.txt 2024-03-16T09:17:10.418 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/cmap_xfbdev.txt 2024-03-16T09:17:10.420 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/deferred_io.txt 2024-03-16T09:17:10.421 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/efifb.txt 2024-03-16T09:17:10.423 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/ep93xx-fb.txt 2024-03-16T09:17:10.424 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/fbcon.txt 2024-03-16T09:17:10.426 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/framebuffer.txt 2024-03-16T09:17:10.428 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/gxfb.txt 2024-03-16T09:17:10.429 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/intel810.txt 2024-03-16T09:17:10.431 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/intelfb.txt 2024-03-16T09:17:10.432 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/internals.txt 2024-03-16T09:17:10.434 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/lxfb.txt 2024-03-16T09:17:10.435 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/matroxfb.txt 2024-03-16T09:17:10.437 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/metronomefb.txt 2024-03-16T09:17:10.439 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/modedb.txt 2024-03-16T09:17:10.440 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/pvr2fb.txt 2024-03-16T09:17:10.442 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/pxafb.txt 2024-03-16T09:17:10.443 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/s3fb.txt 2024-03-16T09:17:10.445 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/sa1100fb.txt 2024-03-16T09:17:10.447 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/sh7760fb.txt 2024-03-16T09:17:10.448 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/sisfb.txt 2024-03-16T09:17:10.449 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/sm501.txt 2024-03-16T09:17:10.451 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/sstfb.txt 2024-03-16T09:17:10.453 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/tgafb.txt 2024-03-16T09:17:10.454 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/tridentfb.txt 2024-03-16T09:17:10.456 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/udlfb.txt 2024-03-16T09:17:10.457 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/uvesafb.txt 2024-03-16T09:17:10.459 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/vesafb.txt 2024-03-16T09:17:10.460 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/viafb.modes 2024-03-16T09:17:10.462 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/viafb.txt 2024-03-16T09:17:10.464 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/vt8623fb.txt 2024-03-16T09:17:10.465 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ 2024-03-16T09:17:10.467 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/.gitignore 2024-03-16T09:17:10.468 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/00-INDEX 2024-03-16T09:17:10.470 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/9p.txt 2024-03-16T09:17:10.471 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/Locking 2024-03-16T09:17:10.473 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/Makefile 2024-03-16T09:17:10.474 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/adfs.txt 2024-03-16T09:17:10.476 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/affs.txt 2024-03-16T09:17:10.478 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/afs.txt 2024-03-16T09:17:10.479 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/autofs4-mount-control.txt 2024-03-16T09:17:10.481 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/autofs4.txt 2024-03-16T09:17:10.482 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/automount-support.txt 2024-03-16T09:17:10.484 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/befs.txt 2024-03-16T09:17:10.486 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/bfs.txt 2024-03-16T09:17:10.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:10 smithi159 ceph-mon[36739]: pgmap v396: 97 pgs: 97 active+clean; 35 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.9 MiB/s wr, 21 op/s 2024-03-16T09:17:10.487 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/btrfs.txt 2024-03-16T09:17:10.488 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/caching/ 2024-03-16T09:17:10.489 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/caching/backend-api.txt 2024-03-16T09:17:10.490 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/caching/cachefiles.txt 2024-03-16T09:17:10.491 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/caching/fscache.txt 2024-03-16T09:17:10.492 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/caching/netfs-api.txt 2024-03-16T09:17:10.493 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/caching/object.txt 2024-03-16T09:17:10.494 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/caching/operations.txt 2024-03-16T09:17:10.495 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ceph.txt 2024-03-16T09:17:10.496 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/cifs/ 2024-03-16T09:17:10.497 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/cifs/AUTHORS 2024-03-16T09:17:10.498 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/cifs/CHANGES 2024-03-16T09:17:10.499 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/cifs/README 2024-03-16T09:17:10.500 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/cifs/TODO 2024-03-16T09:17:10.501 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/cifs/cifs.txt 2024-03-16T09:17:10.502 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/cifs/winucase_convert.pl 2024-03-16T09:17:10.503 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/coda.txt 2024-03-16T09:17:10.505 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/configfs/ 2024-03-16T09:17:10.506 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/configfs/Makefile 2024-03-16T09:17:10.508 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs.txt 2024-03-16T09:17:10.510 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_explicit.c 2024-03-16T09:17:10.511 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_macros.c 2024-03-16T09:17:10.513 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/cramfs.txt 2024-03-16T09:17:10.514 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/dax.txt 2024-03-16T09:17:10.515 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/debugfs.txt 2024-03-16T09:17:10.517 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/devpts.txt 2024-03-16T09:17:10.518 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/directory-locking 2024-03-16T09:17:10.520 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/dlmfs.txt 2024-03-16T09:17:10.522 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/dnotify.txt 2024-03-16T09:17:10.523 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/dnotify_test.c 2024-03-16T09:17:10.525 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ecryptfs.txt 2024-03-16T09:17:10.526 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/efivarfs.txt 2024-03-16T09:17:10.528 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/exofs.txt 2024-03-16T09:17:10.529 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ext2.txt 2024-03-16T09:17:10.531 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ext3.txt 2024-03-16T09:17:10.532 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ext4.txt 2024-03-16T09:17:10.534 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/f2fs.txt 2024-03-16T09:17:10.536 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/fiemap.txt 2024-03-16T09:17:10.537 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/files.txt 2024-03-16T09:17:10.539 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/fuse.txt 2024-03-16T09:17:10.540 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/gfs2-glocks.txt 2024-03-16T09:17:10.542 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/gfs2-uevents.txt 2024-03-16T09:17:10.543 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/gfs2.txt 2024-03-16T09:17:10.545 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/hfs.txt 2024-03-16T09:17:10.546 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/hfsplus.txt 2024-03-16T09:17:10.548 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/hpfs.txt 2024-03-16T09:17:10.549 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/inotify.txt 2024-03-16T09:17:10.551 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/isofs.txt 2024-03-16T09:17:10.552 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/jfs.txt 2024-03-16T09:17:10.554 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/locks.txt 2024-03-16T09:17:10.555 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/logfs.txt 2024-03-16T09:17:10.557 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/mandatory-locking.txt 2024-03-16T09:17:10.559 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ncpfs.txt 2024-03-16T09:17:10.560 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/ 2024-03-16T09:17:10.562 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/00-INDEX 2024-03-16T09:17:10.563 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/Exporting 2024-03-16T09:17:10.565 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/fault_injection.txt 2024-03-16T09:17:10.566 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/idmapper.txt 2024-03-16T09:17:10.568 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/knfsd-stats.txt 2024-03-16T09:17:10.569 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs-rdma.txt 2024-03-16T09:17:10.571 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs.txt 2024-03-16T09:17:10.573 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs41-server.txt 2024-03-16T09:17:10.574 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsd-admin-interfaces.txt 2024-03-16T09:17:10.576 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsroot.txt 2024-03-16T09:17:10.578 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs-block-server.txt 2024-03-16T09:17:10.579 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs.txt 2024-03-16T09:17:10.580 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-cache.txt 2024-03-16T09:17:10.581 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-server-gss.txt 2024-03-16T09:17:10.582 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nilfs2.txt 2024-03-16T09:17:10.583 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ntfs.txt 2024-03-16T09:17:10.584 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ocfs2.txt 2024-03-16T09:17:10.585 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/omfs.txt 2024-03-16T09:17:10.586 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/overlayfs.txt 2024-03-16T09:17:10.587 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/path-lookup.txt 2024-03-16T09:17:10.588 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/ 2024-03-16T09:17:10.589 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/design_notes.txt 2024-03-16T09:17:10.589 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/info.txt 2024-03-16T09:17:10.591 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/network_protocol.txt 2024-03-16T09:17:10.592 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/porting 2024-03-16T09:17:10.593 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/proc.txt 2024-03-16T09:17:10.595 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/qnx6.txt 2024-03-16T09:17:10.596 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/quota.txt 2024-03-16T09:17:10.597 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ramfs-rootfs-initramfs.txt 2024-03-16T09:17:10.599 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/relay.txt 2024-03-16T09:17:10.599 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/romfs.txt 2024-03-16T09:17:10.600 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/seq_file.txt 2024-03-16T09:17:10.602 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/sharedsubtree.txt 2024-03-16T09:17:10.603 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/spufs.txt 2024-03-16T09:17:10.604 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/squashfs.txt 2024-03-16T09:17:10.606 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/sysfs-pci.txt 2024-03-16T09:17:10.606 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/sysfs-tagging.txt 2024-03-16T09:17:10.606 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/sysfs.txt 2024-03-16T09:17:10.608 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/sysv-fs.txt 2024-03-16T09:17:10.608 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/tmpfs.txt 2024-03-16T09:17:10.609 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ubifs.txt 2024-03-16T09:17:10.610 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/udf.txt 2024-03-16T09:17:10.611 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ufs.txt 2024-03-16T09:17:10.612 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/vfat.txt 2024-03-16T09:17:10.613 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/vfs.txt 2024-03-16T09:17:10.614 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/xfs-delayed-logging-design.txt 2024-03-16T09:17:10.615 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/xfs-self-describing-metadata.txt 2024-03-16T09:17:10.616 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/xfs.txt 2024-03-16T09:17:10.617 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/firmware_class/ 2024-03-16T09:17:10.618 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/firmware_class/README 2024-03-16T09:17:10.619 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/firmware_class/hotplug-script 2024-03-16T09:17:10.620 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/flexible-arrays.txt 2024-03-16T09:17:10.621 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/ 2024-03-16T09:17:10.622 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/00-INDEX 2024-03-16T09:17:10.623 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/API.txt 2024-03-16T09:17:10.625 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/FMC-and-SDB.txt 2024-03-16T09:17:10.625 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/carrier.txt 2024-03-16T09:17:10.626 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/fmc-chardev.txt 2024-03-16T09:17:10.628 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/fmc-fakedev.txt 2024-03-16T09:17:10.628 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/fmc-trivial.txt 2024-03-16T09:17:10.630 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/fmc-write-eeprom.txt 2024-03-16T09:17:10.630 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/identifiers.txt 2024-03-16T09:17:10.631 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/mezzanine.txt 2024-03-16T09:17:10.633 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/parameters.txt 2024-03-16T09:17:10.634 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/ 2024-03-16T09:17:10.636 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/README.txt 2024-03-16T09:17:10.637 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/atomic-ops.txt 2024-03-16T09:17:10.638 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/booting.txt 2024-03-16T09:17:10.638 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/clock.txt 2024-03-16T09:17:10.639 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/configuring.txt 2024-03-16T09:17:10.639 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/features.txt 2024-03-16T09:17:10.640 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/gdbinit 2024-03-16T09:17:10.641 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/gdbstub.txt 2024-03-16T09:17:10.642 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/kernel-ABI.txt 2024-03-16T09:17:10.643 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/mmu-layout.txt 2024-03-16T09:17:10.644 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/futex-requeue-pi.txt 2024-03-16T09:17:10.646 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gcov.txt 2024-03-16T09:17:10.647 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gdb-kernel-debugging.txt 2024-03-16T09:17:10.648 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gpio/ 2024-03-16T09:17:10.649 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gpio/00-INDEX 2024-03-16T09:17:10.650 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gpio/board.txt 2024-03-16T09:17:10.651 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gpio/consumer.txt 2024-03-16T09:17:10.653 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gpio/driver.txt 2024-03-16T09:17:10.655 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gpio/gpio-legacy.txt 2024-03-16T09:17:10.656 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gpio/gpio.txt 2024-03-16T09:17:10.658 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gpio/sysfs.txt 2024-03-16T09:17:10.660 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hid/ 2024-03-16T09:17:10.661 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hid/hid-sensor.txt 2024-03-16T09:17:10.663 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hid/hid-transport.txt 2024-03-16T09:17:10.665 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hid/hiddev.txt 2024-03-16T09:17:10.666 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hid/hidraw.txt 2024-03-16T09:17:10.667 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hid/uhid.txt 2024-03-16T09:17:10.669 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/highuid.txt 2024-03-16T09:17:10.670 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hsi.txt 2024-03-16T09:17:10.671 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hw_random.txt 2024-03-16T09:17:10.672 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ 2024-03-16T09:17:10.673 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ab8500 2024-03-16T09:17:10.673 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/abituguru 2024-03-16T09:17:10.674 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/abituguru-datasheet 2024-03-16T09:17:10.676 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/abituguru3 2024-03-16T09:17:10.676 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/abx500 2024-03-16T09:17:10.678 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/acpi_power_meter 2024-03-16T09:17:10.678 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ad7314 2024-03-16T09:17:10.679 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adc128d818 2024-03-16T09:17:10.680 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adm1021 2024-03-16T09:17:10.682 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adm1025 2024-03-16T09:17:10.683 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adm1026 2024-03-16T09:17:10.684 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adm1031 2024-03-16T09:17:10.685 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adm1275 2024-03-16T09:17:10.687 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adm9240 2024-03-16T09:17:10.688 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ads1015 2024-03-16T09:17:10.690 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ads7828 2024-03-16T09:17:10.691 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adt7410 2024-03-16T09:17:10.693 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adt7411 2024-03-16T09:17:10.694 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adt7462 2024-03-16T09:17:10.695 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adt7470 2024-03-16T09:17:10.696 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adt7475 2024-03-16T09:17:10.697 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/amc6821 2024-03-16T09:17:10.697 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/asb100 2024-03-16T09:17:10.698 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/asc7621 2024-03-16T09:17:10.700 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/coretemp 2024-03-16T09:17:10.700 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/da9052 2024-03-16T09:17:10.701 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/da9055 2024-03-16T09:17:10.702 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/dme1737 2024-03-16T09:17:10.704 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ds1621 2024-03-16T09:17:10.704 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ds620 2024-03-16T09:17:10.706 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/emc1403 2024-03-16T09:17:10.706 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/emc2103 2024-03-16T09:17:10.708 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/emc6w201 2024-03-16T09:17:10.709 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/f71805f 2024-03-16T09:17:10.710 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/f71882fg 2024-03-16T09:17:10.711 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/fam15h_power 2024-03-16T09:17:10.712 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/g760a 2024-03-16T09:17:10.713 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/g762 2024-03-16T09:17:10.714 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/gl518sm 2024-03-16T09:17:10.715 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/hih6130 2024-03-16T09:17:10.716 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/htu21 2024-03-16T09:17:10.717 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/hwmon-kernel-api.txt 2024-03-16T09:17:10.718 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ibmaem 2024-03-16T09:17:10.719 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ibmpowernv 2024-03-16T09:17:10.720 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ina209 2024-03-16T09:17:10.721 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ina2xx 2024-03-16T09:17:10.722 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/it87 2024-03-16T09:17:10.724 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/jc42 2024-03-16T09:17:10.725 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/k10temp 2024-03-16T09:17:10.726 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/k8temp 2024-03-16T09:17:10.727 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lineage-pem 2024-03-16T09:17:10.728 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm25066 2024-03-16T09:17:10.729 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm63 2024-03-16T09:17:10.730 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm70 2024-03-16T09:17:10.731 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm73 2024-03-16T09:17:10.732 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm75 2024-03-16T09:17:10.733 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm77 2024-03-16T09:17:10.735 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm78 2024-03-16T09:17:10.735 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm80 2024-03-16T09:17:10.737 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm83 2024-03-16T09:17:10.737 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm85 2024-03-16T09:17:10.738 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm87 2024-03-16T09:17:10.739 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm90 2024-03-16T09:17:10.741 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm92 2024-03-16T09:17:10.741 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm93 2024-03-16T09:17:10.742 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm95234 2024-03-16T09:17:10.743 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm95245 2024-03-16T09:17:10.745 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ltc2945 2024-03-16T09:17:10.745 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ltc2978 2024-03-16T09:17:10.746 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ltc4151 2024-03-16T09:17:10.748 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ltc4215 2024-03-16T09:17:10.749 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ltc4245 2024-03-16T09:17:10.750 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ltc4260 2024-03-16T09:17:10.750 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ltc4261 2024-03-16T09:17:10.751 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max16064 2024-03-16T09:17:10.752 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max16065 2024-03-16T09:17:10.753 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max1619 2024-03-16T09:17:10.754 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max1668 2024-03-16T09:17:10.755 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max197 2024-03-16T09:17:10.756 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max34440 2024-03-16T09:17:10.757 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max6639 2024-03-16T09:17:10.757 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max6642 2024-03-16T09:17:10.759 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max6650 2024-03-16T09:17:10.759 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max6697 2024-03-16T09:17:10.760 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max8688 2024-03-16T09:17:10.761 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/mc13783-adc 2024-03-16T09:17:10.762 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/mcp3021 2024-03-16T09:17:10.763 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/menf21bmc 2024-03-16T09:17:10.764 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/nct6683 2024-03-16T09:17:10.765 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/nct6775 2024-03-16T09:17:10.766 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/nct7802 2024-03-16T09:17:10.767 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ntc_thermistor 2024-03-16T09:17:10.768 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/pc87360 2024-03-16T09:17:10.769 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/pc87427 2024-03-16T09:17:10.770 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/pcf8591 2024-03-16T09:17:10.771 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/pmbus 2024-03-16T09:17:10.772 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/pmbus-core 2024-03-16T09:17:10.773 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/powr1220 2024-03-16T09:17:10.776 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/pwm-fan 2024-03-16T09:17:10.776 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/sch5627 2024-03-16T09:17:10.776 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/sch5636 2024-03-16T09:17:10.777 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/sht15 2024-03-16T09:17:10.778 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/sht21 2024-03-16T09:17:10.779 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/shtc1 2024-03-16T09:17:10.780 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/sis5595 2024-03-16T09:17:10.781 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/smm665 2024-03-16T09:17:10.782 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/smsc47b397 2024-03-16T09:17:10.783 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/smsc47m1 2024-03-16T09:17:10.784 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/smsc47m192 2024-03-16T09:17:10.785 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/submitting-patches 2024-03-16T09:17:10.786 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/sysfs-interface 2024-03-16T09:17:10.787 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/thmc50 2024-03-16T09:17:10.788 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/tmp102 2024-03-16T09:17:10.789 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/tmp103 2024-03-16T09:17:10.790 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/tmp401 2024-03-16T09:17:10.791 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/tmp421 2024-03-16T09:17:10.792 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/tps40422 2024-03-16T09:17:10.793 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/twl4030-madc-hwmon 2024-03-16T09:17:10.794 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ucd9000 2024-03-16T09:17:10.794 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ucd9200 2024-03-16T09:17:10.795 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/userspace-tools 2024-03-16T09:17:10.796 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/vexpress 2024-03-16T09:17:10.797 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/via686a 2024-03-16T09:17:10.798 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/vt1211 2024-03-16T09:17:10.799 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/w83627ehf 2024-03-16T09:17:10.800 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/w83627hf 2024-03-16T09:17:10.802 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/w83781d 2024-03-16T09:17:10.802 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/w83791d 2024-03-16T09:17:10.803 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/w83792d 2024-03-16T09:17:10.805 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/w83793 2024-03-16T09:17:10.805 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/w83795 2024-03-16T09:17:10.806 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/w83l785ts 2024-03-16T09:17:10.807 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/w83l786ng 2024-03-16T09:17:10.811 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/wm831x 2024-03-16T09:17:10.811 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/wm8350 2024-03-16T09:17:10.811 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/zl6100 2024-03-16T09:17:10.812 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwspinlock.txt 2024-03-16T09:17:10.812 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/ 2024-03-16T09:17:10.813 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/ 2024-03-16T09:17:10.814 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1535 2024-03-16T09:17:10.815 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1563 2024-03-16T09:17:10.816 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali15x3 2024-03-16T09:17:10.817 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd756 2024-03-16T09:17:10.818 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd8111 2024-03-16T09:17:10.819 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-diolan-u2c 2024-03-16T09:17:10.820 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-i801 2024-03-16T09:17:10.821 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ismt 2024-03-16T09:17:10.822 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-nforce2 2024-03-16T09:17:10.823 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ocores 2024-03-16T09:17:10.824 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport 2024-03-16T09:17:10.825 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport-light 2024-03-16T09:17:10.826 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-pca-isa 2024-03-16T09:17:10.827 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-piix4 2024-03-16T09:17:10.828 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis5595 2024-03-16T09:17:10.829 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis630 2024-03-16T09:17:10.830 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis96x 2024-03-16T09:17:10.831 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-taos-evm 2024-03-16T09:17:10.832 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-via 2024-03-16T09:17:10.833 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-viapro 2024-03-16T09:17:10.834 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/scx200_acb 2024-03-16T09:17:10.835 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/dev-interface 2024-03-16T09:17:10.836 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/fault-codes 2024-03-16T09:17:10.837 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/functionality 2024-03-16T09:17:10.838 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/i2c-protocol 2024-03-16T09:17:10.839 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/i2c-stub 2024-03-16T09:17:10.840 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/instantiating-devices 2024-03-16T09:17:10.841 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/muxes/ 2024-03-16T09:17:10.842 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/muxes/i2c-mux-gpio 2024-03-16T09:17:10.842 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/old-module-parameters 2024-03-16T09:17:10.843 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/smbus-protocol 2024-03-16T09:17:10.844 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/summary 2024-03-16T09:17:10.845 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/ten-bit-addresses 2024-03-16T09:17:10.846 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/upgrading-clients 2024-03-16T09:17:10.847 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/writing-clients 2024-03-16T09:17:10.848 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2o/ 2024-03-16T09:17:10.849 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2o/README 2024-03-16T09:17:10.850 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2o/ioctl 2024-03-16T09:17:10.851 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/ 2024-03-16T09:17:10.852 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/.gitignore 2024-03-16T09:17:10.853 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/IRQ-redir.txt 2024-03-16T09:17:10.854 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/Makefile 2024-03-16T09:17:10.855 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/README 2024-03-16T09:17:10.856 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/aliasing-test.c 2024-03-16T09:17:10.857 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/aliasing.txt 2024-03-16T09:17:11.069 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/efirtc.txt 2024-03-16T09:17:11.070 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/err_inject.txt 2024-03-16T09:17:11.071 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/fsys.txt 2024-03-16T09:17:11.073 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/mca.txt 2024-03-16T09:17:11.075 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/serial.txt 2024-03-16T09:17:11.076 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/xen.txt 2024-03-16T09:17:11.078 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ide/ 2024-03-16T09:17:11.079 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ide/00-INDEX 2024-03-16T09:17:11.080 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-cd.1994-2004 2024-03-16T09:17:11.081 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-floppy.1996-2002 2024-03-16T09:17:11.083 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-tape.1995-2002 2024-03-16T09:17:11.083 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ide/ide-tape.txt 2024-03-16T09:17:11.084 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ide/ide.txt 2024-03-16T09:17:11.085 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ide/warm-plug-howto.txt 2024-03-16T09:17:11.086 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/infiniband/ 2024-03-16T09:17:11.087 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/infiniband/core_locking.txt 2024-03-16T09:17:11.088 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/infiniband/ipoib.txt 2024-03-16T09:17:11.089 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/infiniband/sysfs.txt 2024-03-16T09:17:11.090 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/infiniband/user_mad.txt 2024-03-16T09:17:11.091 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/infiniband/user_verbs.txt 2024-03-16T09:17:11.093 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/init.txt 2024-03-16T09:17:11.094 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/initrd.txt 2024-03-16T09:17:11.096 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/ 2024-03-16T09:17:11.098 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/alps.txt 2024-03-16T09:17:11.099 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/amijoy.txt 2024-03-16T09:17:11.101 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/appletouch.txt 2024-03-16T09:17:11.102 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/atarikbd.txt 2024-03-16T09:17:11.104 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/bcm5974.txt 2024-03-16T09:17:11.105 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/cd32.txt 2024-03-16T09:17:11.107 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/cma3000_d0x.txt 2024-03-16T09:17:11.108 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/cs461x.txt 2024-03-16T09:17:11.110 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/edt-ft5x06.txt 2024-03-16T09:17:11.112 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/elantech.txt 2024-03-16T09:17:11.113 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/event-codes.txt 2024-03-16T09:17:11.115 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/ff.txt 2024-03-16T09:17:11.116 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/gamepad.txt 2024-03-16T09:17:11.117 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/gameport-programming.txt 2024-03-16T09:17:11.118 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/gpio-tilt.txt 2024-03-16T09:17:11.119 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/iforce-protocol.txt 2024-03-16T09:17:11.119 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/input-programming.txt 2024-03-16T09:17:11.120 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/input.txt 2024-03-16T09:17:11.122 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/interactive.fig 2024-03-16T09:17:11.124 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/joystick-api.txt 2024-03-16T09:17:11.125 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/joystick-parport.txt 2024-03-16T09:17:11.127 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/joystick.txt 2024-03-16T09:17:11.129 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/multi-touch-protocol.txt 2024-03-16T09:17:11.130 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/notifier.txt 2024-03-16T09:17:11.131 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/ntrig.txt 2024-03-16T09:17:11.132 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/rotary-encoder.txt 2024-03-16T09:17:11.133 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/sentelic.txt 2024-03-16T09:17:11.134 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/shape.fig 2024-03-16T09:17:11.135 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/walkera0701.txt 2024-03-16T09:17:11.136 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/xpad.txt 2024-03-16T09:17:11.136 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/yealink.txt 2024-03-16T09:17:11.137 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/intel_txt.txt 2024-03-16T09:17:11.138 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/io-mapping.txt 2024-03-16T09:17:11.139 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/io_ordering.txt 2024-03-16T09:17:11.140 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ioctl/ 2024-03-16T09:17:11.141 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ioctl/00-INDEX 2024-03-16T09:17:11.143 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ioctl/botching-up-ioctls.txt 2024-03-16T09:17:11.143 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ioctl/cdrom.txt 2024-03-16T09:17:11.144 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ioctl/hdio.txt 2024-03-16T09:17:11.145 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ioctl/ioctl-decoding.txt 2024-03-16T09:17:11.146 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ioctl/ioctl-number.txt 2024-03-16T09:17:11.147 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/iostats.txt 2024-03-16T09:17:11.149 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/irqflags-tracing.txt 2024-03-16T09:17:11.150 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isapnp.txt 2024-03-16T09:17:11.151 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/ 2024-03-16T09:17:11.153 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/00-INDEX 2024-03-16T09:17:11.153 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/CREDITS 2024-03-16T09:17:11.153 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/HiSax.cert 2024-03-16T09:17:11.154 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/INTERFACE 2024-03-16T09:17:11.155 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.CAPI 2024-03-16T09:17:11.156 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.fax 2024-03-16T09:17:11.157 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README 2024-03-16T09:17:11.158 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.FAQ 2024-03-16T09:17:11.159 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.HiSax 2024-03-16T09:17:11.160 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.act2000 2024-03-16T09:17:11.161 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.audio 2024-03-16T09:17:11.162 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.avmb1 2024-03-16T09:17:11.163 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.concap 2024-03-16T09:17:11.164 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.diversion 2024-03-16T09:17:11.165 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.fax 2024-03-16T09:17:11.166 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.gigaset 2024-03-16T09:17:11.167 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.hfc-pci 2024-03-16T09:17:11.168 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.hysdn 2024-03-16T09:17:11.169 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.icn 2024-03-16T09:17:11.170 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.mISDN 2024-03-16T09:17:11.171 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.pcbit 2024-03-16T09:17:11.172 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.sc 2024-03-16T09:17:11.172 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.syncppp 2024-03-16T09:17:11.173 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.x25 2024-03-16T09:17:11.174 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/syncPPP.FAQ 2024-03-16T09:17:11.175 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ja_JP/ 2024-03-16T09:17:11.176 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ja_JP/HOWTO 2024-03-16T09:17:11.177 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ja_JP/SubmitChecklist 2024-03-16T09:17:11.178 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ja_JP/SubmittingPatches 2024-03-16T09:17:11.179 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ja_JP/stable_api_nonsense.txt 2024-03-16T09:17:11.180 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ja_JP/stable_kernel_rules.txt 2024-03-16T09:17:11.181 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/java.txt 2024-03-16T09:17:11.182 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kasan.txt 2024-03-16T09:17:11.183 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kbuild/ 2024-03-16T09:17:11.184 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kbuild/00-INDEX 2024-03-16T09:17:11.184 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kbuild/headers_install.txt 2024-03-16T09:17:11.185 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kbuild/kbuild.txt 2024-03-16T09:17:11.186 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kbuild/kconfig-language.txt 2024-03-16T09:17:11.187 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kbuild/kconfig.txt 2024-03-16T09:17:11.188 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kbuild/makefiles.txt 2024-03-16T09:17:11.189 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kbuild/modules.txt 2024-03-16T09:17:11.190 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kdump/ 2024-03-16T09:17:11.191 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kdump/gdbmacros.txt 2024-03-16T09:17:11.192 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kdump/kdump.txt 2024-03-16T09:17:11.193 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kernel-doc-nano-HOWTO.txt 2024-03-16T09:17:11.194 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kernel-docs.txt 2024-03-16T09:17:11.195 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kernel-parameters.txt 2024-03-16T09:17:11.197 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kernel-per-CPU-kthreads.txt 2024-03-16T09:17:11.198 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kmemcheck.txt 2024-03-16T09:17:11.199 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kmemleak.txt 2024-03-16T09:17:11.200 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ko_KR/ 2024-03-16T09:17:11.201 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ko_KR/HOWTO 2024-03-16T09:17:11.202 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ko_KR/stable_api_nonsense.txt 2024-03-16T09:17:11.203 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kobject.txt 2024-03-16T09:17:11.204 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kprobes.txt 2024-03-16T09:17:11.205 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kref.txt 2024-03-16T09:17:11.206 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kselftest.txt 2024-03-16T09:17:11.207 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/ 2024-03-16T09:17:11.208 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/.gitignore 2024-03-16T09:17:11.209 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/00-INDEX 2024-03-16T09:17:11.210 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/Makefile 2024-03-16T09:17:11.211 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/asus-laptop.txt 2024-03-16T09:17:11.212 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/disk-shock-protection.txt 2024-03-16T09:17:11.212 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/dslm.c 2024-03-16T09:17:11.213 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/freefall.c 2024-03-16T09:17:11.214 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/laptop-mode.txt 2024-03-16T09:17:11.215 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/sony-laptop.txt 2024-03-16T09:17:11.216 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/sonypi.txt 2024-03-16T09:17:11.217 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/thinkpad-acpi.txt 2024-03-16T09:17:11.218 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/toshiba_haps.txt 2024-03-16T09:17:11.219 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ldm.txt 2024-03-16T09:17:11.219 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/ 2024-03-16T09:17:11.220 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/00-INDEX 2024-03-16T09:17:11.221 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/leds-blinkm.txt 2024-03-16T09:17:11.222 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/leds-class.txt 2024-03-16T09:17:11.223 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/leds-lm3556.txt 2024-03-16T09:17:11.226 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/leds-lp3944.txt 2024-03-16T09:17:11.226 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/leds-lp5521.txt 2024-03-16T09:17:11.226 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/leds-lp5523.txt 2024-03-16T09:17:11.226 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/leds-lp5562.txt 2024-03-16T09:17:11.227 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/leds-lp55xx.txt 2024-03-16T09:17:11.228 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/ledtrig-oneshot.txt 2024-03-16T09:17:11.229 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/ledtrig-transient.txt 2024-03-16T09:17:11.230 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/local_ops.txt 2024-03-16T09:17:11.230 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/ 2024-03-16T09:17:11.231 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/00-INDEX 2024-03-16T09:17:11.232 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/lglock.txt 2024-03-16T09:17:11.233 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/lockdep-design.txt 2024-03-16T09:17:11.234 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/lockstat.txt 2024-03-16T09:17:11.235 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/locktorture.txt 2024-03-16T09:17:11.237 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/mutex-design.txt 2024-03-16T09:17:11.237 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/rt-mutex-design.txt 2024-03-16T09:17:11.238 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/rt-mutex.txt 2024-03-16T09:17:11.239 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/spinlocks.txt 2024-03-16T09:17:11.240 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/ww-mutex-design.txt 2024-03-16T09:17:11.241 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/lockup-watchdogs.txt 2024-03-16T09:17:11.242 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/logo.gif 2024-03-16T09:17:11.243 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/logo.txt 2024-03-16T09:17:11.244 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/lzo.txt 2024-03-16T09:17:11.245 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/m68k/ 2024-03-16T09:17:11.246 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/m68k/00-INDEX 2024-03-16T09:17:11.247 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/m68k/README.buddha 2024-03-16T09:17:11.248 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/m68k/kernel-options.txt 2024-03-16T09:17:11.249 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/magic-number.txt 2024-03-16T09:17:11.250 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mailbox.txt 2024-03-16T09:17:11.251 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/md.txt 2024-03-16T09:17:11.252 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/media-framework.txt 2024-03-16T09:17:11.253 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/memory-barriers.txt 2024-03-16T09:17:11.344 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/memory-devices/ 2024-03-16T09:17:11.344 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/memory-devices/ti-emif.txt 2024-03-16T09:17:11.346 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/memory-hotplug.txt 2024-03-16T09:17:11.346 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/metag/ 2024-03-16T09:17:11.347 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/metag/00-INDEX 2024-03-16T09:17:11.348 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/metag/kernel-ABI.txt 2024-03-16T09:17:11.349 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/ 2024-03-16T09:17:11.350 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/Makefile 2024-03-16T09:17:11.351 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/mic_overview.txt 2024-03-16T09:17:11.352 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/mpssd/ 2024-03-16T09:17:11.353 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/mpssd/.gitignore 2024-03-16T09:17:11.354 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/mpssd/Makefile 2024-03-16T09:17:11.354 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/mpssd/micctrl 2024-03-16T09:17:11.355 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/mpssd/mpss 2024-03-16T09:17:11.356 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.c 2024-03-16T09:17:11.357 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.h 2024-03-16T09:17:11.358 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/mpssd/sysfs.c 2024-03-16T09:17:11.359 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mips/ 2024-03-16T09:17:11.360 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mips/00-INDEX 2024-03-16T09:17:11.361 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mips/AU1xxx_IDE.README 2024-03-16T09:17:11.362 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/ 2024-03-16T09:17:11.363 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/Makefile 2024-03-16T09:17:11.364 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/ad525x_dpot.txt 2024-03-16T09:17:11.364 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/apds990x.txt 2024-03-16T09:17:11.365 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/bh1770glc.txt 2024-03-16T09:17:11.366 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/c2port.txt 2024-03-16T09:17:11.367 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/eeprom 2024-03-16T09:17:11.368 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/ics932s401 2024-03-16T09:17:11.369 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/isl29003 2024-03-16T09:17:11.370 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/lis3lv02d 2024-03-16T09:17:11.371 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/max6875 2024-03-16T09:17:11.372 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/mei/ 2024-03-16T09:17:11.373 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/mei/.gitignore 2024-03-16T09:17:11.374 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/mei/Makefile 2024-03-16T09:17:11.375 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/mei/TODO 2024-03-16T09:17:11.376 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-amt-version.c 2024-03-16T09:17:11.377 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-client-bus.txt 2024-03-16T09:17:11.378 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei.txt 2024-03-16T09:17:11.379 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/spear-pcie-gadget.txt 2024-03-16T09:17:11.380 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mmc/ 2024-03-16T09:17:11.381 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mmc/00-INDEX 2024-03-16T09:17:11.382 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mmc/mmc-async-req.txt 2024-03-16T09:17:11.383 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-attrs.txt 2024-03-16T09:17:11.385 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-parts.txt 2024-03-16T09:17:11.386 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mn10300/ 2024-03-16T09:17:11.388 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mn10300/ABI.txt 2024-03-16T09:17:11.389 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mn10300/compartmentalisation.txt 2024-03-16T09:17:11.391 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/module-signing.txt 2024-03-16T09:17:11.393 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mono.txt 2024-03-16T09:17:11.394 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mtd/ 2024-03-16T09:17:11.395 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mtd/nand/ 2024-03-16T09:17:11.397 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mtd/nand/pxa3xx-nand.txt 2024-03-16T09:17:11.398 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mtd/nand_ecc.txt 2024-03-16T09:17:11.400 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mtd/spi-nor.txt 2024-03-16T09:17:11.401 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/namespaces/ 2024-03-16T09:17:11.403 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/namespaces/compatibility-list.txt 2024-03-16T09:17:11.404 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/namespaces/resource-control.txt 2024-03-16T09:17:11.406 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/netlabel/ 2024-03-16T09:17:11.407 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/netlabel/00-INDEX 2024-03-16T09:17:11.409 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/netlabel/cipso_ipv4.txt 2024-03-16T09:17:11.410 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/netlabel/draft-ietf-cipso-ipsecurity-01.txt 2024-03-16T09:17:11.411 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/netlabel/introduction.txt 2024-03-16T09:17:11.413 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/netlabel/lsm_interface.txt 2024-03-16T09:17:11.414 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ 2024-03-16T09:17:11.416 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/00-INDEX 2024-03-16T09:17:11.417 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/3c509.txt 2024-03-16T09:17:11.418 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/6pack.txt 2024-03-16T09:17:11.419 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/LICENSE.qla3xxx 2024-03-16T09:17:11.420 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlcnic 2024-03-16T09:17:11.421 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlge 2024-03-16T09:17:11.422 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/Makefile 2024-03-16T09:17:11.423 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/PLIP.txt 2024-03-16T09:17:11.424 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/README.ipw2100 2024-03-16T09:17:11.425 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/README.ipw2200 2024-03-16T09:17:11.426 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/README.sb1000 2024-03-16T09:17:11.427 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/alias.txt 2024-03-16T09:17:11.430 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/altera_tse.txt 2024-03-16T09:17:11.430 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/arcnet-hardware.txt 2024-03-16T09:17:11.431 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/arcnet.txt 2024-03-16T09:17:11.431 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/atm.txt 2024-03-16T09:17:11.432 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ax25.txt 2024-03-16T09:17:11.433 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/batman-adv.txt 2024-03-16T09:17:11.434 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/baycom.txt 2024-03-16T09:17:11.435 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/bonding.txt 2024-03-16T09:17:11.437 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/bridge.txt 2024-03-16T09:17:11.437 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/caif/ 2024-03-16T09:17:11.438 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/caif/Linux-CAIF.txt 2024-03-16T09:17:11.439 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/caif/README 2024-03-16T09:17:11.440 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/caif/spi_porting.txt 2024-03-16T09:17:11.441 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/can.txt 2024-03-16T09:17:11.442 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/cdc_mbim.txt 2024-03-16T09:17:11.443 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/cops.txt 2024-03-16T09:17:11.443 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/cs89x0.txt 2024-03-16T09:17:11.448 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/cxacru-cf.py 2024-03-16T09:17:11.450 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/cxacru.txt 2024-03-16T09:17:11.451 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/cxgb.txt 2024-03-16T09:17:11.453 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/dccp.txt 2024-03-16T09:17:11.454 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/dctcp.txt 2024-03-16T09:17:11.455 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/de4x5.txt 2024-03-16T09:17:11.456 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/decnet.txt 2024-03-16T09:17:11.457 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/dl2k.txt 2024-03-16T09:17:11.458 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/dm9000.txt 2024-03-16T09:17:11.459 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/dmfe.txt 2024-03-16T09:17:11.461 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/dns_resolver.txt 2024-03-16T09:17:11.462 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/driver.txt 2024-03-16T09:17:11.463 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/e100.txt 2024-03-16T09:17:11.464 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/e1000.txt 2024-03-16T09:17:11.465 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/e1000e.txt 2024-03-16T09:17:11.466 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/eql.txt 2024-03-16T09:17:11.467 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/fib_trie.txt 2024-03-16T09:17:11.469 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/filter.txt 2024-03-16T09:17:11.470 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/fore200e.txt 2024-03-16T09:17:11.471 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/framerelay.txt 2024-03-16T09:17:11.472 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/gen_stats.txt 2024-03-16T09:17:11.473 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/generic-hdlc.txt 2024-03-16T09:17:11.474 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/generic_netlink.txt 2024-03-16T09:17:11.476 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/gianfar.txt 2024-03-16T09:17:11.477 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/i40e.txt 2024-03-16T09:17:11.478 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/i40evf.txt 2024-03-16T09:17:11.479 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ieee802154.txt 2024-03-16T09:17:11.481 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/igb.txt 2024-03-16T09:17:11.482 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/igbvf.txt 2024-03-16T09:17:11.484 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ip-sysctl.txt 2024-03-16T09:17:11.485 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ip_dynaddr.txt 2024-03-16T09:17:11.487 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ipddp.txt 2024-03-16T09:17:11.489 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/iphase.txt 2024-03-16T09:17:11.491 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ipsec.txt 2024-03-16T09:17:11.493 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ipv6.txt 2024-03-16T09:17:11.495 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ipvlan.txt 2024-03-16T09:17:11.497 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ipvs-sysctl.txt 2024-03-16T09:17:11.499 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/irda.txt 2024-03-16T09:17:11.501 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ixgb.txt 2024-03-16T09:17:11.503 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ixgbe.txt 2024-03-16T09:17:11.505 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ixgbevf.txt 2024-03-16T09:17:11.507 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/l2tp.txt 2024-03-16T09:17:11.509 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/lapb-module.txt 2024-03-16T09:17:11.511 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ltpc.txt 2024-03-16T09:17:11.513 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/mac80211-auth-assoc-deauth.txt 2024-03-16T09:17:11.514 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/mac80211-injection.txt 2024-03-16T09:17:11.516 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/ 2024-03-16T09:17:11.518 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/README 2024-03-16T09:17:11.519 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/hostapd.conf 2024-03-16T09:17:11.521 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/wpa_supplicant.conf 2024-03-16T09:17:11.523 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/multiqueue.txt 2024-03-16T09:17:11.525 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/netconsole.txt 2024-03-16T09:17:11.526 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/netdev-FAQ.txt 2024-03-16T09:17:11.528 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/netdev-features.txt 2024-03-16T09:17:11.530 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/netdevices.txt 2024-03-16T09:17:11.532 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/netif-msg.txt 2024-03-16T09:17:11.534 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/netlink_mmap.txt 2024-03-16T09:17:11.536 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/nf_conntrack-sysctl.txt 2024-03-16T09:17:11.538 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/nfc.txt 2024-03-16T09:17:11.540 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/openvswitch.txt 2024-03-16T09:17:11.542 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/operstates.txt 2024-03-16T09:17:11.544 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/packet_mmap.txt 2024-03-16T09:17:11.546 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/phonet.txt 2024-03-16T09:17:11.547 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/phy.txt 2024-03-16T09:17:11.548 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/pktgen.txt 2024-03-16T09:17:11.550 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/policy-routing.txt 2024-03-16T09:17:11.552 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ppp_generic.txt 2024-03-16T09:17:11.553 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/proc_net_tcp.txt 2024-03-16T09:17:11.555 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/radiotap-headers.txt 2024-03-16T09:17:11.556 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ray_cs.txt 2024-03-16T09:17:11.557 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/rds.txt 2024-03-16T09:17:11.559 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/regulatory.txt 2024-03-16T09:17:11.560 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/rxrpc.txt 2024-03-16T09:17:11.561 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/s2io.txt 2024-03-16T09:17:11.563 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/scaling.txt 2024-03-16T09:17:11.565 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/sctp.txt 2024-03-16T09:17:11.567 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/secid.txt 2024-03-16T09:17:11.569 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/skfp.txt 2024-03-16T09:17:11.572 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/smc9.txt 2024-03-16T09:17:11.573 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/spider_net.txt 2024-03-16T09:17:11.576 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/stmmac.txt 2024-03-16T09:17:11.578 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/switchdev.txt 2024-03-16T09:17:11.580 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/tc-actions-env-rules.txt 2024-03-16T09:17:11.582 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/tcp-thin.txt 2024-03-16T09:17:11.584 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/tcp.txt 2024-03-16T09:17:11.586 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/team.txt 2024-03-16T09:17:11.589 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/timestamping.txt 2024-03-16T09:17:11.591 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/timestamping/ 2024-03-16T09:17:11.592 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/timestamping/.gitignore 2024-03-16T09:17:11.594 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/timestamping/Makefile 2024-03-16T09:17:11.596 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/timestamping/hwtstamp_config.c 2024-03-16T09:17:11.598 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/timestamping/timestamping.c 2024-03-16T09:17:11.600 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/timestamping/txtimestamp.c 2024-03-16T09:17:11.602 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/tlan.txt 2024-03-16T09:17:11.605 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/tproxy.txt 2024-03-16T09:17:11.606 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/tuntap.txt 2024-03-16T09:17:11.608 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/udplite.txt 2024-03-16T09:17:11.610 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/vortex.txt 2024-03-16T09:17:11.612 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/vxge.txt 2024-03-16T09:17:11.613 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/vxlan.txt 2024-03-16T09:17:11.615 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/x25-iface.txt 2024-03-16T09:17:11.618 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/x25.txt 2024-03-16T09:17:11.620 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/xfrm_proc.txt 2024-03-16T09:17:11.621 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/xfrm_sync.txt 2024-03-16T09:17:11.624 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/xfrm_sysctl.txt 2024-03-16T09:17:11.626 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/z8530drv.txt 2024-03-16T09:17:11.628 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/nfc/ 2024-03-16T09:17:11.682 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/nfc/nfc-hci.txt 2024-03-16T09:17:11.684 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/nfc/nfc-pn544.txt 2024-03-16T09:17:11.685 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/nios2/ 2024-03-16T09:17:11.687 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/nios2/README 2024-03-16T09:17:11.688 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/nommu-mmap.txt 2024-03-16T09:17:11.690 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/numastat.txt 2024-03-16T09:17:11.692 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/oops-tracing.txt 2024-03-16T09:17:11.694 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/padata.txt 2024-03-16T09:17:11.696 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/parisc/ 2024-03-16T09:17:11.697 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/parisc/00-INDEX 2024-03-16T09:17:11.699 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/parisc/debugging 2024-03-16T09:17:11.700 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/parisc/registers 2024-03-16T09:17:11.701 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/parport-lowlevel.txt 2024-03-16T09:17:11.703 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/parport.txt 2024-03-16T09:17:11.705 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pcmcia/ 2024-03-16T09:17:11.706 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pcmcia/.gitignore 2024-03-16T09:17:11.707 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pcmcia/Makefile 2024-03-16T09:17:11.709 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pcmcia/crc32hash.c 2024-03-16T09:17:11.711 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pcmcia/devicetable.txt 2024-03-16T09:17:11.712 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pcmcia/driver-changes.txt 2024-03-16T09:17:11.713 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pcmcia/driver.txt 2024-03-16T09:17:11.715 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pcmcia/locking.txt 2024-03-16T09:17:11.716 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/percpu-rw-semaphore.txt 2024-03-16T09:17:11.718 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/phy.txt 2024-03-16T09:17:11.720 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/phy/ 2024-03-16T09:17:11.721 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/phy/samsung-usb2.txt 2024-03-16T09:17:11.723 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pi-futex.txt 2024-03-16T09:17:11.724 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pinctrl.txt 2024-03-16T09:17:11.726 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/platform/ 2024-03-16T09:17:11.727 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/platform/x86-laptop-drivers.txt 2024-03-16T09:17:11.729 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pnp.txt 2024-03-16T09:17:11.731 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/ 2024-03-16T09:17:11.732 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/00-INDEX 2024-03-16T09:17:11.733 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/apm-acpi.txt 2024-03-16T09:17:11.735 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/basic-pm-debugging.txt 2024-03-16T09:17:11.736 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/charger-manager.txt 2024-03-16T09:17:11.738 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/devices.txt 2024-03-16T09:17:11.740 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/drivers-testing.txt 2024-03-16T09:17:11.741 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/freezing-of-tasks.txt 2024-03-16T09:17:11.743 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/interface.txt 2024-03-16T09:17:11.744 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/notifiers.txt 2024-03-16T09:17:11.746 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/opp.txt 2024-03-16T09:17:11.747 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/pci.txt 2024-03-16T09:17:11.749 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/pm_qos_interface.txt 2024-03-16T09:17:11.750 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/power_supply_class.txt 2024-03-16T09:17:11.752 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/powercap/ 2024-03-16T09:17:11.753 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/powercap/powercap.txt 2024-03-16T09:17:11.755 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/regulator/ 2024-03-16T09:17:11.756 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/regulator/consumer.txt 2024-03-16T09:17:11.758 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/regulator/design.txt 2024-03-16T09:17:11.759 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/regulator/machine.txt 2024-03-16T09:17:11.761 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/regulator/overview.txt 2024-03-16T09:17:11.762 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/regulator/regulator.txt 2024-03-16T09:17:11.763 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/runtime_pm.txt 2024-03-16T09:17:11.764 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/s2ram.txt 2024-03-16T09:17:11.765 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/states.txt 2024-03-16T09:17:11.766 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/suspend-and-cpuhotplug.txt 2024-03-16T09:17:11.767 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/suspend-and-interrupts.txt 2024-03-16T09:17:11.769 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/swsusp-and-swap-files.txt 2024-03-16T09:17:11.770 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/swsusp-dmcrypt.txt 2024-03-16T09:17:11.772 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/swsusp.txt 2024-03-16T09:17:11.773 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/tricks.txt 2024-03-16T09:17:11.775 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/userland-swsusp.txt 2024-03-16T09:17:11.776 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/video.txt 2024-03-16T09:17:11.778 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/ 2024-03-16T09:17:11.779 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/00-INDEX 2024-03-16T09:17:11.781 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/bootwrapper.txt 2024-03-16T09:17:11.782 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/cpu_families.txt 2024-03-16T09:17:11.784 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/cpu_features.txt 2024-03-16T09:17:11.785 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/cxl.txt 2024-03-16T09:17:11.787 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/eeh-pci-error-recovery.txt 2024-03-16T09:17:11.788 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/firmware-assisted-dump.txt 2024-03-16T09:17:11.790 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/hvcs.txt 2024-03-16T09:17:11.791 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/mpc52xx.txt 2024-03-16T09:17:11.793 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/pmu-ebb.txt 2024-03-16T09:17:11.794 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/ptrace.txt 2024-03-16T09:17:11.796 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/qe_firmware.txt 2024-03-16T09:17:11.797 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/transactional_memory.txt 2024-03-16T09:17:11.799 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pps/ 2024-03-16T09:17:11.801 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pps/pps.txt 2024-03-16T09:17:11.802 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/prctl/ 2024-03-16T09:17:11.804 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/prctl/.gitignore 2024-03-16T09:17:11.805 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/prctl/Makefile 2024-03-16T09:17:11.807 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-ctxt-sw-stress-test.c 2024-03-16T09:17:11.808 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-on-off-stress-test.c 2024-03-16T09:17:11.810 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-test.c 2024-03-16T09:17:11.811 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/prctl/no_new_privs.txt 2024-03-16T09:17:11.813 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/prctl/seccomp_filter.txt 2024-03-16T09:17:11.814 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/preempt-locking.txt 2024-03-16T09:17:11.816 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/printk-formats.txt 2024-03-16T09:17:11.817 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pti/ 2024-03-16T09:17:11.819 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pti/pti_intel_mid.txt 2024-03-16T09:17:11.820 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ptp/ 2024-03-16T09:17:11.822 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ptp/.gitignore 2024-03-16T09:17:11.823 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ptp/Makefile 2024-03-16T09:17:11.824 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ptp/ptp.txt 2024-03-16T09:17:11.825 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ptp/testptp.c 2024-03-16T09:17:11.826 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ptp/testptp.mk 2024-03-16T09:17:11.827 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pwm.txt 2024-03-16T09:17:11.828 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ramoops.txt 2024-03-16T09:17:11.829 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/rapidio/ 2024-03-16T09:17:11.830 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/rapidio/rapidio.txt 2024-03-16T09:17:11.831 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/rapidio/sysfs.txt 2024-03-16T09:17:11.832 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/rapidio/tsi721.txt 2024-03-16T09:17:11.833 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/rbtree.txt 2024-03-16T09:17:11.834 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/remoteproc.txt 2024-03-16T09:17:11.835 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/rfkill.txt 2024-03-16T09:17:11.836 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/robust-futex-ABI.txt 2024-03-16T09:17:11.837 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/robust-futexes.txt 2024-03-16T09:17:11.838 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/rpmsg.txt 2024-03-16T09:17:11.842 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/rtc.txt 2024-03-16T09:17:11.842 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/ 2024-03-16T09:17:11.842 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/00-INDEX 2024-03-16T09:17:11.842 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/3270.ChangeLog 2024-03-16T09:17:11.843 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/3270.txt 2024-03-16T09:17:11.844 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/CommonIO 2024-03-16T09:17:11.845 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/DASD 2024-03-16T09:17:11.846 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/Debugging390.txt 2024-03-16T09:17:11.847 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/cds.txt 2024-03-16T09:17:11.848 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/config3270.sh 2024-03-16T09:17:11.849 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/driver-model.txt 2024-03-16T09:17:11.850 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/kvm.txt 2024-03-16T09:17:11.851 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/monreader.txt 2024-03-16T09:17:11.852 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/qeth.txt 2024-03-16T09:17:11.853 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/s390dbf.txt 2024-03-16T09:17:11.854 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/zfcpdump.txt 2024-03-16T09:17:11.855 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/ 2024-03-16T09:17:11.856 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/00-INDEX 2024-03-16T09:17:11.857 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/completion.txt 2024-03-16T09:17:11.858 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/sched-arch.txt 2024-03-16T09:17:11.859 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/sched-bwc.txt 2024-03-16T09:17:11.860 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/sched-deadline.txt 2024-03-16T09:17:11.861 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/sched-design-CFS.txt 2024-03-16T09:17:11.862 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/sched-domains.txt 2024-03-16T09:17:11.863 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/sched-nice-design.txt 2024-03-16T09:17:11.864 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/sched-rt-group.txt 2024-03-16T09:17:11.865 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/sched-stats.txt 2024-03-16T09:17:11.866 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ 2024-03-16T09:17:11.867 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/00-INDEX 2024-03-16T09:17:11.868 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/53c700.txt 2024-03-16T09:17:11.869 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/BusLogic.txt 2024-03-16T09:17:11.870 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.1992-1997 2024-03-16T09:17:11.871 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.arcmsr 2024-03-16T09:17:11.872 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ips 2024-03-16T09:17:11.873 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.lpfc 2024-03-16T09:17:11.875 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid 2024-03-16T09:17:11.876 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid_sas 2024-03-16T09:17:11.876 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ncr53c8xx 2024-03-16T09:17:11.877 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx 2024-03-16T09:17:11.878 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx_2 2024-03-16T09:17:11.879 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/FlashPoint.txt 2024-03-16T09:17:11.880 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/LICENSE.FlashPoint 2024-03-16T09:17:11.881 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla2xxx 2024-03-16T09:17:11.882 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla4xxx 2024-03-16T09:17:11.883 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/Mylex.txt 2024-03-16T09:17:11.884 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/NinjaSCSI.txt 2024-03-16T09:17:11.885 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/aacraid.txt 2024-03-16T09:17:11.886 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/advansys.txt 2024-03-16T09:17:11.887 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/aha152x.txt 2024-03-16T09:17:11.888 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/aic79xx.txt 2024-03-16T09:17:11.889 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/aic7xxx.txt 2024-03-16T09:17:11.890 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/arcmsr_spec.txt 2024-03-16T09:17:11.891 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/bfa.txt 2024-03-16T09:17:11.892 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/bnx2fc.txt 2024-03-16T09:17:11.893 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/cxgb3i.txt 2024-03-16T09:17:11.894 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/dc395x.txt 2024-03-16T09:17:11.895 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/dpti.txt 2024-03-16T09:17:11.896 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/dtc3x80.txt 2024-03-16T09:17:11.897 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/g_NCR5380.txt 2024-03-16T09:17:11.898 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/hpsa.txt 2024-03-16T09:17:11.899 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/hptiop.txt 2024-03-16T09:17:11.900 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/in2000.txt 2024-03-16T09:17:11.901 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/libsas.txt 2024-03-16T09:17:11.902 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/link_power_management_policy.txt 2024-03-16T09:17:11.903 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/lpfc.txt 2024-03-16T09:17:11.903 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/megaraid.txt 2024-03-16T09:17:11.904 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ncr53c8xx.txt 2024-03-16T09:17:11.906 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/osd.txt 2024-03-16T09:17:11.907 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/osst.txt 2024-03-16T09:17:11.907 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ppa.txt 2024-03-16T09:17:11.908 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/qlogicfas.txt 2024-03-16T09:17:11.909 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi-changer.txt 2024-03-16T09:17:11.910 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi-generic.txt 2024-03-16T09:17:11.911 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi-parameters.txt 2024-03-16T09:17:11.912 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi.txt 2024-03-16T09:17:11.913 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi_eh.txt 2024-03-16T09:17:11.914 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi_fc_transport.txt 2024-03-16T09:17:11.915 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi_mid_low_api.txt 2024-03-16T09:17:11.916 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/ 2024-03-16T09:17:11.917 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/Makefile 2024-03-16T09:17:11.918 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/rport_state_diagram.dot 2024-03-16T09:17:11.919 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/st.txt 2024-03-16T09:17:11.920 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/sym53c500_cs.txt 2024-03-16T09:17:11.921 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/sym53c8xx_2.txt 2024-03-16T09:17:11.922 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/tmscsim.txt 2024-03-16T09:17:11.923 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ufs.txt 2024-03-16T09:17:11.924 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/wd719x.txt 2024-03-16T09:17:11.925 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/ 2024-03-16T09:17:11.926 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/00-INDEX 2024-03-16T09:17:11.927 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/IMA-templates.txt 2024-03-16T09:17:11.928 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/LSM.txt 2024-03-16T09:17:11.929 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/SELinux.txt 2024-03-16T09:17:11.930 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/Smack.txt 2024-03-16T09:17:11.930 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/Yama.txt 2024-03-16T09:17:11.931 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/apparmor.txt 2024-03-16T09:17:11.932 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/credentials.txt 2024-03-16T09:17:11.933 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/keys-ecryptfs.txt 2024-03-16T09:17:11.934 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/keys-request-key.txt 2024-03-16T09:17:11.935 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/keys-trusted-encrypted.txt 2024-03-16T09:17:11.936 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/keys.txt 2024-03-16T09:17:11.938 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/tomoyo.txt 2024-03-16T09:17:11.938 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial-console.txt 2024-03-16T09:17:11.939 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial/ 2024-03-16T09:17:11.940 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial/00-INDEX 2024-03-16T09:17:11.941 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial/README.cycladesZ 2024-03-16T09:17:11.942 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial/driver 2024-03-16T09:17:11.943 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial/moxa-smartio 2024-03-16T09:17:11.945 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial/n_gsm.txt 2024-03-16T09:17:11.946 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial/rocket.txt 2024-03-16T09:17:11.947 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial/serial-rs485.txt 2024-03-16T09:17:11.949 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial/tty.txt 2024-03-16T09:17:11.950 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sgi-ioc4.txt 2024-03-16T09:17:11.952 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sh/ 2024-03-16T09:17:11.953 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sh/new-machine.txt 2024-03-16T09:17:11.955 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sh/register-banks.txt 2024-03-16T09:17:11.956 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/smsc_ece1099.txt 2024-03-16T09:17:11.958 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/ 2024-03-16T09:17:11.960 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/ 2024-03-16T09:17:11.961 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/ALSA-Configuration.txt 2024-03-16T09:17:11.962 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/Audigy-mixer.txt 2024-03-16T09:17:11.964 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/Audiophile-Usb.txt 2024-03-16T09:17:11.965 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/Bt87x.txt 2024-03-16T09:17:11.967 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/CMIPCI.txt 2024-03-16T09:17:11.968 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/Channel-Mapping-API.txt 2024-03-16T09:17:11.970 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/ControlNames.txt 2024-03-16T09:17:11.972 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Controls.txt 2024-03-16T09:17:11.973 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Models.txt 2024-03-16T09:17:11.974 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio.txt 2024-03-16T09:17:11.976 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/Joystick.txt 2024-03-16T09:17:11.978 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/MIXART.txt 2024-03-16T09:17:11.979 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/OSS-Emulation.txt 2024-03-16T09:17:11.981 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/Procfile.txt 2024-03-16T09:17:11.982 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/README.maya44 2024-03-16T09:17:11.984 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/SB-Live-mixer.txt 2024-03-16T09:17:11.986 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/VIA82xx-mixer.txt 2024-03-16T09:17:11.987 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/alsa-parameters.txt 2024-03-16T09:17:11.989 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/compress_offload.txt 2024-03-16T09:17:11.990 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/emu10k1-jack.txt 2024-03-16T09:17:11.992 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/hda_codec.txt 2024-03-16T09:17:11.994 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/hdspm.txt 2024-03-16T09:17:11.995 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/powersave.txt 2024-03-16T09:17:11.997 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/seq_oss.html 2024-03-16T09:17:11.999 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/serial-u16550.txt 2024-03-16T09:17:12.000 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/ 2024-03-16T09:17:12.002 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DAI.txt 2024-03-16T09:17:12.003 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DPCM.txt 2024-03-16T09:17:12.004 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/clocking.txt 2024-03-16T09:17:12.005 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/codec.txt 2024-03-16T09:17:12.007 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/dapm.txt 2024-03-16T09:17:12.008 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/jack.txt 2024-03-16T09:17:12.010 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/machine.txt 2024-03-16T09:17:12.011 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/overview.txt 2024-03-16T09:17:12.013 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/platform.txt 2024-03-16T09:17:12.014 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/pops_clicks.txt 2024-03-16T09:17:12.016 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/ 2024-03-16T09:17:12.017 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/ALS 2024-03-16T09:17:12.019 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/AudioExcelDSP16 2024-03-16T09:17:12.020 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/CMI8330 2024-03-16T09:17:12.021 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/ESS 2024-03-16T09:17:12.021 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/ESS1868 2024-03-16T09:17:12.022 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/Introduction 2024-03-16T09:17:12.023 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/MultiSound 2024-03-16T09:17:12.024 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/OPL3 2024-03-16T09:17:12.026 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/Opti 2024-03-16T09:17:12.027 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/PAS16 2024-03-16T09:17:12.029 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/PSS 2024-03-16T09:17:12.030 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/PSS-updates 2024-03-16T09:17:12.032 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/README.OSS 2024-03-16T09:17:12.034 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/README.modules 2024-03-16T09:17:12.035 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/README.ymfsb 2024-03-16T09:17:12.037 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/SoundPro 2024-03-16T09:17:12.039 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/Soundblaster 2024-03-16T09:17:12.041 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/Tropez+ 2024-03-16T09:17:12.042 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/VIBRA16 2024-03-16T09:17:12.043 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/WaveArtist 2024-03-16T09:17:12.045 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/btaudio 2024-03-16T09:17:12.050 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/mwave 2024-03-16T09:17:12.050 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/oss-parameters.txt 2024-03-16T09:17:12.050 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/ultrasound 2024-03-16T09:17:12.051 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sparse.txt 2024-03-16T09:17:12.053 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/ 2024-03-16T09:17:12.055 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/.gitignore 2024-03-16T09:17:12.057 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/00-INDEX 2024-03-16T09:17:12.059 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/Makefile 2024-03-16T09:17:12.060 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/butterfly 2024-03-16T09:17:12.062 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/ep93xx_spi 2024-03-16T09:17:12.064 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/pxa2xx 2024-03-16T09:17:12.066 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/spi-lm70llp 2024-03-16T09:17:12.068 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/spi-sc18is602 2024-03-16T09:17:12.069 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/spi-summary 2024-03-16T09:17:12.071 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/spidev 2024-03-16T09:17:12.073 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/spidev_fdx.c 2024-03-16T09:17:12.075 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/spidev_test.c 2024-03-16T09:17:12.076 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/stable_api_nonsense.txt 2024-03-16T09:17:12.077 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/stable_kernel_rules.txt 2024-03-16T09:17:12.079 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/static-keys.txt 2024-03-16T09:17:12.080 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/svga.txt 2024-03-16T09:17:12.082 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysctl/ 2024-03-16T09:17:12.083 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysctl/00-INDEX 2024-03-16T09:17:12.085 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysctl/README 2024-03-16T09:17:12.086 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysctl/abi.txt 2024-03-16T09:17:12.088 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysctl/fs.txt 2024-03-16T09:17:12.090 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysctl/kernel.txt 2024-03-16T09:17:12.091 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysctl/net.txt 2024-03-16T09:17:12.093 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysctl/sunrpc.txt 2024-03-16T09:17:12.094 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysctl/vm.txt 2024-03-16T09:17:12.096 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysfs-rules.txt 2024-03-16T09:17:12.097 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysrq.txt 2024-03-16T09:17:12.099 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/target/ 2024-03-16T09:17:12.101 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.py 2024-03-16T09:17:12.106 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.txt 2024-03-16T09:17:12.171 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/target/tcmu-design.txt 2024-03-16T09:17:12.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:12 smithi112 ceph-mon[32307]: pgmap v397: 97 pgs: 97 active+clean; 76 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.3 MiB/s wr, 26 op/s 2024-03-16T09:17:12.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:12 smithi194 ceph-mon[36729]: pgmap v397: 97 pgs: 97 active+clean; 76 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.3 MiB/s wr, 26 op/s 2024-03-16T09:17:12.413 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/thermal/ 2024-03-16T09:17:12.447 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/thermal/cpu-cooling-api.txt 2024-03-16T09:17:12.456 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal 2024-03-16T09:17:12.462 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal_emulation 2024-03-16T09:17:12.463 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/thermal/intel_powerclamp.txt 2024-03-16T09:17:12.466 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/thermal/nouveau_thermal 2024-03-16T09:17:12.469 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/thermal/sysfs-api.txt 2024-03-16T09:17:12.471 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/thermal/x86_pkg_temperature_thermal 2024-03-16T09:17:12.473 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/this_cpu_ops.txt 2024-03-16T09:17:12.475 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/ 2024-03-16T09:17:12.478 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/.gitignore 2024-03-16T09:17:12.479 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/00-INDEX 2024-03-16T09:17:12.481 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/Makefile 2024-03-16T09:17:12.483 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/NO_HZ.txt 2024-03-16T09:17:12.486 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/highres.txt 2024-03-16T09:17:12.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:12 smithi159 ceph-mon[36739]: pgmap v397: 97 pgs: 97 active+clean; 76 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.3 MiB/s wr, 26 op/s 2024-03-16T09:17:12.490 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/hpet.txt 2024-03-16T09:17:12.493 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/hpet_example.c 2024-03-16T09:17:12.496 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/hrtimers.txt 2024-03-16T09:17:12.499 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/timekeeping.txt 2024-03-16T09:17:12.504 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/timer_stats.txt 2024-03-16T09:17:12.510 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/timers-howto.txt 2024-03-16T09:17:12.514 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/tpm/ 2024-03-16T09:17:12.518 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/tpm/xen-tpmfront.txt 2024-03-16T09:17:12.524 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/ 2024-03-16T09:17:12.530 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/coresight.txt 2024-03-16T09:17:12.535 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/events-kmem.txt 2024-03-16T09:17:12.540 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/events-nmi.txt 2024-03-16T09:17:12.543 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/events-power.txt 2024-03-16T09:17:12.546 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/events.txt 2024-03-16T09:17:12.555 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/ftrace-design.txt 2024-03-16T09:17:12.566 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/ftrace.txt 2024-03-16T09:17:12.574 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/function-graph-fold.vim 2024-03-16T09:17:12.579 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/kprobetrace.txt 2024-03-16T09:17:12.584 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/mmiotrace.txt 2024-03-16T09:17:12.591 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/postprocess/ 2024-03-16T09:17:12.595 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-pagealloc-postprocess.pl 2024-03-16T09:17:12.597 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-vmscan-postprocess.pl 2024-03-16T09:17:12.598 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/ring-buffer-design.txt 2024-03-16T09:17:12.600 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/tracepoint-analysis.txt 2024-03-16T09:17:12.603 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/tracepoints.txt 2024-03-16T09:17:12.605 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/uprobetracer.txt 2024-03-16T09:17:12.610 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/unaligned-memory-access.txt 2024-03-16T09:17:12.618 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/unicode.txt 2024-03-16T09:17:12.621 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/unshare.txt 2024-03-16T09:17:12.626 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/ 2024-03-16T09:17:12.629 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/CREDITS 2024-03-16T09:17:12.634 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/URB.txt 2024-03-16T09:17:12.637 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/WUSB-Design-overview.txt 2024-03-16T09:17:12.642 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/acm.txt 2024-03-16T09:17:12.646 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/anchors.txt 2024-03-16T09:17:12.651 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/authorization.txt 2024-03-16T09:17:12.655 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/bulk-streams.txt 2024-03-16T09:17:12.658 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/callbacks.txt 2024-03-16T09:17:12.662 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/chipidea.txt 2024-03-16T09:17:12.665 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/dma.txt 2024-03-16T09:17:12.671 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/dwc3.txt 2024-03-16T09:17:12.676 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/ehci.txt 2024-03-16T09:17:12.679 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/error-codes.txt 2024-03-16T09:17:12.682 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/functionfs.txt 2024-03-16T09:17:12.688 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/gadget-testing.txt 2024-03-16T09:17:12.692 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/gadget_configfs.txt 2024-03-16T09:17:12.695 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/gadget_hid.txt 2024-03-16T09:17:12.699 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/gadget_multi.txt 2024-03-16T09:17:12.704 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/gadget_printer.txt 2024-03-16T09:17:12.712 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/gadget_serial.txt 2024-03-16T09:17:12.718 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/hotplug.txt 2024-03-16T09:17:12.721 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/iuu_phoenix.txt 2024-03-16T09:17:12.731 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/linux-cdc-acm.inf 2024-03-16T09:17:12.734 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/linux.inf 2024-03-16T09:17:12.736 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/mass-storage.txt 2024-03-16T09:17:12.741 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/misc_usbsevseg.txt 2024-03-16T09:17:12.746 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/mtouchusb.txt 2024-03-16T09:17:12.752 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/ohci.txt 2024-03-16T09:17:12.761 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/persist.txt 2024-03-16T09:17:12.766 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/power-management.txt 2024-03-16T09:17:12.773 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/proc_usb_info.txt 2024-03-16T09:17:12.781 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/rio.txt 2024-03-16T09:17:12.787 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/usb-help.txt 2024-03-16T09:17:12.792 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/usb-serial.txt 2024-03-16T09:17:12.795 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/usbmon.txt 2024-03-16T09:17:12.799 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/wusb-cbaf 2024-03-16T09:17:12.805 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vDSO/ 2024-03-16T09:17:12.820 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vDSO/.gitignore 2024-03-16T09:17:12.825 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vDSO/Makefile 2024-03-16T09:17:12.831 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vDSO/parse_vdso.c 2024-03-16T09:17:12.837 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vDSO/vdso_standalone_test_x86.c 2024-03-16T09:17:12.843 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vDSO/vdso_test.c 2024-03-16T09:17:12.850 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vfio.txt 2024-03-16T09:17:12.854 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vgaarbiter.txt 2024-03-16T09:17:12.860 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video-output.txt 2024-03-16T09:17:12.865 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/ 2024-03-16T09:17:12.873 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/4CCs.txt 2024-03-16T09:17:12.874 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/API.html 2024-03-16T09:17:12.877 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.au0828 2024-03-16T09:17:12.880 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.bttv 2024-03-16T09:17:12.882 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.cx23885 2024-03-16T09:17:12.884 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.cx88 2024-03-16T09:17:12.889 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.em28xx 2024-03-16T09:17:12.891 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.ivtv 2024-03-16T09:17:12.892 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.saa7134 2024-03-16T09:17:12.894 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.saa7164 2024-03-16T09:17:12.895 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.tm6000 2024-03-16T09:17:12.897 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.tuner 2024-03-16T09:17:12.901 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.usbvision 2024-03-16T09:17:12.902 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/Makefile 2024-03-16T09:17:12.903 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/README.cpia2 2024-03-16T09:17:12.908 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/README.cx88 2024-03-16T09:17:12.909 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/README.davinci-vpbe 2024-03-16T09:17:12.913 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/README.ir 2024-03-16T09:17:12.915 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/README.ivtv 2024-03-16T09:17:12.921 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/README.pvrusb2 2024-03-16T09:17:12.922 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/README.saa7134 2024-03-16T09:17:12.923 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/Zoran 2024-03-16T09:17:12.924 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/ 2024-03-16T09:17:12.925 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/CONTRIBUTORS 2024-03-16T09:17:12.926 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/Cards 2024-03-16T09:17:12.927 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/ICs 2024-03-16T09:17:12.927 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/Insmod-options 2024-03-16T09:17:12.928 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/MAKEDEV 2024-03-16T09:17:12.930 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/Modprobe.conf 2024-03-16T09:17:12.932 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/Modules.conf 2024-03-16T09:17:12.934 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/PROBLEMS 2024-03-16T09:17:12.936 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/README 2024-03-16T09:17:12.937 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.WINVIEW 2024-03-16T09:17:12.939 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.freeze 2024-03-16T09:17:12.941 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.quirks 2024-03-16T09:17:12.943 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/Sound-FAQ 2024-03-16T09:17:12.944 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/Specs 2024-03-16T09:17:12.946 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/THANKS 2024-03-16T09:17:12.948 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/Tuners 2024-03-16T09:17:12.950 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cafe_ccic 2024-03-16T09:17:12.952 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cpia2_overview.txt 2024-03-16T09:17:12.954 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx18.txt 2024-03-16T09:17:12.956 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/ 2024-03-16T09:17:12.958 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/README.hm12 2024-03-16T09:17:12.960 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/README.vbi 2024-03-16T09:17:12.961 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-calling.txt 2024-03-16T09:17:12.963 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-decoder-api.txt 2024-03-16T09:17:12.965 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-decoder-regs.txt 2024-03-16T09:17:12.966 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-dma.txt 2024-03-16T09:17:12.968 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-encoder-api.txt 2024-03-16T09:17:12.970 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-memory.txt 2024-03-16T09:17:12.972 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-osd-api.txt 2024-03-16T09:17:12.974 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-upload.txt 2024-03-16T09:17:12.976 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx88/ 2024-03-16T09:17:12.982 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx88/hauppauge-wintv-cx88-ir.txt 2024-03-16T09:17:12.984 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/extract_xc3028.pl 2024-03-16T09:17:12.985 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/fimc.txt 2024-03-16T09:17:12.987 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/gspca.txt 2024-03-16T09:17:12.989 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/hauppauge-wintv-cx88-ir.txt 2024-03-16T09:17:12.991 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/lifeview.txt 2024-03-16T09:17:12.993 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/meye.txt 2024-03-16T09:17:12.995 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/not-in-cx2388x-datasheet.txt 2024-03-16T09:17:12.996 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/omap3isp.txt 2024-03-16T09:17:12.998 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/omap4_camera.txt 2024-03-16T09:17:12.999 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/pxa_camera.txt 2024-03-16T09:17:13.001 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/radiotrack.txt 2024-03-16T09:17:13.002 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/sh_mobile_ceu_camera.txt 2024-03-16T09:17:13.004 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/si470x.txt 2024-03-16T09:17:13.009 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/si4713.txt 2024-03-16T09:17:13.010 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/si476x.txt 2024-03-16T09:17:13.012 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/soc-camera.txt 2024-03-16T09:17:13.014 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/uvcvideo.txt 2024-03-16T09:17:13.015 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/v4l2-controls.txt 2024-03-16T09:17:13.017 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/v4l2-framework.txt 2024-03-16T09:17:13.018 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/v4l2-pci-skeleton.c 2024-03-16T09:17:13.019 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/videobuf 2024-03-16T09:17:13.021 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/vivid.txt 2024-03-16T09:17:13.022 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/zr364xx.txt 2024-03-16T09:17:13.023 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/ 2024-03-16T09:17:13.024 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/00-INDEX 2024-03-16T09:17:13.026 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/ 2024-03-16T09:17:13.027 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/00-INDEX 2024-03-16T09:17:13.028 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/api.txt 2024-03-16T09:17:13.030 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/cpuid.txt 2024-03-16T09:17:13.031 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/ 2024-03-16T09:17:13.032 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/README 2024-03-16T09:17:13.034 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/arm-vgic.txt 2024-03-16T09:17:13.035 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/mpic.txt 2024-03-16T09:17:13.037 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/s390_flic.txt 2024-03-16T09:17:13.039 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/vfio.txt 2024-03-16T09:17:13.041 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/vm.txt 2024-03-16T09:17:13.043 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/xics.txt 2024-03-16T09:17:13.045 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/hypercalls.txt 2024-03-16T09:17:13.047 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/locking.txt 2024-03-16T09:17:13.048 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/mmu.txt 2024-03-16T09:17:13.050 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/msr.txt 2024-03-16T09:17:13.052 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/nested-vmx.txt 2024-03-16T09:17:13.054 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/ppc-pv.txt 2024-03-16T09:17:13.056 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/review-checklist.txt 2024-03-16T09:17:13.057 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/s390-diag.txt 2024-03-16T09:17:13.060 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/timekeeping.txt 2024-03-16T09:17:13.061 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/paravirt_ops.txt 2024-03-16T09:17:13.063 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/uml/ 2024-03-16T09:17:13.065 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/uml/UserModeLinux-HOWTO.txt 2024-03-16T09:17:13.068 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/ 2024-03-16T09:17:13.069 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/.gitignore 2024-03-16T09:17:13.071 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/00-INDEX 2024-03-16T09:17:13.073 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/active_mm.txt 2024-03-16T09:17:13.077 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/balance 2024-03-16T09:17:13.115 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/cleancache.txt 2024-03-16T09:17:13.117 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/frontswap.txt 2024-03-16T09:17:13.118 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/highmem.txt 2024-03-16T09:17:13.120 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/hugetlbpage.txt 2024-03-16T09:17:13.121 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/hwpoison.txt 2024-03-16T09:17:13.122 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/ksm.txt 2024-03-16T09:17:13.124 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/numa 2024-03-16T09:17:13.126 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/numa_memory_policy.txt 2024-03-16T09:17:13.127 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/overcommit-accounting 2024-03-16T09:17:13.128 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/page_migration 2024-03-16T09:17:13.129 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/page_owner.txt 2024-03-16T09:17:13.130 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/pagemap.txt 2024-03-16T09:17:13.131 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/remap_file_pages.txt 2024-03-16T09:17:13.133 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/slub.txt 2024-03-16T09:17:13.135 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/soft-dirty.txt 2024-03-16T09:17:13.135 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/split_page_table_lock 2024-03-16T09:17:13.136 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/transhuge.txt 2024-03-16T09:17:13.137 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/unevictable-lru.txt 2024-03-16T09:17:13.138 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/zswap.txt 2024-03-16T09:17:13.139 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vme_api.txt 2024-03-16T09:17:13.140 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/volatile-considered-harmful.txt 2024-03-16T09:17:13.142 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/ 2024-03-16T09:17:13.143 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/00-INDEX 2024-03-16T09:17:13.145 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/masters/ 2024-03-16T09:17:13.146 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/masters/00-INDEX 2024-03-16T09:17:13.148 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/masters/ds2482 2024-03-16T09:17:13.149 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/masters/ds2490 2024-03-16T09:17:13.151 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/masters/mxc-w1 2024-03-16T09:17:13.153 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/masters/omap-hdq 2024-03-16T09:17:13.154 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/masters/w1-gpio 2024-03-16T09:17:13.156 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/slaves/ 2024-03-16T09:17:13.157 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/slaves/00-INDEX 2024-03-16T09:17:13.158 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds2406 2024-03-16T09:17:13.159 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds2423 2024-03-16T09:17:13.161 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds28e04 2024-03-16T09:17:13.162 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/slaves/w1_therm 2024-03-16T09:17:13.164 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/w1.generic 2024-03-16T09:17:13.165 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/w1.netlink 2024-03-16T09:17:13.167 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/ 2024-03-16T09:17:13.168 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/Makefile 2024-03-16T09:17:13.169 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/convert_drivers_to_kernel_api.txt 2024-03-16T09:17:13.171 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/hpwdt.txt 2024-03-16T09:17:13.173 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/pcwd-watchdog.txt 2024-03-16T09:17:13.174 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/src/ 2024-03-16T09:17:13.176 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/src/.gitignore 2024-03-16T09:17:13.177 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/src/Makefile 2024-03-16T09:17:13.179 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/src/watchdog-simple.c 2024-03-16T09:17:13.181 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/src/watchdog-test.c 2024-03-16T09:17:13.182 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/watchdog-api.txt 2024-03-16T09:17:13.184 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/watchdog-kernel-api.txt 2024-03-16T09:17:13.185 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/watchdog-parameters.txt 2024-03-16T09:17:13.187 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/wdt.txt 2024-03-16T09:17:13.188 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/wimax/ 2024-03-16T09:17:13.190 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/wimax/README.i2400m 2024-03-16T09:17:13.192 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/wimax/README.wimax 2024-03-16T09:17:13.193 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/workqueue.txt 2024-03-16T09:17:13.195 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/ 2024-03-16T09:17:13.197 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/00-INDEX 2024-03-16T09:17:13.198 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/boot.txt 2024-03-16T09:17:13.199 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/early-microcode.txt 2024-03-16T09:17:13.201 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/earlyprintk.txt 2024-03-16T09:17:13.202 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/entry_64.txt 2024-03-16T09:17:13.204 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/exception-tables.txt 2024-03-16T09:17:13.205 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/i386/ 2024-03-16T09:17:13.206 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/i386/IO-APIC.txt 2024-03-16T09:17:13.207 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/intel_mpx.txt 2024-03-16T09:17:13.208 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/mtrr.txt 2024-03-16T09:17:13.209 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/pat.txt 2024-03-16T09:17:13.211 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/tlb.txt 2024-03-16T09:17:13.213 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/usb-legacy-support.txt 2024-03-16T09:17:13.214 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/x86_64/ 2024-03-16T09:17:13.215 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/x86_64/00-INDEX 2024-03-16T09:17:13.216 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/x86_64/boot-options.txt 2024-03-16T09:17:13.218 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/x86_64/cpu-hotplug-spec 2024-03-16T09:17:13.220 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/x86_64/fake-numa-for-cpusets 2024-03-16T09:17:13.221 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/x86_64/kernel-stacks 2024-03-16T09:17:13.223 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/x86_64/machinecheck 2024-03-16T09:17:13.224 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/x86_64/mm.txt 2024-03-16T09:17:13.225 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/x86_64/uefi.txt 2024-03-16T09:17:13.227 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/zero-page.txt 2024-03-16T09:17:13.229 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/xillybus.txt 2024-03-16T09:17:13.230 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/xtensa/ 2024-03-16T09:17:13.232 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/xtensa/atomctl.txt 2024-03-16T09:17:13.233 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/xtensa/mmu.txt 2024-03-16T09:17:13.235 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/xz.txt 2024-03-16T09:17:13.236 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/ 2024-03-16T09:17:13.237 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/CodingStyle 2024-03-16T09:17:13.237 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/HOWTO 2024-03-16T09:17:13.238 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/IRQ.txt 2024-03-16T09:17:13.239 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/SecurityBugs 2024-03-16T09:17:13.240 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/SubmittingDrivers 2024-03-16T09:17:13.241 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/SubmittingPatches 2024-03-16T09:17:13.242 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/arm/ 2024-03-16T09:17:13.243 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/arm/Booting 2024-03-16T09:17:13.244 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/arm/kernel_user_helpers.txt 2024-03-16T09:17:13.245 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/arm64/ 2024-03-16T09:17:13.246 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/arm64/booting.txt 2024-03-16T09:17:13.248 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/arm64/memory.txt 2024-03-16T09:17:13.249 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/arm64/tagged-pointers.txt 2024-03-16T09:17:13.251 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/basic_profiling.txt 2024-03-16T09:17:13.252 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/email-clients.txt 2024-03-16T09:17:13.254 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/filesystems/ 2024-03-16T09:17:13.255 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/filesystems/sysfs.txt 2024-03-16T09:17:13.257 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/gpio.txt 2024-03-16T09:17:13.258 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/io_ordering.txt 2024-03-16T09:17:13.260 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/magic-number.txt 2024-03-16T09:17:13.261 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/oops-tracing.txt 2024-03-16T09:17:13.263 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/sparse.txt 2024-03-16T09:17:13.264 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/stable_api_nonsense.txt 2024-03-16T09:17:13.266 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/stable_kernel_rules.txt 2024-03-16T09:17:13.267 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/ 2024-03-16T09:17:13.267 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/omap3isp.txt 2024-03-16T09:17:13.268 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/v4l2-framework.txt 2024-03-16T09:17:13.269 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/volatile-considered-harmful.txt 2024-03-16T09:17:13.270 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zorro.txt 2024-03-16T09:17:13.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:13 smithi194 ceph-mon[36729]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:17:13.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:13 smithi159 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:17:13.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:13 smithi112 ceph-mon[32307]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:17:14.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.011+0000 7fa12cfd4640 1 -- 172.21.15.112:0/306697062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1281033e0 msgr2=0x7fa128109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.011+0000 7fa12cfd4640 1 --2- 172.21.15.112:0/306697062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1281033e0 0x7fa128109c70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa120009ca0 tx=0x7fa12002c0f0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.011+0000 7fa12cfd4640 1 -- 172.21.15.112:0/306697062 shutdown_connections 2024-03-16T09:17:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.011+0000 7fa12cfd4640 1 --2- 172.21.15.112:0/306697062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1281033e0 0x7fa128109c70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.011+0000 7fa12cfd4640 1 --2- 172.21.15.112:0/306697062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa128102a20 0x7fa128102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.011+0000 7fa12cfd4640 1 --2- 172.21.15.112:0/306697062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa128101830 0x7fa128101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.011+0000 7fa12cfd4640 1 -- 172.21.15.112:0/306697062 >> 172.21.15.112:0/306697062 conn(0x7fa1280fcda0 msgr2=0x7fa1280ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.011+0000 7fa12cfd4640 1 -- 172.21.15.112:0/306697062 shutdown_connections 2024-03-16T09:17:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.011+0000 7fa12cfd4640 1 -- 172.21.15.112:0/306697062 wait complete. 2024-03-16T09:17:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa12cfd4640 1 Processor -- start 2024-03-16T09:17:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa12cfd4640 1 -- start start 2024-03-16T09:17:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa12cfd4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa128101830 0x7fa128197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa12cfd4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa128102a20 0x7fa128198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa12cfd4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1281033e0 0x7fa12819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa12cfd4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa12810d5e0 con 0x7fa1281033e0 2024-03-16T09:17:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa12cfd4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa12810d460 con 0x7fa128102a20 2024-03-16T09:17:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa12cfd4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa12810d760 con 0x7fa128101830 2024-03-16T09:17:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa126d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1281033e0 0x7fa12819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa125d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa128102a20 0x7fa128198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa12659c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa128101830 0x7fa128197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa126d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1281033e0 0x7fa12819f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42600/0 (socket says 172.21.15.112:42600) 2024-03-16T09:17:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa125d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa128102a20 0x7fa128198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:57046/0 (socket says 172.21.15.112:57046) 2024-03-16T09:17:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa126d9d640 1 -- 172.21.15.112:0/1129385058 learned_addr learned my addr 172.21.15.112:0/1129385058 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa12659c640 1 -- 172.21.15.112:0/1129385058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa128102a20 msgr2=0x7fa128198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa12659c640 1 --2- 172.21.15.112:0/1129385058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa128102a20 0x7fa128198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa12659c640 1 -- 172.21.15.112:0/1129385058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1281033e0 msgr2=0x7fa12819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa12659c640 1 --2- 172.21.15.112:0/1129385058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1281033e0 0x7fa12819f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa12659c640 1 -- 172.21.15.112:0/1129385058 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12819f720 con 0x7fa128101830 2024-03-16T09:17:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa126d9d640 1 --2- 172.21.15.112:0/1129385058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1281033e0 0x7fa12819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa125d9b640 1 --2- 172.21.15.112:0/1129385058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa128102a20 0x7fa128198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.012+0000 7fa12659c640 1 --2- 172.21.15.112:0/1129385058 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa128101830 0x7fa128197b00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa11800e9c0 tx=0x7fa11800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.013+0000 7fa1177fe640 1 -- 172.21.15.112:0/1129385058 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11800ccc0 con 0x7fa128101830 2024-03-16T09:17:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.013+0000 7fa1177fe640 1 -- 172.21.15.112:0/1129385058 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa1180035f0 con 0x7fa128101830 2024-03-16T09:17:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.013+0000 7fa12cfd4640 1 -- 172.21.15.112:0/1129385058 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12819fa10 con 0x7fa128101830 2024-03-16T09:17:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.013+0000 7fa1177fe640 1 -- 172.21.15.112:0/1129385058 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11800ce60 con 0x7fa128101830 2024-03-16T09:17:14.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.013+0000 7fa12cfd4640 1 -- 172.21.15.112:0/1129385058 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa12819ff10 con 0x7fa128101830 2024-03-16T09:17:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.015+0000 7fa1177fe640 1 -- 172.21.15.112:0/1129385058 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7fa1180037a0 con 0x7fa128101830 2024-03-16T09:17:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.015+0000 7fa12fbc1740 1 Processor -- start 2024-03-16T09:17:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.015+0000 7fa1177fe640 1 --2- 172.21.15.112:0/1129385058 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa1100a28a0 0x7fa1100a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.015+0000 7fa126d9d640 1 --2- 172.21.15.112:0/1129385058 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa1100a28a0 0x7fa1100a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.015+0000 7fa1177fe640 1 -- 172.21.15.112:0/1129385058 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa1180cedb0 con 0x7fa128101830 2024-03-16T09:17:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.015+0000 7fa12fbc1740 1 -- start start 2024-03-16T09:17:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa12fbc1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2223340 0x55dec2223760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa12fbc1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2223d30 0x55dec2228830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa12fbc1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2228ea0 0x55dec222b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:14.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa12fbc1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dec2188a40 con 0x55dec2228ea0 2024-03-16T09:17:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa12fbc1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dec2131920 con 0x55dec2223340 2024-03-16T09:17:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa12fbc1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55dec2184ae0 con 0x55dec2223d30 2024-03-16T09:17:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa125d9b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2223d30 0x55dec2228830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa12659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2223340 0x55dec2223760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa125d9b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2223d30 0x55dec2228830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:34314/0 (socket says 172.21.15.112:34314) 2024-03-16T09:17:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa125d9b640 1 -- 172.21.15.112:0/2049138716 learned_addr learned my addr 172.21.15.112:0/2049138716 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa126d9d640 1 --2- 172.21.15.112:0/2049138716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2228ea0 0x55dec222b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa126d9d640 1 --2- 172.21.15.112:0/1129385058 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa1100a28a0 0x7fa1100a4d60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa120035fd0 tx=0x7fa120032690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa12659c640 1 -- 172.21.15.112:0/2049138716 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2223d30 msgr2=0x55dec2228830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa12659c640 1 --2- 172.21.15.112:0/2049138716 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2223d30 0x55dec2228830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa12659c640 1 -- 172.21.15.112:0/2049138716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2228ea0 msgr2=0x55dec222b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa12659c640 1 --2- 172.21.15.112:0/2049138716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2228ea0 0x55dec222b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa12659c640 1 -- 172.21.15.112:0/2049138716 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dec2149460 con 0x55dec2223340 2024-03-16T09:17:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa12659c640 1 --2- 172.21.15.112:0/2049138716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2223340 0x55dec2223760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa1180ccc30 tx=0x7fa11809be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=675f05ed8181f16e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa114ff9640 1 -- 172.21.15.112:0/2049138716 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11809f8b0 con 0x55dec2223340 2024-03-16T09:17:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa114ff9640 1 -- 172.21.15.112:0/2049138716 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa1180a3f00 con 0x55dec2223340 2024-03-16T09:17:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.016+0000 7fa114ff9640 1 -- 172.21.15.112:0/2049138716 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1180a3cd0 con 0x55dec2223340 2024-03-16T09:17:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.017+0000 7fa12fbc1740 1 -- 172.21.15.112:0/2049138716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2223340 msgr2=0x55dec2223760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.017+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/2049138716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2223340 0x55dec2223760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa1180ccc30 tx=0x7fa11809be60 comp rx=0 tx=0).stop 2024-03-16T09:17:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.017+0000 7fa12fbc1740 1 -- 172.21.15.112:0/2049138716 shutdown_connections 2024-03-16T09:17:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.017+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/2049138716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2228ea0 0x55dec222b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.017+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/2049138716 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2223d30 0x55dec2228830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.017+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/2049138716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2223340 0x55dec2223760 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.017+0000 7fa12fbc1740 1 -- 172.21.15.112:0/2049138716 >> 172.21.15.112:0/2049138716 conn(0x55dec221e8b0 msgr2=0x55dec221ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.017+0000 7fa12fbc1740 1 -- 172.21.15.112:0/2049138716 shutdown_connections 2024-03-16T09:17:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.017+0000 7fa12fbc1740 1 -- 172.21.15.112:0/2049138716 wait complete. 2024-03-16T09:17:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa12fbc1740 1 -- start start 2024-03-16T09:17:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa12fbc1740 1 Processor -- start 2024-03-16T09:17:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa12fbc1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2223340 0x55dec222d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa12fbc1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223d30 0x55dec222dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa12fbc1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2228ea0 0x55dec2234c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa12fbc1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dec222be50 con 0x55dec2223d30 2024-03-16T09:17:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa12fbc1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dec2144d70 con 0x55dec2228ea0 2024-03-16T09:17:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa12fbc1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55dec222bfd0 con 0x55dec2223340 2024-03-16T09:17:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa12fbc1740 10 client.0 _ll_register_callbacks cb 0x55dec1e9ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa12659c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2223340 0x55dec222d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa125d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223d30 0x55dec222dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa12659c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2223340 0x55dec222d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:34318/0 (socket says 172.21.15.112:34318) 2024-03-16T09:17:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa12659c640 1 -- 172.21.15.112:0/840561307 learned_addr learned my addr 172.21.15.112:0/840561307 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa126d9d640 1 --2- 172.21.15.112:0/840561307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2228ea0 0x55dec2234c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa12659c640 1 -- 172.21.15.112:0/840561307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2228ea0 msgr2=0x55dec2234c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa12659c640 1 --2- 172.21.15.112:0/840561307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2228ea0 0x55dec2234c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa12659c640 1 -- 172.21.15.112:0/840561307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223d30 msgr2=0x55dec222dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa12659c640 1 --2- 172.21.15.112:0/840561307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223d30 0x55dec222dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa12659c640 1 -- 172.21.15.112:0/840561307 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dec1e9d210 con 0x55dec2223340 2024-03-16T09:17:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.018+0000 7fa125d9b640 1 --2- 172.21.15.112:0/840561307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223d30 0x55dec222dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.019+0000 7fa12659c640 1 --2- 172.21.15.112:0/840561307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2223340 0x55dec222d710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa1180cc910 tx=0x7fa11809f930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=278d55ef41d4cff4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.019+0000 7fa10dffb640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T09:17:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.019+0000 7fa10dffb640 1 -- 172.21.15.112:0/840561307 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1180b8070 con 0x55dec2223340 2024-03-16T09:17:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.019+0000 7fa10dffb640 1 -- 172.21.15.112:0/840561307 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa1180a3a70 con 0x55dec2223340 2024-03-16T09:17:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.019+0000 7fa12fbc1740 1 -- 172.21.15.112:0/840561307 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1180a3f00 con 0x55dec2223340 2024-03-16T09:17:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.019+0000 7fa10dffb640 1 -- 172.21.15.112:0/840561307 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa118099ae0 con 0x55dec2223340 2024-03-16T09:17:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.019+0000 7fa12fbc1740 10 client.25615 fetch_fsmap learned FSMap version 23 2024-03-16T09:17:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.019+0000 7fa12fbc1740 1 -- 172.21.15.112:0/840561307 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55dec1f69620 con 0x55dec2223340 2024-03-16T09:17:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.019+0000 7fa10dffb640 1 -- 172.21.15.112:0/840561307 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa1180a3f00 con 0x55dec2223340 2024-03-16T09:17:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.019+0000 7fa10dffb640 1 -- 172.21.15.112:0/840561307 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fa118003fe0 con 0x55dec2223340 2024-03-16T09:17:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.019+0000 7fa12fbc1740 10 client.25615 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T09:17:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.019+0000 7fa12fbc1740 10 client.25615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.019+0000 7fa12fbc1740 20 client.25615 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.019+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/840561307 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55dec2239110 0x55dec223b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.019+0000 7fa12fbc1740 4 client.25615 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:17:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.019+0000 7fa12fbc1740 1 -- 172.21.15.112:0/840561307 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dec2195f80 con 0x55dec2239110 2024-03-16T09:17:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.020+0000 7fa125d9b640 1 --2- 172.21.15.112:0/840561307 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55dec2239110 0x55dec223b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.020+0000 7fa10dffb640 10 client.25615 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.020+0000 7fa125d9b640 1 --2- 172.21.15.112:0/840561307 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55dec2239110 0x55dec223b500 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad6870f6920189d1 server_cookie=259d91fc8141125f in_seq=0 out_seq=0 2024-03-16T09:17:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.021+0000 7fa10dffb640 1 -- 172.21.15.112:0/840561307 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55dec2195f80 con 0x55dec2239110 2024-03-16T09:17:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.021+0000 7fa10dffb640 10 client.25615 handle_command_reply: tid=0 2024-03-16T09:17:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.021+0000 7fa12fbc1740 1 client.25615 shutdown 2024-03-16T09:17:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.021+0000 7fa12fbc1740 1 -- 172.21.15.112:0/840561307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2223340 msgr2=0x55dec222d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.021+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/840561307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2223340 0x55dec222d710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa1180cc910 tx=0x7fa11809f930 comp rx=0 tx=0).stop 2024-03-16T09:17:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.021+0000 7fa12fbc1740 1 -- 172.21.15.112:0/840561307 shutdown_connections 2024-03-16T09:17:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.021+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/840561307 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55dec2239110 0x55dec223b500 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.021+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/840561307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2228ea0 0x55dec2234c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.021+0000 7fa10dffb640 0 client.25615 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.021+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/840561307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223d30 0x55dec222dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.021+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/840561307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2223340 0x55dec222d710 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.021+0000 7fa12fbc1740 1 -- 172.21.15.112:0/840561307 >> 172.21.15.112:0/840561307 conn(0x55dec221e8b0 msgr2=0x55dec221dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.021+0000 7fa12fbc1740 1 -- 172.21.15.112:0/840561307 shutdown_connections 2024-03-16T09:17:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.021+0000 7fa12fbc1740 1 -- 172.21.15.112:0/840561307 wait complete. 2024-03-16T09:17:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.021+0000 7fa12fbc1740 20 client.25615 trim_cache size 0 max 16384 2024-03-16T09:17:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.035+0000 7fa12fbc1740 1 Processor -- start 2024-03-16T09:17:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.035+0000 7fa12fbc1740 1 -- start start 2024-03-16T09:17:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.035+0000 7fa12fbc1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223340 0x55dec222dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.035+0000 7fa12fbc1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2223d30 0x55dec221d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.035+0000 7fa12fbc1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2228ea0 0x55dec221d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.035+0000 7fa12fbc1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281ab5c0 con 0x55dec2223340 2024-03-16T09:17:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.035+0000 7fa12fbc1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281a8e90 con 0x55dec2223d30 2024-03-16T09:17:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.035+0000 7fa12fbc1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281a6ce0 con 0x55dec2228ea0 2024-03-16T09:17:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.036+0000 7fa12659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223340 0x55dec222dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.036+0000 7fa12659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223340 0x55dec222dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42614/0 (socket says 172.21.15.112:42614) 2024-03-16T09:17:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.036+0000 7fa12659c640 1 -- 172.21.15.112:0/168104773 learned_addr learned my addr 172.21.15.112:0/168104773 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.036+0000 7fa125d9b640 1 --2- 172.21.15.112:0/168104773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2223d30 0x55dec221d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.036+0000 7fa126d9d640 1 --2- 172.21.15.112:0/168104773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2228ea0 0x55dec221d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.036+0000 7fa12659c640 1 -- 172.21.15.112:0/168104773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2228ea0 msgr2=0x55dec221d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.036+0000 7fa12659c640 1 --2- 172.21.15.112:0/168104773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2228ea0 0x55dec221d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.036+0000 7fa12659c640 1 -- 172.21.15.112:0/168104773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2223d30 msgr2=0x55dec221d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.036+0000 7fa12659c640 1 --2- 172.21.15.112:0/168104773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2223d30 0x55dec221d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.036+0000 7fa12659c640 1 -- 172.21.15.112:0/168104773 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dec221f620 con 0x55dec2223340 2024-03-16T09:17:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.036+0000 7fa125d9b640 1 --2- 172.21.15.112:0/168104773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2223d30 0x55dec221d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.036+0000 7fa12659c640 1 --2- 172.21.15.112:0/168104773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223340 0x55dec222dbb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa11800c6a0 tx=0x7fa118086050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de28c827165c24e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.036+0000 7fa10ffff640 1 -- 172.21.15.112:0/168104773 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1180b8070 con 0x55dec2223340 2024-03-16T09:17:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.036+0000 7fa10ffff640 1 -- 172.21.15.112:0/168104773 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa1180a3f00 con 0x55dec2223340 2024-03-16T09:17:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.036+0000 7fa10ffff640 1 -- 172.21.15.112:0/168104773 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11809abb0 con 0x55dec2223340 2024-03-16T09:17:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.037+0000 7fa12fbc1740 1 -- 172.21.15.112:0/168104773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223340 msgr2=0x55dec222dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.037+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/168104773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223340 0x55dec222dbb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa11800c6a0 tx=0x7fa118086050 comp rx=0 tx=0).stop 2024-03-16T09:17:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.037+0000 7fa12fbc1740 1 -- 172.21.15.112:0/168104773 shutdown_connections 2024-03-16T09:17:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.037+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/168104773 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2228ea0 0x55dec221d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.037+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/168104773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2223d30 0x55dec221d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.037+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/168104773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223340 0x55dec222dbb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.037+0000 7fa12fbc1740 1 -- 172.21.15.112:0/168104773 >> 172.21.15.112:0/168104773 conn(0x55dec221e8b0 msgr2=0x55dec221ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.037+0000 7fa12fbc1740 1 -- 172.21.15.112:0/168104773 shutdown_connections 2024-03-16T09:17:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.037+0000 7fa12fbc1740 1 -- 172.21.15.112:0/168104773 wait complete. 2024-03-16T09:17:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.037+0000 7fa12fbc1740 1 -- start start 2024-03-16T09:17:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.037+0000 7fa12fbc1740 1 Processor -- start 2024-03-16T09:17:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.037+0000 7fa12fbc1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2223340 0x55dec2232650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.037+0000 7fa12fbc1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223d30 0x55dec2232b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.037+0000 7fa12fbc1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2228ea0 0x55dec2233b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.037+0000 7fa12fbc1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281a7350 con 0x55dec2223d30 2024-03-16T09:17:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa12fbc1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281a87c0 con 0x55dec2223340 2024-03-16T09:17:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa12fbc1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa1281a7a20 con 0x55dec2228ea0 2024-03-16T09:17:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa12fbc1740 10 client.0 _ll_register_callbacks cb 0x7fa11c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa125d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223d30 0x55dec2232b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa125d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223d30 0x55dec2232b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42630/0 (socket says 172.21.15.112:42630) 2024-03-16T09:17:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa125d9b640 1 -- 172.21.15.112:0/4076722746 learned_addr learned my addr 172.21.15.112:0/4076722746 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa12659c640 1 --2- 172.21.15.112:0/4076722746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2223340 0x55dec2232650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa126d9d640 1 --2- 172.21.15.112:0/4076722746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2228ea0 0x55dec2233b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa125d9b640 1 -- 172.21.15.112:0/4076722746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2228ea0 msgr2=0x55dec2233b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa125d9b640 1 --2- 172.21.15.112:0/4076722746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2228ea0 0x55dec2233b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa125d9b640 1 -- 172.21.15.112:0/4076722746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2223340 msgr2=0x55dec2232650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa125d9b640 1 --2- 172.21.15.112:0/4076722746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2223340 0x55dec2232650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa125d9b640 1 -- 172.21.15.112:0/4076722746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dec22340f0 con 0x55dec2223d30 2024-03-16T09:17:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa12659c640 1 --2- 172.21.15.112:0/4076722746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2223340 0x55dec2232650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa125d9b640 1 --2- 172.21.15.112:0/4076722746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223d30 0x55dec2232b90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa11c004700 tx=0x7fa11c01f7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d3c08376bbc6a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa10dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa10dffb640 1 -- 172.21.15.112:0/4076722746 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c012e30 con 0x55dec2223d30 2024-03-16T09:17:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa10dffb640 1 -- 172.21.15.112:0/4076722746 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fa11c004050 con 0x55dec2223d30 2024-03-16T09:17:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa12fbc1740 1 -- 172.21.15.112:0/4076722746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1180a3f00 con 0x55dec2223d30 2024-03-16T09:17:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa10dffb640 1 -- 172.21.15.112:0/4076722746 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa11c004250 con 0x55dec2223d30 2024-03-16T09:17:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa12fbc1740 10 client.15759 fetch_fsmap learned FSMap version 23 2024-03-16T09:17:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.038+0000 7fa12fbc1740 1 -- 172.21.15.112:0/4076722746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55dec229f970 con 0x55dec2223d30 2024-03-16T09:17:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.039+0000 7fa10dffb640 1 -- 172.21.15.112:0/4076722746 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa1180a3f00 con 0x55dec2223d30 2024-03-16T09:17:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.039+0000 7fa10dffb640 1 -- 172.21.15.112:0/4076722746 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fa11c0037f0 con 0x55dec2223d30 2024-03-16T09:17:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.039+0000 7fa12fbc1740 10 client.15759 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T09:17:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.039+0000 7fa12fbc1740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.039+0000 7fa12fbc1740 20 client.15759 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.039+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/4076722746 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55dec2239110 0x55dec22a2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.039+0000 7fa12fbc1740 4 client.15759 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T09:17:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.039+0000 7fa12fbc1740 1 -- 172.21.15.112:0/4076722746 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dec21902d0 con 0x55dec2239110 2024-03-16T09:17:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.039+0000 7fa12659c640 1 --2- 172.21.15.112:0/4076722746 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55dec2239110 0x55dec22a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.039+0000 7fa10dffb640 10 client.15759 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.039+0000 7fa12659c640 1 --2- 172.21.15.112:0/4076722746 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55dec2239110 0x55dec22a2490 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16dd0c8eb9532771 server_cookie=949b9716934fa7d in_seq=0 out_seq=0 2024-03-16T09:17:14.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:14 smithi194 ceph-mon[36729]: pgmap v398: 97 pgs: 97 active+clean; 87 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 7.3 MiB/s wr, 37 op/s 2024-03-16T09:17:14.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:14 smithi159 ceph-mon[36739]: pgmap v398: 97 pgs: 97 active+clean; 87 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 7.3 MiB/s wr, 37 op/s 2024-03-16T09:17:14.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:14 smithi112 ceph-mon[32307]: pgmap v398: 97 pgs: 97 active+clean; 87 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 7.3 MiB/s wr, 37 op/s 2024-03-16T09:17:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.600+0000 7fa10dffb640 1 -- 172.21.15.112:0/4076722746 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dec21902d0 con 0x55dec2239110 2024-03-16T09:17:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.600+0000 7fa10dffb640 10 client.15759 handle_command_reply: tid=0 2024-03-16T09:17:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.600+0000 7fa12fbc1740 1 client.15759 shutdown 2024-03-16T09:17:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.600+0000 7fa12fbc1740 1 -- 172.21.15.112:0/4076722746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223d30 msgr2=0x55dec2232b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.600+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/4076722746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223d30 0x55dec2232b90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa11c004700 tx=0x7fa11c01f7c0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.600+0000 7fa12fbc1740 1 -- 172.21.15.112:0/4076722746 shutdown_connections 2024-03-16T09:17:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.600+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/4076722746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec2228ea0 0x55dec2233b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.600+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/4076722746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dec2223d30 0x55dec2232b90 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.600+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/4076722746 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55dec2239110 0x55dec22a2490 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.600+0000 7fa12fbc1740 1 --2- 172.21.15.112:0/4076722746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dec2223340 0x55dec2232650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.600+0000 7fa10dffb640 0 client.15759 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.600+0000 7fa12fbc1740 1 -- 172.21.15.112:0/4076722746 >> 172.21.15.112:0/4076722746 conn(0x55dec221e8b0 msgr2=0x55dec2221b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.600+0000 7fa12fbc1740 1 -- 172.21.15.112:0/4076722746 shutdown_connections 2024-03-16T09:17:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.600+0000 7fa12fbc1740 1 -- 172.21.15.112:0/4076722746 wait complete. 2024-03-16T09:17:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.600+0000 7fa12fbc1740 20 client.15759 trim_cache size 0 max 16384 2024-03-16T09:17:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-03-16T09:17:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-03-16T09:17:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-03-16T09:17:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-03-16T09:17:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.601+0000 7fa12cfd4640 1 -- 172.21.15.112:0/1129385058 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa1100a28a0 msgr2=0x7fa1100a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.601+0000 7fa12cfd4640 1 --2- 172.21.15.112:0/1129385058 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa1100a28a0 0x7fa1100a4d60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa120035fd0 tx=0x7fa120032690 comp rx=0 tx=0).stop 2024-03-16T09:17:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.601+0000 7fa12cfd4640 1 -- 172.21.15.112:0/1129385058 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa128101830 msgr2=0x7fa128197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.601+0000 7fa12cfd4640 1 --2- 172.21.15.112:0/1129385058 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa128101830 0x7fa128197b00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa11800e9c0 tx=0x7fa11800ee80 comp rx=0 tx=0).stop 2024-03-16T09:17:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.601+0000 7fa12cfd4640 1 -- 172.21.15.112:0/1129385058 shutdown_connections 2024-03-16T09:17:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.601+0000 7fa12cfd4640 1 --2- 172.21.15.112:0/1129385058 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fa1100a28a0 0x7fa1100a4d60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.601+0000 7fa12cfd4640 1 --2- 172.21.15.112:0/1129385058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1281033e0 0x7fa12819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.601+0000 7fa12cfd4640 1 --2- 172.21.15.112:0/1129385058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa128102a20 0x7fa128198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.601+0000 7fa12cfd4640 1 --2- 172.21.15.112:0/1129385058 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa128101830 0x7fa128197b00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.601+0000 7fa12cfd4640 1 -- 172.21.15.112:0/1129385058 >> 172.21.15.112:0/1129385058 conn(0x7fa1280fcda0 msgr2=0x7fa128105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.601+0000 7fa12cfd4640 1 -- 172.21.15.112:0/1129385058 shutdown_connections 2024-03-16T09:17:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:14.601+0000 7fa12cfd4640 1 -- 172.21.15.112:0/1129385058 wait complete. 2024-03-16T09:17:15.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:15 smithi194 ceph-mon[36729]: pgmap v399: 97 pgs: 97 active+clean; 89 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 7.4 MiB/s wr, 112 op/s 2024-03-16T09:17:15.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:15 smithi159 ceph-mon[36739]: pgmap v399: 97 pgs: 97 active+clean; 89 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 7.4 MiB/s wr, 112 op/s 2024-03-16T09:17:15.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:15 smithi112 ceph-mon[32307]: pgmap v399: 97 pgs: 97 active+clean; 89 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 7.4 MiB/s wr, 112 op/s 2024-03-16T09:17:17.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T09:17:17.166 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T09:17:17.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.271+0000 7f9a4b459640 1 -- 172.21.15.112:0/2300182951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a44107560 msgr2=0x7f9a44109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.271+0000 7f9a4b459640 1 --2- 172.21.15.112:0/2300182951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a44107560 0x7f9a44109950 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c009ca0 tx=0x7f9a3c02c0f0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.271+0000 7f9a4b459640 1 -- 172.21.15.112:0/2300182951 shutdown_connections 2024-03-16T09:17:17.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.271+0000 7f9a4b459640 1 --2- 172.21.15.112:0/2300182951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a44107560 0x7f9a44109950 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.271+0000 7f9a4b459640 1 --2- 172.21.15.112:0/2300182951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a44104c30 0x7f9a44107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.271+0000 7f9a4b459640 1 --2- 172.21.15.112:0/2300182951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a44102300 0x7f9a441046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.271+0000 7f9a4b459640 1 -- 172.21.15.112:0/2300182951 >> 172.21.15.112:0/2300182951 conn(0x7f9a440fbc70 msgr2=0x7f9a440fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.271+0000 7f9a4b459640 1 -- 172.21.15.112:0/2300182951 shutdown_connections 2024-03-16T09:17:17.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.271+0000 7f9a4b459640 1 -- 172.21.15.112:0/2300182951 wait complete. 2024-03-16T09:17:17.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.271+0000 7f9a4b459640 1 Processor -- start 2024-03-16T09:17:17.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a4b459640 1 -- start start 2024-03-16T09:17:17.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a4b459640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a44102300 0x7f9a44197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a4b459640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a44104c30 0x7f9a44198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a4b459640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a44107560 0x7f9a4419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a4b459640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4410d5b0 con 0x7f9a44104c30 2024-03-16T09:17:17.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a4b459640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4410d430 con 0x7f9a44102300 2024-03-16T09:17:17.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a4b459640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4410d730 con 0x7f9a44107560 2024-03-16T09:17:17.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a489f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a44104c30 0x7f9a44198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a489f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a44104c30 0x7f9a44198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42648/0 (socket says 172.21.15.112:42648) 2024-03-16T09:17:17.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a489f4640 1 -- 172.21.15.112:0/471163969 learned_addr learned my addr 172.21.15.112:0/471163969 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:17.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a491f5640 1 --2- 172.21.15.112:0/471163969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a44102300 0x7f9a44197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a499f6640 1 --2- 172.21.15.112:0/471163969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a44107560 0x7f9a4419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a489f4640 1 -- 172.21.15.112:0/471163969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a44107560 msgr2=0x7f9a4419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a489f4640 1 --2- 172.21.15.112:0/471163969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a44107560 0x7f9a4419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a489f4640 1 -- 172.21.15.112:0/471163969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a44102300 msgr2=0x7f9a44197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a489f4640 1 --2- 172.21.15.112:0/471163969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a44102300 0x7f9a44197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a489f4640 1 -- 172.21.15.112:0/471163969 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4419f810 con 0x7f9a44104c30 2024-03-16T09:17:17.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a489f4640 1 --2- 172.21.15.112:0/471163969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a44104c30 0x7f9a44198130 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400e9c0 tx=0x7f9a3400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a3a7fc640 1 -- 172.21.15.112:0/471163969 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3400cd80 con 0x7f9a44104c30 2024-03-16T09:17:17.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a4b459640 1 -- 172.21.15.112:0/471163969 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4419fb00 con 0x7f9a44104c30 2024-03-16T09:17:17.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.272+0000 7f9a3a7fc640 1 -- 172.21.15.112:0/471163969 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9a34005520 con 0x7f9a44104c30 2024-03-16T09:17:17.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.273+0000 7f9a3a7fc640 1 -- 172.21.15.112:0/471163969 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34018630 con 0x7f9a44104c30 2024-03-16T09:17:17.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.273+0000 7f9a4b459640 1 -- 172.21.15.112:0/471163969 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a441a0050 con 0x7f9a44104c30 2024-03-16T09:17:17.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.275+0000 7f9a3a7fc640 1 -- 172.21.15.112:0/471163969 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7f9a340046c0 con 0x7f9a44104c30 2024-03-16T09:17:17.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.275+0000 7f9a4e040740 1 Processor -- start 2024-03-16T09:17:17.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.275+0000 7f9a4e040740 1 -- start start 2024-03-16T09:17:17.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.275+0000 7f9a3a7fc640 1 --2- 172.21.15.112:0/471163969 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9a300a2800 0x7f9a300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a4e040740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4b7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a4e040740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4b7b60 0x562eeb4bc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a4e040740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4bccd0 0x562eeb4bf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a4e040740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562eeb41c500 con 0x562eeb4bccd0 2024-03-16T09:17:17.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a4e040740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562eeb3c4be0 con 0x562eeb4b7b60 2024-03-16T09:17:17.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a4e040740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562eeb4185a0 con 0x562eeb4b7170 2024-03-16T09:17:17.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a3a7fc640 1 -- 172.21.15.112:0/471163969 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f9a340ceef0 con 0x7f9a44104c30 2024-03-16T09:17:17.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a499f6640 1 --2- 172.21.15.112:0/471163969 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9a300a2800 0x7f9a300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a491f5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a491f5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4b7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:34362/0 (socket says 172.21.15.112:34362) 2024-03-16T09:17:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a491f5640 1 -- 172.21.15.112:0/4232533372 learned_addr learned my addr 172.21.15.112:0/4232533372 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a489f4640 1 --2- 172.21.15.112:0/4232533372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4b7b60 0x562eeb4bc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a499f6640 1 --2- 172.21.15.112:0/4232533372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4bccd0 0x562eeb4bf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a491f5640 1 -- 172.21.15.112:0/4232533372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4b7b60 msgr2=0x562eeb4bc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a491f5640 1 --2- 172.21.15.112:0/4232533372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4b7b60 0x562eeb4bc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a491f5640 1 -- 172.21.15.112:0/4232533372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4bccd0 msgr2=0x562eeb4bf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a491f5640 1 --2- 172.21.15.112:0/4232533372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4bccd0 0x562eeb4bf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a491f5640 1 -- 172.21.15.112:0/4232533372 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eeb3d84b0 con 0x562eeb4b7170 2024-03-16T09:17:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a499f6640 1 --2- 172.21.15.112:0/471163969 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9a300a2800 0x7f9a300a4cc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c005e30 tx=0x7f9a3c032e10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a491f5640 1 --2- 172.21.15.112:0/4232533372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4b7590 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009d20 tx=0x7f9a40011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e1c15cb7347cca3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.276+0000 7f9a2f7fe640 1 -- 172.21.15.112:0/4232533372 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4001e070 con 0x562eeb4b7170 2024-03-16T09:17:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.277+0000 7f9a2f7fe640 1 -- 172.21.15.112:0/4232533372 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9a40009e10 con 0x562eeb4b7170 2024-03-16T09:17:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.277+0000 7f9a2f7fe640 1 -- 172.21.15.112:0/4232533372 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a400189a0 con 0x562eeb4b7170 2024-03-16T09:17:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.277+0000 7f9a4e040740 1 -- 172.21.15.112:0/4232533372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 msgr2=0x562eeb4b7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.277+0000 7f9a4e040740 1 --2- 172.21.15.112:0/4232533372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4b7590 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009d20 tx=0x7f9a40011810 comp rx=0 tx=0).stop 2024-03-16T09:17:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.277+0000 7f9a4e040740 1 -- 172.21.15.112:0/4232533372 shutdown_connections 2024-03-16T09:17:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.277+0000 7f9a4e040740 1 --2- 172.21.15.112:0/4232533372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4bccd0 0x562eeb4bf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.277+0000 7f9a4e040740 1 --2- 172.21.15.112:0/4232533372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4b7b60 0x562eeb4bc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.277+0000 7f9a4e040740 1 --2- 172.21.15.112:0/4232533372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4b7590 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.277+0000 7f9a4e040740 1 -- 172.21.15.112:0/4232533372 >> 172.21.15.112:0/4232533372 conn(0x562eeb4b26e0 msgr2=0x562eeb4b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.277+0000 7f9a4e040740 1 -- 172.21.15.112:0/4232533372 shutdown_connections 2024-03-16T09:17:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.277+0000 7f9a4e040740 1 -- 172.21.15.112:0/4232533372 wait complete. 2024-03-16T09:17:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a4e040740 1 -- start start 2024-03-16T09:17:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a4e040740 1 Processor -- start 2024-03-16T09:17:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a4e040740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4c16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a4e040740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4b7b60 0x562eeb4c1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a4e040740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4bccd0 0x562eeb4c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a4e040740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562eeb4bfc80 con 0x562eeb4b7b60 2024-03-16T09:17:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a4e040740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562eeb3d8160 con 0x562eeb4bccd0 2024-03-16T09:17:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a4e040740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562eeb4bfe00 con 0x562eeb4b7170 2024-03-16T09:17:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a4e040740 10 client.0 _ll_register_callbacks cb 0x562eeb3bc150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a489f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4b7b60 0x562eeb4c1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a491f5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4c16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a489f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4b7b60 0x562eeb4c1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42658/0 (socket says 172.21.15.112:42658) 2024-03-16T09:17:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a489f4640 1 -- 172.21.15.112:0/1969694762 learned_addr learned my addr 172.21.15.112:0/1969694762 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a499f6640 1 --2- 172.21.15.112:0/1969694762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4bccd0 0x562eeb4c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a491f5640 1 -- 172.21.15.112:0/1969694762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4bccd0 msgr2=0x562eeb4c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a491f5640 1 --2- 172.21.15.112:0/1969694762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4bccd0 0x562eeb4c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a491f5640 1 -- 172.21.15.112:0/1969694762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4b7b60 msgr2=0x562eeb4c1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a491f5640 1 --2- 172.21.15.112:0/1969694762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4b7b60 0x562eeb4c1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a491f5640 1 -- 172.21.15.112:0/1969694762 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eeb1fa7d0 con 0x562eeb4b7170 2024-03-16T09:17:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.278+0000 7f9a489f4640 1 --2- 172.21.15.112:0/1969694762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4b7b60 0x562eeb4c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.279+0000 7f9a491f5640 1 --2- 172.21.15.112:0/1969694762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4c16a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x562eeb4b3650 tx=0x7f9a40005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f67b10acb91f27f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.279+0000 7f9a2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T09:17:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.279+0000 7f9a2cff9640 1 -- 172.21.15.112:0/1969694762 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4001e040 con 0x562eeb4b7170 2024-03-16T09:17:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.279+0000 7f9a4e040740 1 -- 172.21.15.112:0/1969694762 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a40009e10 con 0x562eeb4b7170 2024-03-16T09:17:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.279+0000 7f9a2cff9640 1 -- 172.21.15.112:0/1969694762 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f9a40018720 con 0x562eeb4b7170 2024-03-16T09:17:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.279+0000 7f9a2cff9640 1 -- 172.21.15.112:0/1969694762 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9a40018920 con 0x562eeb4b7170 2024-03-16T09:17:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.279+0000 7f9a4e040740 10 client.25633 fetch_fsmap learned FSMap version 23 2024-03-16T09:17:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.279+0000 7f9a4e040740 1 -- 172.21.15.112:0/1969694762 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562eeb1303f0 con 0x562eeb4b7170 2024-03-16T09:17:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.279+0000 7f9a2cff9640 1 -- 172.21.15.112:0/1969694762 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9a40009e10 con 0x562eeb4b7170 2024-03-16T09:17:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.279+0000 7f9a2cff9640 1 -- 172.21.15.112:0/1969694762 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9a40021070 con 0x562eeb4b7170 2024-03-16T09:17:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.279+0000 7f9a4e040740 10 client.25633 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T09:17:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.279+0000 7f9a4e040740 10 client.25633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.279+0000 7f9a4e040740 20 client.25633 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.279+0000 7f9a4e040740 1 --2- 172.21.15.112:0/1969694762 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x562eeb4ccef0 0x562eeb4cf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.279+0000 7f9a4e040740 4 client.25633 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:17:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.279+0000 7f9a4e040740 1 -- 172.21.15.112:0/1969694762 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eeb429a40 con 0x562eeb4ccef0 2024-03-16T09:17:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.280+0000 7f9a489f4640 1 --2- 172.21.15.112:0/1969694762 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x562eeb4ccef0 0x562eeb4cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.280+0000 7f9a2cff9640 10 client.25633 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.280+0000 7f9a489f4640 1 --2- 172.21.15.112:0/1969694762 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x562eeb4ccef0 0x562eeb4cf360 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7de0edec7cc6953a server_cookie=7f9968a5e2e6fd1d in_seq=0 out_seq=0 2024-03-16T09:17:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.281+0000 7f9a2cff9640 1 -- 172.21.15.112:0/1969694762 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562eeb429a40 con 0x562eeb4ccef0 2024-03-16T09:17:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.281+0000 7f9a2cff9640 10 client.25633 handle_command_reply: tid=0 2024-03-16T09:17:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.281+0000 7f9a4e040740 1 client.25633 shutdown 2024-03-16T09:17:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.281+0000 7f9a4e040740 1 -- 172.21.15.112:0/1969694762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 msgr2=0x562eeb4c16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.281+0000 7f9a4e040740 1 --2- 172.21.15.112:0/1969694762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4c16a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x562eeb4b3650 tx=0x7f9a40005db0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.281+0000 7f9a4e040740 1 -- 172.21.15.112:0/1969694762 shutdown_connections 2024-03-16T09:17:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.281+0000 7f9a4e040740 1 --2- 172.21.15.112:0/1969694762 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x562eeb4ccef0 0x562eeb4cf360 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.281+0000 7f9a4e040740 1 --2- 172.21.15.112:0/1969694762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4bccd0 0x562eeb4c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.281+0000 7f9a4e040740 1 --2- 172.21.15.112:0/1969694762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4b7b60 0x562eeb4c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.281+0000 7f9a4e040740 1 --2- 172.21.15.112:0/1969694762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4c16a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.281+0000 7f9a2cff9640 0 client.25633 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.281+0000 7f9a4e040740 1 -- 172.21.15.112:0/1969694762 >> 172.21.15.112:0/1969694762 conn(0x562eeb4b26e0 msgr2=0x562eeb4b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.281+0000 7f9a4e040740 1 -- 172.21.15.112:0/1969694762 shutdown_connections 2024-03-16T09:17:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.281+0000 7f9a4e040740 1 -- 172.21.15.112:0/1969694762 wait complete. 2024-03-16T09:17:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.281+0000 7f9a4e040740 20 client.25633 trim_cache size 0 max 16384 2024-03-16T09:17:17.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.298+0000 7f9a4e040740 1 Processor -- start 2024-03-16T09:17:17.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.298+0000 7f9a4e040740 1 -- start start 2024-03-16T09:17:17.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.298+0000 7f9a4e040740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.298+0000 7f9a4e040740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4b7b60 0x562eeb4c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.298+0000 7f9a4e040740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4bccd0 0x562eeb4b6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.298+0000 7f9a4e040740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441ab730 con 0x562eeb4bccd0 2024-03-16T09:17:17.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.298+0000 7f9a4e040740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441a9000 con 0x562eeb4b7b60 2024-03-16T09:17:17.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.298+0000 7f9a4e040740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441a6e50 con 0x562eeb4b7170 2024-03-16T09:17:17.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.298+0000 7f9a499f6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4bccd0 0x562eeb4b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.298+0000 7f9a499f6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4bccd0 0x562eeb4b6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42662/0 (socket says 172.21.15.112:42662) 2024-03-16T09:17:17.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.298+0000 7f9a489f4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4b7b60 0x562eeb4c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.298+0000 7f9a499f6640 1 -- 172.21.15.112:0/2051320937 learned_addr learned my addr 172.21.15.112:0/2051320937 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:17.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.298+0000 7f9a491f5640 1 --2- 172.21.15.112:0/2051320937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.299+0000 7f9a489f4640 1 -- 172.21.15.112:0/2051320937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 msgr2=0x562eeb4ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.299+0000 7f9a489f4640 1 --2- 172.21.15.112:0/2051320937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.299+0000 7f9a489f4640 1 -- 172.21.15.112:0/2051320937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4bccd0 msgr2=0x562eeb4b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.299+0000 7f9a489f4640 1 --2- 172.21.15.112:0/2051320937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4bccd0 0x562eeb4b6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.299+0000 7f9a489f4640 1 -- 172.21.15.112:0/2051320937 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eeb0f00f0 con 0x562eeb4b7b60 2024-03-16T09:17:17.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.299+0000 7f9a499f6640 1 --2- 172.21.15.112:0/2051320937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4bccd0 0x562eeb4b6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:17.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.299+0000 7f9a491f5640 1 --2- 172.21.15.112:0/2051320937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:17.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.299+0000 7f9a489f4640 1 --2- 172.21.15.112:0/2051320937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4b7b60 0x562eeb4c8be0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a340ccd70 tx=0x7f9a340ce040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35815a7628af424a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.299+0000 7f9a2effd640 1 -- 172.21.15.112:0/2051320937 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3409b8a0 con 0x562eeb4b7b60 2024-03-16T09:17:17.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.299+0000 7f9a2effd640 1 -- 172.21.15.112:0/2051320937 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9a3409f9f0 con 0x562eeb4b7b60 2024-03-16T09:17:17.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.299+0000 7f9a2effd640 1 -- 172.21.15.112:0/2051320937 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3409bd40 con 0x562eeb4b7b60 2024-03-16T09:17:17.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.300+0000 7f9a4e040740 1 -- 172.21.15.112:0/2051320937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4b7b60 msgr2=0x562eeb4c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.300+0000 7f9a4e040740 1 --2- 172.21.15.112:0/2051320937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4b7b60 0x562eeb4c8be0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a340ccd70 tx=0x7f9a340ce040 comp rx=0 tx=0).stop 2024-03-16T09:17:17.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.300+0000 7f9a4e040740 1 -- 172.21.15.112:0/2051320937 shutdown_connections 2024-03-16T09:17:17.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.300+0000 7f9a4e040740 1 --2- 172.21.15.112:0/2051320937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4bccd0 0x562eeb4b6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.300+0000 7f9a4e040740 1 --2- 172.21.15.112:0/2051320937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4b7b60 0x562eeb4c8be0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.300+0000 7f9a4e040740 1 --2- 172.21.15.112:0/2051320937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.300+0000 7f9a4e040740 1 -- 172.21.15.112:0/2051320937 >> 172.21.15.112:0/2051320937 conn(0x562eeb4b26e0 msgr2=0x562eeb4b5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.300+0000 7f9a4e040740 1 -- 172.21.15.112:0/2051320937 shutdown_connections 2024-03-16T09:17:17.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.300+0000 7f9a4e040740 1 -- 172.21.15.112:0/2051320937 wait complete. 2024-03-16T09:17:17.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.300+0000 7f9a4e040740 1 -- start start 2024-03-16T09:17:17.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.300+0000 7f9a4e040740 1 Processor -- start 2024-03-16T09:17:17.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.300+0000 7f9a4e040740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4c5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.300+0000 7f9a4e040740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4b7b60 0x562eeb4c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.300+0000 7f9a4e040740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4bccd0 0x562eeb531290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.300+0000 7f9a4e040740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441a74c0 con 0x562eeb4b7b60 2024-03-16T09:17:17.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.300+0000 7f9a4e040740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441a8930 con 0x562eeb4bccd0 2024-03-16T09:17:17.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.300+0000 7f9a4e040740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9a441a7b90 con 0x562eeb4b7170 2024-03-16T09:17:17.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a4e040740 10 client.0 _ll_register_callbacks cb 0x7f9a4001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:17.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a489f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4b7b60 0x562eeb4c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a499f6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4bccd0 0x562eeb531290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a489f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4b7b60 0x562eeb4c5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42678/0 (socket says 172.21.15.112:42678) 2024-03-16T09:17:17.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a489f4640 1 -- 172.21.15.112:0/2133891209 learned_addr learned my addr 172.21.15.112:0/2133891209 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:17.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a491f5640 1 --2- 172.21.15.112:0/2133891209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a489f4640 1 -- 172.21.15.112:0/2133891209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 msgr2=0x562eeb4c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a489f4640 1 --2- 172.21.15.112:0/2133891209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a489f4640 1 -- 172.21.15.112:0/2133891209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4bccd0 msgr2=0x562eeb531290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a489f4640 1 --2- 172.21.15.112:0/2133891209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4bccd0 0x562eeb531290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a489f4640 1 -- 172.21.15.112:0/2133891209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eeb3e39d0 con 0x562eeb4b7b60 2024-03-16T09:17:17.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a499f6640 1 --2- 172.21.15.112:0/2133891209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4bccd0 0x562eeb531290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:17.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a491f5640 1 --2- 172.21.15.112:0/2133891209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:17.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a489f4640 1 --2- 172.21.15.112:0/2133891209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4b7b60 0x562eeb4c5cb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9a34098220 tx=0x7f9a34060370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2992c3f61eec1c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:17.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a2cff9640 1 -- 172.21.15.112:0/2133891209 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3409b8a0 con 0x562eeb4b7b60 2024-03-16T09:17:17.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a4e040740 1 -- 172.21.15.112:0/2133891209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a3409f9f0 con 0x562eeb4b7b60 2024-03-16T09:17:17.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a2cff9640 1 -- 172.21.15.112:0/2133891209 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f9a3409cbd0 con 0x562eeb4b7b60 2024-03-16T09:17:17.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a2cff9640 1 -- 172.21.15.112:0/2133891209 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9a3409cdd0 con 0x562eeb4b7b60 2024-03-16T09:17:17.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a4e040740 10 client.15777 fetch_fsmap learned FSMap version 23 2024-03-16T09:17:17.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.301+0000 7f9a4e040740 1 -- 172.21.15.112:0/2133891209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562eeb532570 con 0x562eeb4b7b60 2024-03-16T09:17:17.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.302+0000 7f9a2cff9640 1 -- 172.21.15.112:0/2133891209 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9a3409f9f0 con 0x562eeb4b7b60 2024-03-16T09:17:17.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.302+0000 7f9a2cff9640 1 -- 172.21.15.112:0/2133891209 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9a3409a760 con 0x562eeb4b7b60 2024-03-16T09:17:17.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.302+0000 7f9a4e040740 10 client.15777 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T09:17:17.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.302+0000 7f9a4e040740 10 client.15777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:17.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.302+0000 7f9a4e040740 20 client.15777 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:17.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.302+0000 7f9a4e040740 1 --2- 172.21.15.112:0/2133891209 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x562eeb532ef0 0x562eeb535310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.302+0000 7f9a4e040740 4 client.15777 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T09:17:17.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.302+0000 7f9a4e040740 1 -- 172.21.15.112:0/2133891209 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562eeb423d90 con 0x562eeb532ef0 2024-03-16T09:17:17.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.302+0000 7f9a491f5640 1 --2- 172.21.15.112:0/2133891209 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x562eeb532ef0 0x562eeb535310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.302+0000 7f9a2cff9640 10 client.15777 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:17.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.302+0000 7f9a491f5640 1 --2- 172.21.15.112:0/2133891209 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x562eeb532ef0 0x562eeb535310 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2249c0dfa73ae17 server_cookie=3f3285391eca857 in_seq=0 out_seq=0 2024-03-16T09:17:17.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.302+0000 7f9a2cff9640 1 -- 172.21.15.112:0/2133891209 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562eeb423d90 con 0x562eeb532ef0 2024-03-16T09:17:17.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.302+0000 7f9a2cff9640 10 client.15777 handle_command_reply: tid=0 2024-03-16T09:17:17.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.302+0000 7f9a4e040740 1 client.15777 shutdown 2024-03-16T09:17:17.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.302+0000 7f9a4e040740 1 -- 172.21.15.112:0/2133891209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4b7b60 msgr2=0x562eeb4c5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.302+0000 7f9a4e040740 1 --2- 172.21.15.112:0/2133891209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4b7b60 0x562eeb4c5cb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9a34098220 tx=0x7f9a34060370 comp rx=0 tx=0).stop 2024-03-16T09:17:17.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4e040740 1 -- 172.21.15.112:0/2133891209 shutdown_connections 2024-03-16T09:17:17.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4e040740 1 --2- 172.21.15.112:0/2133891209 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x562eeb532ef0 0x562eeb535310 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4e040740 1 --2- 172.21.15.112:0/2133891209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eeb4bccd0 0x562eeb531290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a2cff9640 0 client.15777 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:17.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4e040740 1 --2- 172.21.15.112:0/2133891209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562eeb4b7b60 0x562eeb4c5cb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4e040740 1 --2- 172.21.15.112:0/2133891209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562eeb4b7170 0x562eeb4c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4e040740 1 -- 172.21.15.112:0/2133891209 >> 172.21.15.112:0/2133891209 conn(0x562eeb4b26e0 msgr2=0x562eeb4b61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4e040740 1 -- 172.21.15.112:0/2133891209 shutdown_connections 2024-03-16T09:17:17.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4e040740 1 -- 172.21.15.112:0/2133891209 wait complete. 2024-03-16T09:17:17.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4e040740 20 client.15777 trim_cache size 0 max 16384 2024-03-16T09:17:17.313 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:17:17.313 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-03-16T09:17:17.313 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-03-16T09:17:17.313 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:17:17.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4b459640 1 -- 172.21.15.112:0/471163969 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9a300a2800 msgr2=0x7f9a300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4b459640 1 --2- 172.21.15.112:0/471163969 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9a300a2800 0x7f9a300a4cc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c005e30 tx=0x7f9a3c032e10 comp rx=0 tx=0).stop 2024-03-16T09:17:17.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4b459640 1 -- 172.21.15.112:0/471163969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a44104c30 msgr2=0x7f9a44198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4b459640 1 --2- 172.21.15.112:0/471163969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a44104c30 0x7f9a44198130 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400e9c0 tx=0x7f9a3400ee80 comp rx=0 tx=0).stop 2024-03-16T09:17:17.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4b459640 1 -- 172.21.15.112:0/471163969 shutdown_connections 2024-03-16T09:17:17.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4b459640 1 --2- 172.21.15.112:0/471163969 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9a300a2800 0x7f9a300a4cc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4b459640 1 --2- 172.21.15.112:0/471163969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a44107560 0x7f9a4419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4b459640 1 --2- 172.21.15.112:0/471163969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a44104c30 0x7f9a44198130 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4b459640 1 --2- 172.21.15.112:0/471163969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a44102300 0x7f9a44197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4b459640 1 -- 172.21.15.112:0/471163969 >> 172.21.15.112:0/471163969 conn(0x7f9a440fbc70 msgr2=0x7f9a44102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4b459640 1 -- 172.21.15.112:0/471163969 shutdown_connections 2024-03-16T09:17:17.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.303+0000 7f9a4b459640 1 -- 172.21.15.112:0/471163969 wait complete. 2024-03-16T09:17:17.317 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T09:17:17.317 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:17:17.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.416+0000 7efcaf916640 1 -- 172.21.15.112:0/812060690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efca81022b0 msgr2=0x7efca8108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.416+0000 7efcaf916640 1 --2- 172.21.15.112:0/812060690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efca81022b0 0x7efca8108b40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7efca0009ca0 tx=0x7efca002c0f0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.416+0000 7efcaf916640 1 -- 172.21.15.112:0/812060690 shutdown_connections 2024-03-16T09:17:17.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.417+0000 7efcaf916640 1 --2- 172.21.15.112:0/812060690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efca81022b0 0x7efca8108b40 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.417+0000 7efcaf916640 1 --2- 172.21.15.112:0/812060690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efca81018f0 0x7efca8101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.417+0000 7efcaf916640 1 --2- 172.21.15.112:0/812060690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efca8100700 0x7efca8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.417+0000 7efcaf916640 1 -- 172.21.15.112:0/812060690 >> 172.21.15.112:0/812060690 conn(0x7efca80fbc70 msgr2=0x7efca80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.417+0000 7efcaf916640 1 -- 172.21.15.112:0/812060690 shutdown_connections 2024-03-16T09:17:17.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.417+0000 7efcaf916640 1 -- 172.21.15.112:0/812060690 wait complete. 2024-03-16T09:17:17.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.417+0000 7efcaf916640 1 Processor -- start 2024-03-16T09:17:17.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.417+0000 7efcaf916640 1 -- start start 2024-03-16T09:17:17.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.417+0000 7efcaf916640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efca8100700 0x7efca80694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.417+0000 7efcaf916640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efca81018f0 0x7efca819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.417+0000 7efcaf916640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efca81022b0 0x7efca819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.417+0000 7efcaf916640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efca810c4a0 con 0x7efca81018f0 2024-03-16T09:17:17.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.417+0000 7efcaf916640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efca810c320 con 0x7efca8100700 2024-03-16T09:17:17.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.417+0000 7efcaf916640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efca810c620 con 0x7efca81022b0 2024-03-16T09:17:17.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.417+0000 7efcaceb1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efca81018f0 0x7efca819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.417+0000 7efcaceb1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efca81018f0 0x7efca819c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42700/0 (socket says 172.21.15.112:42700) 2024-03-16T09:17:17.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.417+0000 7efcaceb1640 1 -- 172.21.15.112:0/3765221262 learned_addr learned my addr 172.21.15.112:0/3765221262 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:17.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.417+0000 7efcad6b2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efca8100700 0x7efca80694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.418+0000 7efcadeb3640 1 --2- 172.21.15.112:0/3765221262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efca81022b0 0x7efca819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.418+0000 7efcaceb1640 1 -- 172.21.15.112:0/3765221262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efca81022b0 msgr2=0x7efca819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.418+0000 7efcaceb1640 1 --2- 172.21.15.112:0/3765221262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efca81022b0 0x7efca819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.418+0000 7efcaceb1640 1 -- 172.21.15.112:0/3765221262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efca8100700 msgr2=0x7efca80694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.418+0000 7efcaceb1640 1 --2- 172.21.15.112:0/3765221262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efca8100700 0x7efca80694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.418+0000 7efcaceb1640 1 -- 172.21.15.112:0/3765221262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca819f890 con 0x7efca81018f0 2024-03-16T09:17:17.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.418+0000 7efcad6b2640 1 --2- 172.21.15.112:0/3765221262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efca8100700 0x7efca80694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:17.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.418+0000 7efcaceb1640 1 --2- 172.21.15.112:0/3765221262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efca81018f0 0x7efca819c960 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efc9800e9f0 tx=0x7efc9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.418+0000 7efc9e7fc640 1 -- 172.21.15.112:0/3765221262 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9800cd80 con 0x7efca81018f0 2024-03-16T09:17:17.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.418+0000 7efcaf916640 1 -- 172.21.15.112:0/3765221262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca819fb20 con 0x7efca81018f0 2024-03-16T09:17:17.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.418+0000 7efc9e7fc640 1 -- 172.21.15.112:0/3765221262 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efc98010930 con 0x7efca81018f0 2024-03-16T09:17:17.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.418+0000 7efc9e7fc640 1 -- 172.21.15.112:0/3765221262 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98018b00 con 0x7efca81018f0 2024-03-16T09:17:17.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.418+0000 7efcaf916640 1 -- 172.21.15.112:0/3765221262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca81a0070 con 0x7efca81018f0 2024-03-16T09:17:17.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.419+0000 7efcaf916640 1 -- 172.21.15.112:0/3765221262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca8107950 con 0x7efca81018f0 2024-03-16T09:17:17.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.420+0000 7efc9e7fc640 1 -- 172.21.15.112:0/3765221262 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7efc98018ca0 con 0x7efca81018f0 2024-03-16T09:17:17.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.421+0000 7efc9e7fc640 1 --2- 172.21.15.112:0/3765221262 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efc940a2800 0x7efc940a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.421+0000 7efcad6b2640 1 --2- 172.21.15.112:0/3765221262 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efc940a2800 0x7efc940a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.421+0000 7efc9e7fc640 1 -- 172.21.15.112:0/3765221262 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7efc980ced50 con 0x7efca81018f0 2024-03-16T09:17:17.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.421+0000 7efcad6b2640 1 --2- 172.21.15.112:0/3765221262 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efc940a2800 0x7efc940a4cc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efca8101750 tx=0x7efca4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.426+0000 7efc9e7fc640 1 -- 172.21.15.112:0/3765221262 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efc980d2030 con 0x7efca81018f0 2024-03-16T09:17:17.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.661+0000 7efcaf916640 1 -- 172.21.15.112:0/3765221262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efca80623e0 con 0x7efca81018f0 2024-03-16T09:17:17.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.662+0000 7efc9e7fc640 1 -- 172.21.15.112:0/3765221262 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11884 (secure 0 0 0) 0x7efc9809b7b0 con 0x7efca81018f0 2024-03-16T09:17:17.665 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:17:17.665 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3939748854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3939748854},{"type":"v1","addr":"172.21.15.112:6839","nonce":3939748854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3118523258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3118523258},{"type":"v1","addr":"172.21.15.112:6841","nonce":3118523258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T09:13:36.450216+0000","modified":"2024-03-16T09:17:11.447353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/1682076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":1682076},{"type":"v1","addr":"172.21.15.194:6837","nonce":1682076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.194:6839/2528943309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2528943309},{"type":"v1","addr":"172.21.15.194:6839","nonce":2528943309}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/272895965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":272895965},{"type":"v1","addr":"172.21.15.159:6839","nonce":272895965}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24452]},"id":1}]} 2024-03-16T09:17:17.665 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 23 2024-03-16T09:17:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.664+0000 7efcaf916640 1 -- 172.21.15.112:0/3765221262 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efc940a2800 msgr2=0x7efc940a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.665+0000 7efcaf916640 1 --2- 172.21.15.112:0/3765221262 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efc940a2800 0x7efc940a4cc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efca8101750 tx=0x7efca4005f90 comp rx=0 tx=0).stop 2024-03-16T09:17:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.665+0000 7efcaf916640 1 -- 172.21.15.112:0/3765221262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efca81018f0 msgr2=0x7efca819c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.665+0000 7efcaf916640 1 --2- 172.21.15.112:0/3765221262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efca81018f0 0x7efca819c960 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efc9800e9f0 tx=0x7efc9800eeb0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.665+0000 7efcaf916640 1 -- 172.21.15.112:0/3765221262 shutdown_connections 2024-03-16T09:17:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.665+0000 7efcaf916640 1 --2- 172.21.15.112:0/3765221262 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efc940a2800 0x7efc940a4cc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.665+0000 7efcaf916640 1 --2- 172.21.15.112:0/3765221262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efca81022b0 0x7efca819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.665+0000 7efcaf916640 1 --2- 172.21.15.112:0/3765221262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efca81018f0 0x7efca819c960 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.665+0000 7efcaf916640 1 --2- 172.21.15.112:0/3765221262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efca8100700 0x7efca80694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.665+0000 7efcaf916640 1 -- 172.21.15.112:0/3765221262 >> 172.21.15.112:0/3765221262 conn(0x7efca80fbc70 msgr2=0x7efca8104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.665+0000 7efcaf916640 1 -- 172.21.15.112:0/3765221262 shutdown_connections 2024-03-16T09:17:17.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.665+0000 7efcaf916640 1 -- 172.21.15.112:0/3765221262 wait complete. 2024-03-16T09:17:17.682 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T09:17:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.785+0000 7fdcf3af8640 1 -- 172.21.15.112:0/3880486591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcec1018d0 msgr2=0x7fdcec101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.785+0000 7fdcf3af8640 1 --2- 172.21.15.112:0/3880486591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcec1018d0 0x7fdcec101d50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009900 tx=0x7fdcdc02bc30 comp rx=0 tx=0).stop 2024-03-16T09:17:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.785+0000 7fdcf3af8640 1 -- 172.21.15.112:0/3880486591 shutdown_connections 2024-03-16T09:17:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.785+0000 7fdcf3af8640 1 --2- 172.21.15.112:0/3880486591 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcec102290 0x7fdcec108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.785+0000 7fdcf3af8640 1 --2- 172.21.15.112:0/3880486591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcec1018d0 0x7fdcec101d50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.785+0000 7fdcf3af8640 1 --2- 172.21.15.112:0/3880486591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcec1006e0 0x7fdcec100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.785+0000 7fdcf3af8640 1 -- 172.21.15.112:0/3880486591 >> 172.21.15.112:0/3880486591 conn(0x7fdcec0fbc70 msgr2=0x7fdcec0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.785+0000 7fdcf3af8640 1 -- 172.21.15.112:0/3880486591 shutdown_connections 2024-03-16T09:17:17.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.785+0000 7fdcf3af8640 1 -- 172.21.15.112:0/3880486591 wait complete. 2024-03-16T09:17:17.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.785+0000 7fdcf3af8640 1 Processor -- start 2024-03-16T09:17:17.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.785+0000 7fdcf3af8640 1 -- start start 2024-03-16T09:17:17.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.785+0000 7fdcf3af8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcec1006e0 0x7fdcec0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.785+0000 7fdcf3af8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcec1018d0 0x7fdcec19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdcf3af8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcec102290 0x7fdcec19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdcf3af8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec10c440 con 0x7fdcec1018d0 2024-03-16T09:17:17.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdcf3af8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec10c2c0 con 0x7fdcec102290 2024-03-16T09:17:17.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdcf3af8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec10c5c0 con 0x7fdcec1006e0 2024-03-16T09:17:17.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdcf1093640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcec1018d0 0x7fdcec19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdcf1894640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcec1006e0 0x7fdcec0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdcf1093640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcec1018d0 0x7fdcec19c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42706/0 (socket says 172.21.15.112:42706) 2024-03-16T09:17:17.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdcf1093640 1 -- 172.21.15.112:0/1763553412 learned_addr learned my addr 172.21.15.112:0/1763553412 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:17.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdcf2095640 1 --2- 172.21.15.112:0/1763553412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcec102290 0x7fdcec19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdcf1894640 1 -- 172.21.15.112:0/1763553412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcec102290 msgr2=0x7fdcec19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdcf1894640 1 --2- 172.21.15.112:0/1763553412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcec102290 0x7fdcec19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdcf1894640 1 -- 172.21.15.112:0/1763553412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcec1018d0 msgr2=0x7fdcec19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdcf1894640 1 --2- 172.21.15.112:0/1763553412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcec1018d0 0x7fdcec19c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdcf1894640 1 -- 172.21.15.112:0/1763553412 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec19f830 con 0x7fdcec1006e0 2024-03-16T09:17:17.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdcf1093640 1 --2- 172.21.15.112:0/1763553412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcec1018d0 0x7fdcec19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:17.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdcf1894640 1 --2- 172.21.15.112:0/1763553412 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcec1006e0 0x7fdcec0694b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdce800e9c0 tx=0x7fdce800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdce2ffd640 1 -- 172.21.15.112:0/1763553412 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce800cd80 con 0x7fdcec1006e0 2024-03-16T09:17:17.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdcf3af8640 1 -- 172.21.15.112:0/1763553412 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcec19fac0 con 0x7fdcec1006e0 2024-03-16T09:17:17.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdce2ffd640 1 -- 172.21.15.112:0/1763553412 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdce8010930 con 0x7fdcec1006e0 2024-03-16T09:17:17.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdce2ffd640 1 -- 172.21.15.112:0/1763553412 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8018980 con 0x7fdcec1006e0 2024-03-16T09:17:17.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.786+0000 7fdcf3af8640 1 -- 172.21.15.112:0/1763553412 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcec19ffe0 con 0x7fdcec1006e0 2024-03-16T09:17:17.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.789+0000 7fdce2ffd640 1 -- 172.21.15.112:0/1763553412 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7fdce801f020 con 0x7fdcec1006e0 2024-03-16T09:17:17.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.789+0000 7fdcf67c1740 1 Processor -- start 2024-03-16T09:17:17.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdcf67c1740 1 -- start start 2024-03-16T09:17:17.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdcf67c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d449c850 0x55a9d449cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdcf67c1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d449d240 0x55a9d44a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdcf67c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d44a23b0 0x55a9d44a4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdcf67c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a9d4401490 con 0x55a9d44a23b0 2024-03-16T09:17:17.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdcf67c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a9d43a9c90 con 0x55a9d449c850 2024-03-16T09:17:17.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdcf67c1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a9d43fd530 con 0x55a9d449d240 2024-03-16T09:17:17.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdce2ffd640 1 --2- 172.21.15.112:0/1763553412 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdcd80a2800 0x7fdcd80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdcf1093640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d44a23b0 0x55a9d44a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdcf1894640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d449d240 0x55a9d44a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdcf1093640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d44a23b0 0x55a9d44a4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42708/0 (socket says 172.21.15.112:42708) 2024-03-16T09:17:17.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdcf2095640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d449c850 0x55a9d449cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdcf1093640 1 -- 172.21.15.112:0/330704570 learned_addr learned my addr 172.21.15.112:0/330704570 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:17.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdcf2095640 1 --2- 172.21.15.112:0/1763553412 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdcd80a2800 0x7fdcd80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdce2ffd640 1 -- 172.21.15.112:0/1763553412 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdce80ce9f0 con 0x7fdcec1006e0 2024-03-16T09:17:17.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdcf2095640 1 -- 172.21.15.112:0/330704570 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d449d240 msgr2=0x55a9d44a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdcf2095640 1 --2- 172.21.15.112:0/330704570 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d449d240 0x55a9d44a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdcf2095640 1 -- 172.21.15.112:0/330704570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d44a23b0 msgr2=0x55a9d44a4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdcf2095640 1 --2- 172.21.15.112:0/330704570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d44a23b0 0x55a9d44a4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdcf2095640 1 -- 172.21.15.112:0/330704570 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9d4464500 con 0x55a9d449c850 2024-03-16T09:17:17.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.790+0000 7fdcf1093640 1 --2- 172.21.15.112:0/330704570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d44a23b0 0x55a9d44a4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:17.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.791+0000 7fdcf2095640 1 --2- 172.21.15.112:0/330704570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d449c850 0x55a9d449cc70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdce4005fd0 tx=0x7fdce400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fb1300654cb978a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.791+0000 7fdcf2095640 1 --2- 172.21.15.112:0/1763553412 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdcd80a2800 0x7fdcd80a4cc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdce400fc90 tx=0x7fdce401e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.791+0000 7fdcd7fff640 1 -- 172.21.15.112:0/330704570 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4011e30 con 0x55a9d449c850 2024-03-16T09:17:17.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.791+0000 7fdcd7fff640 1 -- 172.21.15.112:0/330704570 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdce401dbf0 con 0x55a9d449c850 2024-03-16T09:17:17.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.791+0000 7fdcd7fff640 1 -- 172.21.15.112:0/330704570 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce40042b0 con 0x55a9d449c850 2024-03-16T09:17:17.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.792+0000 7fdcf67c1740 1 -- 172.21.15.112:0/330704570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d449c850 msgr2=0x55a9d449cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.792+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/330704570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d449c850 0x55a9d449cc70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdce4005fd0 tx=0x7fdce400f790 comp rx=0 tx=0).stop 2024-03-16T09:17:17.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.792+0000 7fdcf67c1740 1 -- 172.21.15.112:0/330704570 shutdown_connections 2024-03-16T09:17:17.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.792+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/330704570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d44a23b0 0x55a9d44a4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.792+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/330704570 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d449d240 0x55a9d44a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.792+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/330704570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d449c850 0x55a9d449cc70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.792+0000 7fdcf67c1740 1 -- 172.21.15.112:0/330704570 >> 172.21.15.112:0/330704570 conn(0x55a9d4497dc0 msgr2=0x55a9d44981c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.792+0000 7fdcf67c1740 1 -- 172.21.15.112:0/330704570 shutdown_connections 2024-03-16T09:17:17.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.792+0000 7fdcf67c1740 1 -- 172.21.15.112:0/330704570 wait complete. 2024-03-16T09:17:17.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.792+0000 7fdcf67c1740 1 -- start start 2024-03-16T09:17:17.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.792+0000 7fdcf67c1740 1 Processor -- start 2024-03-16T09:17:17.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.792+0000 7fdcf67c1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d449c850 0x55a9d44a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcf67c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d449d240 0x55a9d44a71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcf67c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d44a23b0 0x55a9d44ae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcf67c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a9d44a5300 con 0x55a9d449d240 2024-03-16T09:17:17.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcf67c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a9d43bd230 con 0x55a9d44a23b0 2024-03-16T09:17:17.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcf67c1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a9d44a5480 con 0x55a9d449c850 2024-03-16T09:17:17.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcf67c1740 10 client.0 _ll_register_callbacks cb 0x55a9d43f9010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:17.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcf1093640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d449d240 0x55a9d44a71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcf1093640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d449d240 0x55a9d44a71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42712/0 (socket says 172.21.15.112:42712) 2024-03-16T09:17:17.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcf1093640 1 -- 172.21.15.112:0/3554105357 learned_addr learned my addr 172.21.15.112:0/3554105357 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:17.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcf1894640 1 --2- 172.21.15.112:0/3554105357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d449c850 0x55a9d44a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcf2095640 1 --2- 172.21.15.112:0/3554105357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d44a23b0 0x55a9d44ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcf1093640 1 -- 172.21.15.112:0/3554105357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d449c850 msgr2=0x55a9d44a6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcf1093640 1 --2- 172.21.15.112:0/3554105357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d449c850 0x55a9d44a6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcf1093640 1 -- 172.21.15.112:0/3554105357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d44a23b0 msgr2=0x55a9d44ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcf1093640 1 --2- 172.21.15.112:0/3554105357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d44a23b0 0x55a9d44ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcf1093640 1 -- 172.21.15.112:0/3554105357 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9d43cc100 con 0x55a9d449d240 2024-03-16T09:17:17.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcf1093640 1 --2- 172.21.15.112:0/3554105357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d449d240 0x55a9d44a71f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc02c7e0 tx=0x7fdcdc02c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f93dbb44a3d76d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:17.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcd57fa640 1 -- 172.21.15.112:0/3554105357 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc045070 con 0x55a9d449d240 2024-03-16T09:17:17.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcd57fa640 1 -- 172.21.15.112:0/3554105357 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdcdc005310 con 0x55a9d449d240 2024-03-16T09:17:17.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcf67c1740 1 -- 172.21.15.112:0/3554105357 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdce401dbf0 con 0x55a9d449d240 2024-03-16T09:17:17.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.793+0000 7fdcd57fa640 1 -- 172.21.15.112:0/3554105357 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fdcdc051050 con 0x55a9d449d240 2024-03-16T09:17:17.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.794+0000 7fdcf67c1740 10 client.15795 fetch_fsmap learned FSMap version 23 2024-03-16T09:17:17.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.794+0000 7fdcf67c1740 1 -- 172.21.15.112:0/3554105357 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a9d41153f0 con 0x55a9d449d240 2024-03-16T09:17:17.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.794+0000 7fdcd57fa640 1 -- 172.21.15.112:0/3554105357 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdce401dbf0 con 0x55a9d449d240 2024-03-16T09:17:17.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.794+0000 7fdcd57fa640 1 -- 172.21.15.112:0/3554105357 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fdcdc003850 con 0x55a9d449d240 2024-03-16T09:17:17.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.794+0000 7fdcf67c1740 10 client.15795 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T09:17:17.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.794+0000 7fdcf67c1740 10 client.15795 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T09:17:17.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.794+0000 7fdcf67c1740 20 client.15795 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:17.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.794+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/3554105357 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x55a9d44b2af0 0x55a9d44b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.794+0000 7fdcf67c1740 4 client.15795 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:17:17.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.794+0000 7fdcf67c1740 1 -- 172.21.15.112:0/3554105357 --> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9d440e9d0 con 0x55a9d44b2af0 2024-03-16T09:17:17.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.794+0000 7fdcf1894640 1 --2- 172.21.15.112:0/3554105357 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x55a9d44b2af0 0x55a9d44b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.795+0000 7fdcd57fa640 10 client.15795 ms_handle_connect on v2:172.21.15.159:6836/4023942203 2024-03-16T09:17:17.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.795+0000 7fdcf1894640 1 --2- 172.21.15.112:0/3554105357 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x55a9d44b2af0 0x55a9d44b4f30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ecbc36053e0ff169 server_cookie=df2d293c9f2eabe in_seq=0 out_seq=0 2024-03-16T09:17:17.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.795+0000 7fdcd57fa640 1 -- 172.21.15.112:0/3554105357 <== mds.2 v2:172.21.15.159:6836/4023942203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a9d440e9d0 con 0x55a9d44b2af0 2024-03-16T09:17:17.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.795+0000 7fdcd57fa640 10 client.15795 handle_command_reply: tid=0 2024-03-16T09:17:17.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.795+0000 7fdcf67c1740 1 client.15795 shutdown 2024-03-16T09:17:17.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.795+0000 7fdcf67c1740 1 -- 172.21.15.112:0/3554105357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d449d240 msgr2=0x55a9d44a71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.795+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/3554105357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d449d240 0x55a9d44a71f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc02c7e0 tx=0x7fdcdc02c810 comp rx=0 tx=0).stop 2024-03-16T09:17:17.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.795+0000 7fdcf67c1740 1 -- 172.21.15.112:0/3554105357 shutdown_connections 2024-03-16T09:17:17.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.795+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/3554105357 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x55a9d44b2af0 0x55a9d44b4f30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.795+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/3554105357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d44a23b0 0x55a9d44ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.795+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/3554105357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d449d240 0x55a9d44a71f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.795+0000 7fdcd57fa640 0 client.15795 ms_handle_reset on v2:172.21.15.159:6836/4023942203 2024-03-16T09:17:17.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.795+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/3554105357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d449c850 0x55a9d44a6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.795+0000 7fdcf67c1740 1 -- 172.21.15.112:0/3554105357 >> 172.21.15.112:0/3554105357 conn(0x55a9d4497dc0 msgr2=0x55a9d4496fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.795+0000 7fdcf67c1740 1 -- 172.21.15.112:0/3554105357 shutdown_connections 2024-03-16T09:17:17.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.796+0000 7fdcf67c1740 1 -- 172.21.15.112:0/3554105357 wait complete. 2024-03-16T09:17:17.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.796+0000 7fdcf67c1740 20 client.15795 trim_cache size 0 max 16384 2024-03-16T09:17:17.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf67c1740 1 Processor -- start 2024-03-16T09:17:17.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf67c1740 1 -- start start 2024-03-16T09:17:17.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf67c1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d449c850 0x55a9d4496750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf67c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d449d240 0x55a9d44ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf67c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d44a23b0 0x55a9d4496c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf67c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec1ab590 con 0x55a9d44a23b0 2024-03-16T09:17:17.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf67c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec1a8e60 con 0x55a9d449d240 2024-03-16T09:17:17.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf67c1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec1a6cb0 con 0x55a9d449c850 2024-03-16T09:17:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf1093640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d449d240 0x55a9d44ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf1894640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d449c850 0x55a9d4496750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf2095640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d44a23b0 0x55a9d4496c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf1894640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d449c850 0x55a9d4496750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:34446/0 (socket says 172.21.15.112:34446) 2024-03-16T09:17:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf1093640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d449d240 0x55a9d44ae1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:57210/0 (socket says 172.21.15.112:57210) 2024-03-16T09:17:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf1894640 1 -- 172.21.15.112:0/2414335609 learned_addr learned my addr 172.21.15.112:0/2414335609 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf1894640 1 -- 172.21.15.112:0/2414335609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d449d240 msgr2=0x55a9d44ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf1894640 1 --2- 172.21.15.112:0/2414335609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d449d240 0x55a9d44ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf1894640 1 -- 172.21.15.112:0/2414335609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d44a23b0 msgr2=0x55a9d4496c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf1894640 1 --2- 172.21.15.112:0/2414335609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d44a23b0 0x55a9d4496c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf1894640 1 -- 172.21.15.112:0/2414335609 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9d4493640 con 0x55a9d449c850 2024-03-16T09:17:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf1093640 1 --2- 172.21.15.112:0/2414335609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d449d240 0x55a9d44ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:17.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.809+0000 7fdcf1894640 1 --2- 172.21.15.112:0/2414335609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d449c850 0x55a9d4496750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009fb0 tx=0x7fdce80bb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5435394213b0b00d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.810+0000 7fdcd77fe640 1 -- 172.21.15.112:0/2414335609 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8099440 con 0x55a9d449c850 2024-03-16T09:17:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.810+0000 7fdcd77fe640 1 -- 172.21.15.112:0/2414335609 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdce80996a0 con 0x55a9d449c850 2024-03-16T09:17:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.810+0000 7fdcd77fe640 1 -- 172.21.15.112:0/2414335609 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce80a36b0 con 0x55a9d449c850 2024-03-16T09:17:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.810+0000 7fdcf67c1740 1 -- 172.21.15.112:0/2414335609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d449c850 msgr2=0x55a9d4496750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.810+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/2414335609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d449c850 0x55a9d4496750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdce8009fb0 tx=0x7fdce80bb040 comp rx=0 tx=0).stop 2024-03-16T09:17:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.810+0000 7fdcf67c1740 1 -- 172.21.15.112:0/2414335609 shutdown_connections 2024-03-16T09:17:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.810+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/2414335609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d44a23b0 0x55a9d4496c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.810+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/2414335609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d449d240 0x55a9d44ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.810+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/2414335609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d449c850 0x55a9d4496750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.810+0000 7fdcf67c1740 1 -- 172.21.15.112:0/2414335609 >> 172.21.15.112:0/2414335609 conn(0x55a9d4497dc0 msgr2=0x55a9d44a6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.810+0000 7fdcf67c1740 1 -- 172.21.15.112:0/2414335609 shutdown_connections 2024-03-16T09:17:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.810+0000 7fdcf67c1740 1 -- 172.21.15.112:0/2414335609 wait complete. 2024-03-16T09:17:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf67c1740 1 -- start start 2024-03-16T09:17:17.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf67c1740 1 Processor -- start 2024-03-16T09:17:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf67c1740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d449c850 0x55a9d44abab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf67c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d449d240 0x55a9d44abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf67c1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d44a23b0 0x55a9d44acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf67c1740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec1a7320 con 0x55a9d449c850 2024-03-16T09:17:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf67c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec1a8790 con 0x55a9d449d240 2024-03-16T09:17:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf67c1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec1a79f0 con 0x55a9d44a23b0 2024-03-16T09:17:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf67c1740 10 client.0 _ll_register_callbacks cb 0x7fdcdc004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf1894640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d449c850 0x55a9d44abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf1894640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d449c850 0x55a9d44abab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42724/0 (socket says 172.21.15.112:42724) 2024-03-16T09:17:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf1894640 1 -- 172.21.15.112:0/936623950 learned_addr learned my addr 172.21.15.112:0/936623950 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf1093640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d449d240 0x55a9d44abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf2095640 1 --2- 172.21.15.112:0/936623950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d44a23b0 0x55a9d44acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf1894640 1 -- 172.21.15.112:0/936623950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d44a23b0 msgr2=0x55a9d44acfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf1894640 1 --2- 172.21.15.112:0/936623950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d44a23b0 0x55a9d44acfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf1894640 1 -- 172.21.15.112:0/936623950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d449d240 msgr2=0x55a9d44abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf1894640 1 --2- 172.21.15.112:0/936623950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d449d240 0x55a9d44abff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf1894640 1 -- 172.21.15.112:0/936623950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9d41df7d0 con 0x55a9d449c850 2024-03-16T09:17:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.811+0000 7fdcf1093640 1 --2- 172.21.15.112:0/936623950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d449d240 0x55a9d44abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.812+0000 7fdcf1894640 1 --2- 172.21.15.112:0/936623950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d449c850 0x55a9d44abab0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fdce809b0a0 tx=0x7fdce80a3c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d0a5ba1d1fd7e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.812+0000 7fdcd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.812+0000 7fdcd57fa640 1 -- 172.21.15.112:0/936623950 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce809b670 con 0x55a9d449c850 2024-03-16T09:17:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.812+0000 7fdcd57fa640 1 -- 172.21.15.112:0/936623950 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fdce809b810 con 0x55a9d449c850 2024-03-16T09:17:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.812+0000 7fdcf67c1740 1 -- 172.21.15.112:0/936623950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdce80996a0 con 0x55a9d449c850 2024-03-16T09:17:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.812+0000 7fdcd57fa640 1 -- 172.21.15.112:0/936623950 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fdce80b4180 con 0x55a9d449c850 2024-03-16T09:17:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.812+0000 7fdcf67c1740 10 client.15801 fetch_fsmap learned FSMap version 23 2024-03-16T09:17:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.812+0000 7fdcf67c1740 1 -- 172.21.15.112:0/936623950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a9d4518e30 con 0x55a9d449c850 2024-03-16T09:17:17.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.812+0000 7fdcd57fa640 1 -- 172.21.15.112:0/936623950 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdce80996a0 con 0x55a9d449c850 2024-03-16T09:17:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.812+0000 7fdcd57fa640 1 -- 172.21.15.112:0/936623950 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fdce80bb140 con 0x55a9d449c850 2024-03-16T09:17:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.812+0000 7fdcf67c1740 10 client.15801 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T09:17:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.812+0000 7fdcf67c1740 10 client.15801 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T09:17:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.812+0000 7fdcf67c1740 20 client.15801 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.812+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/936623950 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x55a9d4519500 0x55a9d451b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.812+0000 7fdcf67c1740 4 client.15801 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T09:17:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.812+0000 7fdcf67c1740 1 -- 172.21.15.112:0/936623950 --> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a9d4408d20 con 0x55a9d4519500 2024-03-16T09:17:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.813+0000 7fdcf1093640 1 --2- 172.21.15.112:0/936623950 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x55a9d4519500 0x55a9d451b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.813+0000 7fdcf1093640 1 --2- 172.21.15.112:0/936623950 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x55a9d4519500 0x55a9d451b920 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a8041f0cad2c618 server_cookie=8e3e89339e4b60d6 in_seq=0 out_seq=0 2024-03-16T09:17:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.813+0000 7fdcd57fa640 10 client.15801 ms_handle_connect on v2:172.21.15.159:6836/4023942203 2024-03-16T09:17:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.813+0000 7fdcd57fa640 1 -- 172.21.15.112:0/936623950 <== mds.2 v2:172.21.15.159:6836/4023942203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a9d4408d20 con 0x55a9d4519500 2024-03-16T09:17:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.813+0000 7fdcd57fa640 10 client.15801 handle_command_reply: tid=0 2024-03-16T09:17:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.813+0000 7fdcf67c1740 1 client.15801 shutdown 2024-03-16T09:17:17.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.813+0000 7fdcf67c1740 1 -- 172.21.15.112:0/936623950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d449c850 msgr2=0x55a9d44abab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.813+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/936623950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d449c850 0x55a9d44abab0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fdce809b0a0 tx=0x7fdce80a3c80 comp rx=0 tx=0).stop 2024-03-16T09:17:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.813+0000 7fdcf67c1740 1 -- 172.21.15.112:0/936623950 shutdown_connections 2024-03-16T09:17:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.813+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/936623950 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x55a9d4519500 0x55a9d451b920 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.813+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/936623950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a9d44a23b0 0x55a9d44acfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.813+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/936623950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a9d449d240 0x55a9d44abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.813+0000 7fdcd57fa640 0 client.15801 ms_handle_reset on v2:172.21.15.159:6836/4023942203 2024-03-16T09:17:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.813+0000 7fdcf67c1740 1 --2- 172.21.15.112:0/936623950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9d449c850 0x55a9d44abab0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.813+0000 7fdcf67c1740 1 -- 172.21.15.112:0/936623950 >> 172.21.15.112:0/936623950 conn(0x55a9d4497dc0 msgr2=0x55a9d449b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.813+0000 7fdcf67c1740 1 -- 172.21.15.112:0/936623950 shutdown_connections 2024-03-16T09:17:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.813+0000 7fdcf67c1740 1 -- 172.21.15.112:0/936623950 wait complete. 2024-03-16T09:17:17.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.814+0000 7fdcf67c1740 20 client.15801 trim_cache size 0 max 16384 2024-03-16T09:17:17.824 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-03-16T09:17:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.814+0000 7fdcf3af8640 1 -- 172.21.15.112:0/1763553412 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdcd80a2800 msgr2=0x7fdcd80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.814+0000 7fdcf3af8640 1 --2- 172.21.15.112:0/1763553412 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdcd80a2800 0x7fdcd80a4cc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdce400fc90 tx=0x7fdce401e000 comp rx=0 tx=0).stop 2024-03-16T09:17:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.814+0000 7fdcf3af8640 1 -- 172.21.15.112:0/1763553412 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcec1006e0 msgr2=0x7fdcec0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.814+0000 7fdcf3af8640 1 --2- 172.21.15.112:0/1763553412 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcec1006e0 0x7fdcec0694b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdce800e9c0 tx=0x7fdce800ee80 comp rx=0 tx=0).stop 2024-03-16T09:17:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.814+0000 7fdcf3af8640 1 -- 172.21.15.112:0/1763553412 shutdown_connections 2024-03-16T09:17:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.814+0000 7fdcf3af8640 1 --2- 172.21.15.112:0/1763553412 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fdcd80a2800 0x7fdcd80a4cc0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.814+0000 7fdcf3af8640 1 --2- 172.21.15.112:0/1763553412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcec102290 0x7fdcec19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.814+0000 7fdcf3af8640 1 --2- 172.21.15.112:0/1763553412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcec1018d0 0x7fdcec19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.814+0000 7fdcf3af8640 1 --2- 172.21.15.112:0/1763553412 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdcec1006e0 0x7fdcec0694b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.814+0000 7fdcf3af8640 1 -- 172.21.15.112:0/1763553412 >> 172.21.15.112:0/1763553412 conn(0x7fdcec0fbc70 msgr2=0x7fdcec104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.814+0000 7fdcf3af8640 1 -- 172.21.15.112:0/1763553412 shutdown_connections 2024-03-16T09:17:17.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.814+0000 7fdcf3af8640 1 -- 172.21.15.112:0/1763553412 wait complete. 2024-03-16T09:17:17.830 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T09:17:17.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.931+0000 7f127d4dc640 1 -- 172.21.15.112:0/4214881729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12781029e0 msgr2=0x7f1278102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.931+0000 7f127d4dc640 1 --2- 172.21.15.112:0/4214881729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12781029e0 0x7f1278102e60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1268009900 tx=0x7f126802bc30 comp rx=0 tx=0).stop 2024-03-16T09:17:17.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.931+0000 7f127d4dc640 1 -- 172.21.15.112:0/4214881729 shutdown_connections 2024-03-16T09:17:17.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.931+0000 7f127d4dc640 1 --2- 172.21.15.112:0/4214881729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12781033a0 0x7f1278109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.931+0000 7f127d4dc640 1 --2- 172.21.15.112:0/4214881729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12781029e0 0x7f1278102e60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.931+0000 7f127d4dc640 1 --2- 172.21.15.112:0/4214881729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12781017f0 0x7f1278101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.931+0000 7f127d4dc640 1 -- 172.21.15.112:0/4214881729 >> 172.21.15.112:0/4214881729 conn(0x7f12780fcd40 msgr2=0x7f12780ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.931+0000 7f127d4dc640 1 -- 172.21.15.112:0/4214881729 shutdown_connections 2024-03-16T09:17:17.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.931+0000 7f127d4dc640 1 -- 172.21.15.112:0/4214881729 wait complete. 2024-03-16T09:17:17.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.932+0000 7f127d4dc640 1 Processor -- start 2024-03-16T09:17:17.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.932+0000 7f127d4dc640 1 -- start start 2024-03-16T09:17:17.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.932+0000 7f127d4dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12781017f0 0x7f1278197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.932+0000 7f127d4dc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12781029e0 0x7f1278198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.932+0000 7f127d4dc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12781033a0 0x7f127819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.932+0000 7f127d4dc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f127810d550 con 0x7f12781017f0 2024-03-16T09:17:17.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.932+0000 7f127d4dc640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f127810d3d0 con 0x7f12781033a0 2024-03-16T09:17:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.932+0000 7f127d4dc640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f127810d6d0 con 0x7f12781029e0 2024-03-16T09:17:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.932+0000 7f1276ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12781017f0 0x7f1278197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.932+0000 7f1276ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12781017f0 0x7f1278197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42748/0 (socket says 172.21.15.112:42748) 2024-03-16T09:17:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.932+0000 7f1276ffd640 1 -- 172.21.15.112:0/1787957166 learned_addr learned my addr 172.21.15.112:0/1787957166 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.932+0000 7f12777fe640 1 --2- 172.21.15.112:0/1787957166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12781033a0 0x7f127819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.933+0000 7f1276ffd640 1 -- 172.21.15.112:0/1787957166 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12781029e0 msgr2=0x7f1278198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.933+0000 7f1276ffd640 1 --2- 172.21.15.112:0/1787957166 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12781029e0 0x7f1278198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.933+0000 7f1276ffd640 1 -- 172.21.15.112:0/1787957166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12781033a0 msgr2=0x7f127819f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.933+0000 7f1276ffd640 1 --2- 172.21.15.112:0/1787957166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12781033a0 0x7f127819f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.933+0000 7f1276ffd640 1 -- 172.21.15.112:0/1787957166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f127819f800 con 0x7f12781017f0 2024-03-16T09:17:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.933+0000 7f1276ffd640 1 --2- 172.21.15.112:0/1787957166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12781017f0 0x7f1278197be0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f126c00e9c0 tx=0x7f126c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.933+0000 7f125ffff640 1 -- 172.21.15.112:0/1787957166 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126c00cd80 con 0x7f12781017f0 2024-03-16T09:17:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.933+0000 7f127d4dc640 1 -- 172.21.15.112:0/1787957166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f127819faf0 con 0x7f12781017f0 2024-03-16T09:17:17.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.933+0000 7f125ffff640 1 -- 172.21.15.112:0/1787957166 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f126c0035f0 con 0x7f12781017f0 2024-03-16T09:17:17.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.933+0000 7f125ffff640 1 -- 172.21.15.112:0/1787957166 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126c010690 con 0x7f12781017f0 2024-03-16T09:17:17.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.933+0000 7f127d4dc640 1 -- 172.21.15.112:0/1787957166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12781a0040 con 0x7f12781017f0 2024-03-16T09:17:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.936+0000 7f12801a0740 1 Processor -- start 2024-03-16T09:17:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.936+0000 7f12801a0740 1 -- start start 2024-03-16T09:17:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.936+0000 7f12801a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807def0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.936+0000 7f12801a0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def1240 0x55807def5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.936+0000 7f12801a0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def63b0 0x55807def8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.936+0000 7f12801a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55807de55490 con 0x55807def0850 2024-03-16T09:17:17.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.936+0000 7f12801a0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55807ddfdc90 con 0x55807def1240 2024-03-16T09:17:17.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.936+0000 7f12801a0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55807de51530 con 0x55807def63b0 2024-03-16T09:17:17.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.936+0000 7f12777fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807def0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.936+0000 7f12777fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807def0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42764/0 (socket says 172.21.15.112:42764) 2024-03-16T09:17:17.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.936+0000 7f12777fe640 1 -- 172.21.15.112:0/2167143568 learned_addr learned my addr 172.21.15.112:0/2167143568 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:17.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.936+0000 7f12767fc640 1 --2- 172.21.15.112:0/2167143568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def63b0 0x55807def8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.936+0000 7f12777fe640 1 -- 172.21.15.112:0/2167143568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def63b0 msgr2=0x55807def8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.936+0000 7f12777fe640 1 --2- 172.21.15.112:0/2167143568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def63b0 0x55807def8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.936+0000 7f125ffff640 1 -- 172.21.15.112:0/1787957166 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7f126c0037a0 con 0x7f12781017f0 2024-03-16T09:17:17.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.936+0000 7f12777fe640 1 -- 172.21.15.112:0/2167143568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def1240 msgr2=0x55807def5d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-16T09:17:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.936+0000 7f12777fe640 1 --2- 172.21.15.112:0/2167143568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def1240 0x55807def5d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.936+0000 7f12777fe640 1 -- 172.21.15.112:0/2167143568 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55807deb8500 con 0x55807def0850 2024-03-16T09:17:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.936+0000 7f12777fe640 1 --2- 172.21.15.112:0/2167143568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807def0c70 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1270009fd0 tx=0x7f12700118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383da318cc47ad37 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.937+0000 7f125d7fa640 1 -- 172.21.15.112:0/2167143568 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1270004d20 con 0x55807def0850 2024-03-16T09:17:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.937+0000 7f125d7fa640 1 -- 172.21.15.112:0/2167143568 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1270004050 con 0x55807def0850 2024-03-16T09:17:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.937+0000 7f125d7fa640 1 -- 172.21.15.112:0/2167143568 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1270005280 con 0x55807def0850 2024-03-16T09:17:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.937+0000 7f125ffff640 1 --2- 172.21.15.112:0/1787957166 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f12600a2800 0x7f12600a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.937+0000 7f1276ffd640 1 --2- 172.21.15.112:0/1787957166 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f12600a2800 0x7f12600a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.937+0000 7f125ffff640 1 -- 172.21.15.112:0/1787957166 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f126c0ceee0 con 0x7f12781017f0 2024-03-16T09:17:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.937+0000 7f12801a0740 1 -- 172.21.15.112:0/2167143568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 msgr2=0x55807def0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.937+0000 7f12801a0740 1 --2- 172.21.15.112:0/2167143568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807def0c70 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1270009fd0 tx=0x7f12700118d0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.937+0000 7f1276ffd640 1 --2- 172.21.15.112:0/1787957166 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f12600a2800 0x7f12600a4cc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f126c0cca40 tx=0x7f126c06e5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.937+0000 7f12801a0740 1 -- 172.21.15.112:0/2167143568 shutdown_connections 2024-03-16T09:17:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.937+0000 7f12801a0740 1 --2- 172.21.15.112:0/2167143568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def63b0 0x55807def8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.937+0000 7f12801a0740 1 --2- 172.21.15.112:0/2167143568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def1240 0x55807def5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.937+0000 7f12801a0740 1 --2- 172.21.15.112:0/2167143568 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807def0c70 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.937+0000 7f12801a0740 1 -- 172.21.15.112:0/2167143568 >> 172.21.15.112:0/2167143568 conn(0x55807deebdc0 msgr2=0x55807deec1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.937+0000 7f12801a0740 1 -- 172.21.15.112:0/2167143568 shutdown_connections 2024-03-16T09:17:17.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12801a0740 1 -- 172.21.15.112:0/2167143568 wait complete. 2024-03-16T09:17:17.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12801a0740 1 -- start start 2024-03-16T09:17:17.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12801a0740 1 Processor -- start 2024-03-16T09:17:17.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12801a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807defad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12801a0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def1240 0x55807defb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12801a0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def63b0 0x55807df02270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12801a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55807def9360 con 0x55807def0850 2024-03-16T09:17:17.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12801a0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55807de11230 con 0x55807def1240 2024-03-16T09:17:17.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12801a0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55807def94e0 con 0x55807def63b0 2024-03-16T09:17:17.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12801a0740 10 client.0 _ll_register_callbacks cb 0x55807de4d010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:17.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12777fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807defad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12777fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807defad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42776/0 (socket says 172.21.15.112:42776) 2024-03-16T09:17:17.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12777fe640 1 -- 172.21.15.112:0/2975297101 learned_addr learned my addr 172.21.15.112:0/2975297101 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:17.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12767fc640 1 --2- 172.21.15.112:0/2975297101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def1240 0x55807defb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12777fe640 1 --2- 172.21.15.112:0/2975297101 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def63b0 0x55807df02270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12777fe640 1 -- 172.21.15.112:0/2975297101 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def63b0 msgr2=0x55807df02270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12777fe640 1 --2- 172.21.15.112:0/2975297101 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def63b0 0x55807df02270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12777fe640 1 -- 172.21.15.112:0/2975297101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def1240 msgr2=0x55807defb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12777fe640 1 --2- 172.21.15.112:0/2975297101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def1240 0x55807defb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.938+0000 7f12777fe640 1 -- 172.21.15.112:0/2975297101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55807de20100 con 0x55807def0850 2024-03-16T09:17:17.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.939+0000 7f12777fe640 1 --2- 172.21.15.112:0/2975297101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807defad30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f127001b040 tx=0x7f1270009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f80efd55628183 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.939+0000 7f125aff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:17.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.939+0000 7f125aff5640 1 -- 172.21.15.112:0/2975297101 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127002c020 con 0x55807def0850 2024-03-16T09:17:17.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.939+0000 7f125aff5640 1 -- 172.21.15.112:0/2975297101 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f127000f4e0 con 0x55807def0850 2024-03-16T09:17:17.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.939+0000 7f12801a0740 1 -- 172.21.15.112:0/2975297101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1270004050 con 0x55807def0850 2024-03-16T09:17:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.939+0000 7f125aff5640 1 -- 172.21.15.112:0/2975297101 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f127000f6e0 con 0x55807def0850 2024-03-16T09:17:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.939+0000 7f12801a0740 10 client.15813 fetch_fsmap learned FSMap version 23 2024-03-16T09:17:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.939+0000 7f12801a0740 1 -- 172.21.15.112:0/2975297101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55807db693f0 con 0x55807def0850 2024-03-16T09:17:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.939+0000 7f125aff5640 1 -- 172.21.15.112:0/2975297101 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1270004050 con 0x55807def0850 2024-03-16T09:17:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.939+0000 7f125aff5640 1 -- 172.21.15.112:0/2975297101 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f12700289e0 con 0x55807def0850 2024-03-16T09:17:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.939+0000 7f12801a0740 10 client.15813 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T09:17:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.939+0000 7f12801a0740 10 client.15813 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T09:17:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.939+0000 7f12801a0740 20 client.15813 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.939+0000 7f12801a0740 1 --2- 172.21.15.112:0/2975297101 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55807df06890 0x55807df08ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.939+0000 7f12801a0740 4 client.15813 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:17:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.939+0000 7f12801a0740 1 -- 172.21.15.112:0/2975297101 --> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55807de629d0 con 0x55807df06890 2024-03-16T09:17:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.940+0000 7f12767fc640 1 --2- 172.21.15.112:0/2975297101 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55807df06890 0x55807df08ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.940+0000 7f125aff5640 10 client.15813 ms_handle_connect on v2:172.21.15.194:6838/2528943309 2024-03-16T09:17:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.940+0000 7f12767fc640 1 --2- 172.21.15.112:0/2975297101 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55807df06890 0x55807df08ca0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=85dcdc0cb67455f7 server_cookie=7b4f1e9b218a322e in_seq=0 out_seq=0 2024-03-16T09:17:17.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.941+0000 7f125aff5640 1 -- 172.21.15.112:0/2975297101 <== mds.1 v2:172.21.15.194:6838/2528943309 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55807de629d0 con 0x55807df06890 2024-03-16T09:17:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.941+0000 7f125aff5640 10 client.15813 handle_command_reply: tid=0 2024-03-16T09:17:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.941+0000 7f12801a0740 1 client.15813 shutdown 2024-03-16T09:17:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.941+0000 7f12801a0740 1 -- 172.21.15.112:0/2975297101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 msgr2=0x55807defad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.941+0000 7f12801a0740 1 --2- 172.21.15.112:0/2975297101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807defad30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f127001b040 tx=0x7f1270009a50 comp rx=0 tx=0).stop 2024-03-16T09:17:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.941+0000 7f12801a0740 1 -- 172.21.15.112:0/2975297101 shutdown_connections 2024-03-16T09:17:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.941+0000 7f12801a0740 1 --2- 172.21.15.112:0/2975297101 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55807df06890 0x55807df08ca0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.941+0000 7f12801a0740 1 --2- 172.21.15.112:0/2975297101 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def63b0 0x55807df02270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.941+0000 7f12801a0740 1 --2- 172.21.15.112:0/2975297101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def1240 0x55807defb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.941+0000 7f125aff5640 0 client.15813 ms_handle_reset on v2:172.21.15.194:6838/2528943309 2024-03-16T09:17:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.941+0000 7f12801a0740 1 --2- 172.21.15.112:0/2975297101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807defad30 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.941+0000 7f12801a0740 1 -- 172.21.15.112:0/2975297101 >> 172.21.15.112:0/2975297101 conn(0x55807deebdc0 msgr2=0x55807deeafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.941+0000 7f12801a0740 1 -- 172.21.15.112:0/2975297101 shutdown_connections 2024-03-16T09:17:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.941+0000 7f12801a0740 1 -- 172.21.15.112:0/2975297101 wait complete. 2024-03-16T09:17:17.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.941+0000 7f12801a0740 20 client.15813 trim_cache size 0 max 16384 2024-03-16T09:17:17.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.954+0000 7f12801a0740 1 Processor -- start 2024-03-16T09:17:17.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.954+0000 7f12801a0740 1 -- start start 2024-03-16T09:17:17.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.954+0000 7f12801a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807deea750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.954+0000 7f12801a0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def1240 0x55807df02230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.954+0000 7f12801a0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def63b0 0x55807deeac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.954+0000 7f12801a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12781ab7b0 con 0x55807def0850 2024-03-16T09:17:17.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.954+0000 7f12801a0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f12781a9080 con 0x55807def1240 2024-03-16T09:17:17.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.954+0000 7f12801a0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f12781a6ed0 con 0x55807def63b0 2024-03-16T09:17:17.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.954+0000 7f12777fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807deea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.954+0000 7f12777fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807deea750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42782/0 (socket says 172.21.15.112:42782) 2024-03-16T09:17:17.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.954+0000 7f12777fe640 1 -- 172.21.15.112:0/248770261 learned_addr learned my addr 172.21.15.112:0/248770261 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.955+0000 7f12777fe640 1 --2- 172.21.15.112:0/248770261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def63b0 0x55807deeac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.955+0000 7f12777fe640 1 -- 172.21.15.112:0/248770261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def63b0 msgr2=0x55807deeac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.955+0000 7f12777fe640 1 --2- 172.21.15.112:0/248770261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def63b0 0x55807deeac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.955+0000 7f12767fc640 1 --2- 172.21.15.112:0/248770261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def1240 0x55807df02230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.955+0000 7f12777fe640 1 -- 172.21.15.112:0/248770261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def1240 msgr2=0x55807df02230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.955+0000 7f12777fe640 1 --2- 172.21.15.112:0/248770261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def1240 0x55807df02230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.955+0000 7f12777fe640 1 -- 172.21.15.112:0/248770261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55807dc337d0 con 0x55807def0850 2024-03-16T09:17:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.955+0000 7f12767fc640 1 --2- 172.21.15.112:0/248770261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def1240 0x55807df02230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:17:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.955+0000 7f12777fe640 1 --2- 172.21.15.112:0/248770261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807deea750 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1270006010 tx=0x7f1270014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707f9df052b774a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.955+0000 7f125cff9640 1 -- 172.21.15.112:0/248770261 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127002c030 con 0x55807def0850 2024-03-16T09:17:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.955+0000 7f125cff9640 1 -- 172.21.15.112:0/248770261 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1270009ac0 con 0x55807def0850 2024-03-16T09:17:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.955+0000 7f125cff9640 1 -- 172.21.15.112:0/248770261 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1270028740 con 0x55807def0850 2024-03-16T09:17:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12801a0740 1 -- 172.21.15.112:0/248770261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 msgr2=0x55807deea750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12801a0740 1 --2- 172.21.15.112:0/248770261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807deea750 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1270006010 tx=0x7f1270014040 comp rx=0 tx=0).stop 2024-03-16T09:17:17.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12801a0740 1 -- 172.21.15.112:0/248770261 shutdown_connections 2024-03-16T09:17:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12801a0740 1 --2- 172.21.15.112:0/248770261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def63b0 0x55807deeac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12801a0740 1 --2- 172.21.15.112:0/248770261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def1240 0x55807df02230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12801a0740 1 --2- 172.21.15.112:0/248770261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807deea750 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12801a0740 1 -- 172.21.15.112:0/248770261 >> 172.21.15.112:0/248770261 conn(0x55807deebdc0 msgr2=0x55807defacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12801a0740 1 -- 172.21.15.112:0/248770261 shutdown_connections 2024-03-16T09:17:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12801a0740 1 -- 172.21.15.112:0/248770261 wait complete. 2024-03-16T09:17:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12801a0740 1 -- start start 2024-03-16T09:17:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12801a0740 1 Processor -- start 2024-03-16T09:17:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12801a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807deff730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12801a0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def1240 0x55807deffc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12801a0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def63b0 0x55807df00c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12801a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12781a7540 con 0x55807def0850 2024-03-16T09:17:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12801a0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f12781a89b0 con 0x55807def63b0 2024-03-16T09:17:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12801a0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f12781a7c10 con 0x55807def1240 2024-03-16T09:17:17.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12801a0740 10 client.0 _ll_register_callbacks cb 0x7f126c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12777fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807deff730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12777fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807deff730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42790/0 (socket says 172.21.15.112:42790) 2024-03-16T09:17:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12767fc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def1240 0x55807deffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.956+0000 7f12777fe640 1 -- 172.21.15.112:0/1892722673 learned_addr learned my addr 172.21.15.112:0/1892722673 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.957+0000 7f12777fe640 1 --2- 172.21.15.112:0/1892722673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def63b0 0x55807df00c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.957+0000 7f12777fe640 1 -- 172.21.15.112:0/1892722673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def1240 msgr2=0x55807deffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.957+0000 7f12777fe640 1 --2- 172.21.15.112:0/1892722673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def1240 0x55807deffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.957+0000 7f12777fe640 1 -- 172.21.15.112:0/1892722673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def63b0 msgr2=0x55807df00c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.957+0000 7f12777fe640 1 --2- 172.21.15.112:0/1892722673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def63b0 0x55807df00c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.957+0000 7f12777fe640 1 -- 172.21.15.112:0/1892722673 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55807df01270 con 0x55807def0850 2024-03-16T09:17:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.957+0000 7f12777fe640 1 --2- 172.21.15.112:0/1892722673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807deff730 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f12700039e0 tx=0x7f1270009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08844a44286c752 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.957+0000 7f125aff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:17.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.957+0000 7f125aff5640 1 -- 172.21.15.112:0/1892722673 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1270043070 con 0x55807def0850 2024-03-16T09:17:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.957+0000 7f125aff5640 1 -- 172.21.15.112:0/1892722673 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f127000fa80 con 0x55807def0850 2024-03-16T09:17:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.957+0000 7f12801a0740 1 -- 172.21.15.112:0/1892722673 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1270009ac0 con 0x55807def0850 2024-03-16T09:17:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.957+0000 7f125aff5640 1 -- 172.21.15.112:0/1892722673 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f127002c040 con 0x55807def0850 2024-03-16T09:17:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.957+0000 7f12801a0740 10 client.15819 fetch_fsmap learned FSMap version 23 2024-03-16T09:17:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.957+0000 7f12801a0740 1 -- 172.21.15.112:0/1892722673 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55807df024e0 con 0x55807def0850 2024-03-16T09:17:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.957+0000 7f125aff5640 1 -- 172.21.15.112:0/1892722673 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1270009ac0 con 0x55807def0850 2024-03-16T09:17:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f125aff5640 1 -- 172.21.15.112:0/1892722673 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f12700194c0 con 0x55807def0850 2024-03-16T09:17:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f12801a0740 10 client.15819 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T09:17:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f12801a0740 10 client.15819 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T09:17:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f12801a0740 20 client.15819 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f12801a0740 1 --2- 172.21.15.112:0/1892722673 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55807df06890 0x55807df6f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f12801a0740 4 client.15819 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T09:17:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f12801a0740 1 -- 172.21.15.112:0/1892722673 --> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55807de5cd20 con 0x55807df06890 2024-03-16T09:17:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f12767fc640 1 --2- 172.21.15.112:0/1892722673 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55807df06890 0x55807df6f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:17.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f125aff5640 10 client.15819 ms_handle_connect on v2:172.21.15.194:6838/2528943309 2024-03-16T09:17:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f12767fc640 1 --2- 172.21.15.112:0/1892722673 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55807df06890 0x55807df6f030 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2da107ca1789507e server_cookie=2ec16484fa57d0b4 in_seq=0 out_seq=0 2024-03-16T09:17:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f125aff5640 1 -- 172.21.15.112:0/1892722673 <== mds.1 v2:172.21.15.194:6838/2528943309 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55807de5cd20 con 0x55807df06890 2024-03-16T09:17:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f125aff5640 10 client.15819 handle_command_reply: tid=0 2024-03-16T09:17:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f12801a0740 1 client.15819 shutdown 2024-03-16T09:17:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f12801a0740 1 -- 172.21.15.112:0/1892722673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 msgr2=0x55807deff730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f12801a0740 1 --2- 172.21.15.112:0/1892722673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807deff730 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f12700039e0 tx=0x7f1270009230 comp rx=0 tx=0).stop 2024-03-16T09:17:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f12801a0740 1 -- 172.21.15.112:0/1892722673 shutdown_connections 2024-03-16T09:17:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f12801a0740 1 --2- 172.21.15.112:0/1892722673 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x55807df06890 0x55807df6f030 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f12801a0740 1 --2- 172.21.15.112:0/1892722673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55807def63b0 0x55807df00c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f12801a0740 1 --2- 172.21.15.112:0/1892722673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55807def1240 0x55807deffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.958+0000 7f12801a0740 1 --2- 172.21.15.112:0/1892722673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55807def0850 0x55807deff730 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.959+0000 7f12801a0740 1 -- 172.21.15.112:0/1892722673 >> 172.21.15.112:0/1892722673 conn(0x55807deebdc0 msgr2=0x55807deef020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.959+0000 7f125aff5640 0 client.15819 ms_handle_reset on v2:172.21.15.194:6838/2528943309 2024-03-16T09:17:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.959+0000 7f12801a0740 1 -- 172.21.15.112:0/1892722673 shutdown_connections 2024-03-16T09:17:17.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.959+0000 7f12801a0740 1 -- 172.21.15.112:0/1892722673 wait complete. 2024-03-16T09:17:17.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.959+0000 7f12801a0740 20 client.15819 trim_cache size 0 max 16384 2024-03-16T09:17:17.969 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-03-16T09:17:17.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.959+0000 7f127d4dc640 1 -- 172.21.15.112:0/1787957166 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f12600a2800 msgr2=0x7f12600a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.959+0000 7f127d4dc640 1 --2- 172.21.15.112:0/1787957166 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f12600a2800 0x7f12600a4cc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f126c0cca40 tx=0x7f126c06e5c0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.959+0000 7f127d4dc640 1 -- 172.21.15.112:0/1787957166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12781017f0 msgr2=0x7f1278197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:17.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.959+0000 7f127d4dc640 1 --2- 172.21.15.112:0/1787957166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12781017f0 0x7f1278197be0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f126c00e9c0 tx=0x7f126c00ee80 comp rx=0 tx=0).stop 2024-03-16T09:17:17.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.959+0000 7f127d4dc640 1 -- 172.21.15.112:0/1787957166 shutdown_connections 2024-03-16T09:17:17.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.959+0000 7f127d4dc640 1 --2- 172.21.15.112:0/1787957166 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f12600a2800 0x7f12600a4cc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.959+0000 7f127d4dc640 1 --2- 172.21.15.112:0/1787957166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12781033a0 0x7f127819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.959+0000 7f127d4dc640 1 --2- 172.21.15.112:0/1787957166 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12781029e0 0x7f1278198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.959+0000 7f127d4dc640 1 --2- 172.21.15.112:0/1787957166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12781017f0 0x7f1278197be0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:17.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.959+0000 7f127d4dc640 1 -- 172.21.15.112:0/1787957166 >> 172.21.15.112:0/1787957166 conn(0x7f12780fcd40 msgr2=0x7f1278105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:17.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.959+0000 7f127d4dc640 1 -- 172.21.15.112:0/1787957166 shutdown_connections 2024-03-16T09:17:17.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:17.959+0000 7f127d4dc640 1 -- 172.21.15.112:0/1787957166 wait complete. 2024-03-16T09:17:17.973 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T09:17:18.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.076+0000 7f07b3b10640 1 -- 172.21.15.112:0/1639285355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07ac102290 msgr2=0x7f07ac108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.076+0000 7f07b3b10640 1 --2- 172.21.15.112:0/1639285355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07ac102290 0x7f07ac108b20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f07a4009ca0 tx=0x7f07a402c0f0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.076+0000 7f07b3b10640 1 -- 172.21.15.112:0/1639285355 shutdown_connections 2024-03-16T09:17:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.076+0000 7f07b3b10640 1 --2- 172.21.15.112:0/1639285355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07ac102290 0x7f07ac108b20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.076+0000 7f07b3b10640 1 --2- 172.21.15.112:0/1639285355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07ac1018d0 0x7f07ac101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.076+0000 7f07b3b10640 1 --2- 172.21.15.112:0/1639285355 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07ac1006e0 0x7f07ac100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.076+0000 7f07b3b10640 1 -- 172.21.15.112:0/1639285355 >> 172.21.15.112:0/1639285355 conn(0x7f07ac0fbc70 msgr2=0x7f07ac0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.076+0000 7f07b3b10640 1 -- 172.21.15.112:0/1639285355 shutdown_connections 2024-03-16T09:17:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.076+0000 7f07b3b10640 1 -- 172.21.15.112:0/1639285355 wait complete. 2024-03-16T09:17:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.077+0000 7f07b3b10640 1 Processor -- start 2024-03-16T09:17:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.077+0000 7f07b3b10640 1 -- start start 2024-03-16T09:17:18.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.077+0000 7f07b3b10640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07ac1006e0 0x7f07ac0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:18.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.077+0000 7f07b3b10640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07ac1018d0 0x7f07ac19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:18.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.077+0000 7f07b3b10640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07ac102290 0x7f07ac19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:18.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.077+0000 7f07b3b10640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac10c490 con 0x7f07ac102290 2024-03-16T09:17:18.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.077+0000 7f07b3b10640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac10c310 con 0x7f07ac1018d0 2024-03-16T09:17:18.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.077+0000 7f07b3b10640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac10c610 con 0x7f07ac1006e0 2024-03-16T09:17:18.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.077+0000 7f07b20ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07ac102290 0x7f07ac19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:18.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.077+0000 7f07b18ac640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07ac1006e0 0x7f07ac0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:18.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.077+0000 7f07b10ab640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07ac1018d0 0x7f07ac19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:18.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.077+0000 7f07b20ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07ac102290 0x7f07ac19ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42800/0 (socket says 172.21.15.112:42800) 2024-03-16T09:17:18.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.077+0000 7f07b20ad640 1 -- 172.21.15.112:0/645839589 learned_addr learned my addr 172.21.15.112:0/645839589 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:18.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.078+0000 7f07b20ad640 1 -- 172.21.15.112:0/645839589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07ac1006e0 msgr2=0x7f07ac0694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:18.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.078+0000 7f07b20ad640 1 --2- 172.21.15.112:0/645839589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07ac1006e0 0x7f07ac0694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.078+0000 7f07b20ad640 1 -- 172.21.15.112:0/645839589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07ac1018d0 msgr2=0x7f07ac19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:18.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.078+0000 7f07b20ad640 1 --2- 172.21.15.112:0/645839589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07ac1018d0 0x7f07ac19c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.078+0000 7f07b20ad640 1 -- 172.21.15.112:0/645839589 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ac19f830 con 0x7f07ac102290 2024-03-16T09:17:18.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.078+0000 7f07b20ad640 1 --2- 172.21.15.112:0/645839589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07ac102290 0x7f07ac19ef20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f07a402c700 tx=0x7f07a40346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:18.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.078+0000 7f07a2ffd640 1 -- 172.21.15.112:0/645839589 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4039040 con 0x7f07ac102290 2024-03-16T09:17:18.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.078+0000 7f07b3b10640 1 -- 172.21.15.112:0/645839589 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07ac19fa60 con 0x7f07ac102290 2024-03-16T09:17:18.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.078+0000 7f07a2ffd640 1 -- 172.21.15.112:0/645839589 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f07a4044070 con 0x7f07ac102290 2024-03-16T09:17:18.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.078+0000 7f07a2ffd640 1 -- 172.21.15.112:0/645839589 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a40043c0 con 0x7f07ac102290 2024-03-16T09:17:18.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.078+0000 7f07b3b10640 1 -- 172.21.15.112:0/645839589 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07ac19ff50 con 0x7f07ac102290 2024-03-16T09:17:18.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.080+0000 7f07a2ffd640 1 -- 172.21.15.112:0/645839589 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7f07a4004580 con 0x7f07ac102290 2024-03-16T09:17:18.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.081+0000 7f07b67db740 1 Processor -- start 2024-03-16T09:17:18.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.081+0000 7f07b67db740 1 -- start start 2024-03-16T09:17:18.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.081+0000 7f07b67db740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 0x55ed82bbac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:18.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.081+0000 7f07b67db740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bbb240 0x55ed82bbfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:18.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.081+0000 7f07b67db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bc03b0 0x55ed82bc2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:18.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.081+0000 7f07a2ffd640 1 --2- 172.21.15.112:0/645839589 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f07980a2800 0x7f07980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:18.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.081+0000 7f07b67db740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ed82b1f490 con 0x55ed82bba850 2024-03-16T09:17:18.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.081+0000 7f07b67db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ed82ac7c90 con 0x55ed82bc03b0 2024-03-16T09:17:18.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.081+0000 7f07b67db740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ed82b1b530 con 0x55ed82bbb240 2024-03-16T09:17:18.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.081+0000 7f07b18ac640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bbb240 0x55ed82bbfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:18.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.081+0000 7f07b10ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 0x55ed82bbac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:18.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.081+0000 7f07b10ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 0x55ed82bbac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42812/0 (socket says 172.21.15.112:42812) 2024-03-16T09:17:18.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.081+0000 7f07a2ffd640 1 -- 172.21.15.112:0/645839589 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f07a40f0f90 con 0x7f07ac102290 2024-03-16T09:17:18.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.081+0000 7f07b18ac640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bbb240 0x55ed82bbfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:34548/0 (socket says 172.21.15.112:34548) 2024-03-16T09:17:18.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.081+0000 7f07b10ab640 1 -- 172.21.15.112:0/3872965278 learned_addr learned my addr 172.21.15.112:0/3872965278 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:18.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.081+0000 7f07b20ad640 1 --2- 172.21.15.112:0/3872965278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bc03b0 0x55ed82bc2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:18.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.081+0000 7f07b10ab640 1 --2- 172.21.15.112:0/645839589 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f07980a2800 0x7f07980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:18.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.082+0000 7f07b20ad640 1 -- 172.21.15.112:0/3872965278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bbb240 msgr2=0x55ed82bbfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:18.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.082+0000 7f07b20ad640 1 --2- 172.21.15.112:0/3872965278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bbb240 0x55ed82bbfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.082+0000 7f07b20ad640 1 -- 172.21.15.112:0/3872965278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 msgr2=0x55ed82bbac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:18.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.082+0000 7f07b20ad640 1 --2- 172.21.15.112:0/3872965278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 0x55ed82bbac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.082+0000 7f07b20ad640 1 -- 172.21.15.112:0/3872965278 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed82b82500 con 0x55ed82bc03b0 2024-03-16T09:17:18.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.082+0000 7f07b18ac640 1 --2- 172.21.15.112:0/3872965278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bbb240 0x55ed82bbfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:18.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.082+0000 7f07b20ad640 1 --2- 172.21.15.112:0/3872965278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bc03b0 0x55ed82bc2840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f07a40eeb90 tx=0x7f07a402c0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5918fe4b8b932c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:18.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.082+0000 7f07b10ab640 1 --2- 172.21.15.112:0/645839589 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f07980a2800 0x7f07980a4cc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55ed82bbbaa0 tx=0x7f079c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:18.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.082+0000 7f0797fff640 1 -- 172.21.15.112:0/3872965278 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4032340 con 0x55ed82bc03b0 2024-03-16T09:17:18.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.082+0000 7f0797fff640 1 -- 172.21.15.112:0/3872965278 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f07a4032970 con 0x55ed82bc03b0 2024-03-16T09:17:18.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.082+0000 7f0797fff640 1 -- 172.21.15.112:0/3872965278 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a40e0dc0 con 0x55ed82bc03b0 2024-03-16T09:17:18.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.083+0000 7f07b67db740 1 -- 172.21.15.112:0/3872965278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bc03b0 msgr2=0x55ed82bc2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:18.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.083+0000 7f07b67db740 1 --2- 172.21.15.112:0/3872965278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bc03b0 0x55ed82bc2840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f07a40eeb90 tx=0x7f07a402c0f0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.083+0000 7f07b67db740 1 -- 172.21.15.112:0/3872965278 shutdown_connections 2024-03-16T09:17:18.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.083+0000 7f07b67db740 1 --2- 172.21.15.112:0/3872965278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bc03b0 0x55ed82bc2840 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.083+0000 7f07b67db740 1 --2- 172.21.15.112:0/3872965278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bbb240 0x55ed82bbfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.083+0000 7f07b67db740 1 --2- 172.21.15.112:0/3872965278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 0x55ed82bbac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.083+0000 7f07b67db740 1 -- 172.21.15.112:0/3872965278 >> 172.21.15.112:0/3872965278 conn(0x55ed82bb5dc0 msgr2=0x55ed82bb61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:18.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.083+0000 7f07b67db740 1 -- 172.21.15.112:0/3872965278 shutdown_connections 2024-03-16T09:17:18.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.083+0000 7f07b67db740 1 -- 172.21.15.112:0/3872965278 wait complete. 2024-03-16T09:17:18.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.083+0000 7f07b67db740 1 -- start start 2024-03-16T09:17:18.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.083+0000 7f07b67db740 1 Processor -- start 2024-03-16T09:17:18.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b67db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bba850 0x55ed82bc4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:18.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b67db740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bbb240 0x55ed82bc50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:18.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b67db740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bc03b0 0x55ed82bcc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:18.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b67db740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ed82bc3120 con 0x55ed82bc03b0 2024-03-16T09:17:18.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b67db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ed82adb230 con 0x55ed82bba850 2024-03-16T09:17:18.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b67db740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ed82bc32a0 con 0x55ed82bbb240 2024-03-16T09:17:18.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b67db740 10 client.0 _ll_register_callbacks cb 0x55ed82b17010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:18.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b20ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bc03b0 0x55ed82bcc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:18.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b18ac640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bba850 0x55ed82bc4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:18.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b20ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bc03b0 0x55ed82bcc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42822/0 (socket says 172.21.15.112:42822) 2024-03-16T09:17:18.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b20ad640 1 -- 172.21.15.112:0/3053094454 learned_addr learned my addr 172.21.15.112:0/3053094454 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:18.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b10ab640 1 --2- 172.21.15.112:0/3053094454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bbb240 0x55ed82bc50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:18.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b20ad640 1 -- 172.21.15.112:0/3053094454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bbb240 msgr2=0x55ed82bc50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:18.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b20ad640 1 --2- 172.21.15.112:0/3053094454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bbb240 0x55ed82bc50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b20ad640 1 -- 172.21.15.112:0/3053094454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bba850 msgr2=0x55ed82bc4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:18.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b20ad640 1 --2- 172.21.15.112:0/3053094454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bba850 0x55ed82bc4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b20ad640 1 -- 172.21.15.112:0/3053094454 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed82aea100 con 0x55ed82bc03b0 2024-03-16T09:17:18.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b18ac640 1 --2- 172.21.15.112:0/3053094454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bba850 0x55ed82bc4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:18.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b20ad640 1 --2- 172.21.15.112:0/3053094454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bc03b0 0x55ed82bcc0e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f07a40c5c50 tx=0x7f07a40bbea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f484fa203a72bbd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:18.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07957fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:18.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07957fa640 1 -- 172.21.15.112:0/3053094454 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a40bd9e0 con 0x55ed82bc03b0 2024-03-16T09:17:18.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07957fa640 1 -- 172.21.15.112:0/3053094454 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f07a40c5cf0 con 0x55ed82bc03b0 2024-03-16T09:17:18.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b67db740 1 -- 172.21.15.112:0/3053094454 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ed82b2c9d0 con 0x55ed82bc03b0 2024-03-16T09:17:18.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07957fa640 1 -- 172.21.15.112:0/3053094454 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f07a40c1b30 con 0x55ed82bc03b0 2024-03-16T09:17:18.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b67db740 10 client.15831 fetch_fsmap learned FSMap version 23 2024-03-16T09:17:18.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.084+0000 7f07b67db740 1 -- 172.21.15.112:0/3053094454 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ed828333f0 con 0x55ed82bc03b0 2024-03-16T09:17:18.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.085+0000 7f07957fa640 1 -- 172.21.15.112:0/3053094454 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x55ed82b2c9d0 con 0x55ed82bc03b0 2024-03-16T09:17:18.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.085+0000 7f07957fa640 1 -- 172.21.15.112:0/3053094454 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f07a403c030 con 0x55ed82bc03b0 2024-03-16T09:17:18.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.085+0000 7f07b67db740 10 client.15831 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T09:17:18.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.085+0000 7f07b67db740 10 client.15831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:18.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.085+0000 7f07b67db740 20 client.15831 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:18.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.085+0000 7f07b67db740 1 --2- 172.21.15.112:0/3053094454 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55ed82bd06e0 0x55ed82bd2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:18.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.085+0000 7f07b67db740 4 client.15831 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:17:18.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.085+0000 7f07b67db740 1 -- 172.21.15.112:0/3053094454 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed82b26d20 con 0x55ed82bd06e0 2024-03-16T09:17:18.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.085+0000 7f07b18ac640 1 --2- 172.21.15.112:0/3053094454 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55ed82bd06e0 0x55ed82bd2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:18.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.085+0000 7f07957fa640 10 client.15831 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:18.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.085+0000 7f07b18ac640 1 --2- 172.21.15.112:0/3053094454 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55ed82bd06e0 0x55ed82bd2b40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0fd4512a39227d server_cookie=f3da86b750202dc in_seq=0 out_seq=0 2024-03-16T09:17:18.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.086+0000 7f07957fa640 1 -- 172.21.15.112:0/3053094454 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ed82b26d20 con 0x55ed82bd06e0 2024-03-16T09:17:18.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.086+0000 7f07957fa640 10 client.15831 handle_command_reply: tid=0 2024-03-16T09:17:18.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.086+0000 7f07b67db740 1 client.15831 shutdown 2024-03-16T09:17:18.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.087+0000 7f07b67db740 1 -- 172.21.15.112:0/3053094454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bc03b0 msgr2=0x55ed82bcc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:18.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.087+0000 7f07b67db740 1 --2- 172.21.15.112:0/3053094454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bc03b0 0x55ed82bcc0e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f07a40c5c50 tx=0x7f07a40bbea0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.087+0000 7f07b67db740 1 -- 172.21.15.112:0/3053094454 shutdown_connections 2024-03-16T09:17:18.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.087+0000 7f07b67db740 1 --2- 172.21.15.112:0/3053094454 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bc03b0 0x55ed82bcc0e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.087+0000 7f07b67db740 1 --2- 172.21.15.112:0/3053094454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bbb240 0x55ed82bc50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.087+0000 7f07b67db740 1 --2- 172.21.15.112:0/3053094454 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55ed82bd06e0 0x55ed82bd2b40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.087+0000 7f07b67db740 1 --2- 172.21.15.112:0/3053094454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bba850 0x55ed82bc4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.087+0000 7f07b67db740 1 -- 172.21.15.112:0/3053094454 >> 172.21.15.112:0/3053094454 conn(0x55ed82bb5dc0 msgr2=0x55ed82bb4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:18.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.087+0000 7f07957fa640 0 client.15831 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:18.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.087+0000 7f07b67db740 1 -- 172.21.15.112:0/3053094454 shutdown_connections 2024-03-16T09:17:18.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.087+0000 7f07b67db740 1 -- 172.21.15.112:0/3053094454 wait complete. 2024-03-16T09:17:18.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.087+0000 7f07b67db740 20 client.15831 trim_cache size 0 max 16384 2024-03-16T09:17:18.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b67db740 1 Processor -- start 2024-03-16T09:17:18.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b67db740 1 -- start start 2024-03-16T09:17:18.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b67db740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 0x55ed82bb4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:18.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b67db740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bbb240 0x55ed82bcc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:18.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b67db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bc03b0 0x55ed82bb4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:18.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b67db740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac1a9280 con 0x55ed82bba850 2024-03-16T09:17:18.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b67db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac1a6b50 con 0x55ed82bc03b0 2024-03-16T09:17:18.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b67db740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac1a49a0 con 0x55ed82bbb240 2024-03-16T09:17:18.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b18ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 0x55ed82bb4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:18.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b18ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 0x55ed82bb4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42830/0 (socket says 172.21.15.112:42830) 2024-03-16T09:17:18.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b10ab640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bbb240 0x55ed82bcc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:18.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b18ac640 1 -- 172.21.15.112:0/1158084649 learned_addr learned my addr 172.21.15.112:0/1158084649 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:18.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b20ad640 1 --2- 172.21.15.112:0/1158084649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bc03b0 0x55ed82bb4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:18.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b18ac640 1 -- 172.21.15.112:0/1158084649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bbb240 msgr2=0x55ed82bcc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b18ac640 1 --2- 172.21.15.112:0/1158084649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bbb240 0x55ed82bcc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b18ac640 1 -- 172.21.15.112:0/1158084649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bc03b0 msgr2=0x55ed82bb4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b18ac640 1 --2- 172.21.15.112:0/1158084649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bc03b0 0x55ed82bb4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b18ac640 1 -- 172.21.15.112:0/1158084649 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed828fd7d0 con 0x55ed82bba850 2024-03-16T09:17:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b20ad640 1 --2- 172.21.15.112:0/1158084649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bc03b0 0x55ed82bb4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.101+0000 7f07b18ac640 1 --2- 172.21.15.112:0/1158084649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 0x55ed82bb4750 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x55ed82bbb640 tx=0x7f07a8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e287de3f020707e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.102+0000 7f07977fe640 1 -- 172.21.15.112:0/1158084649 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8024070 con 0x55ed82bba850 2024-03-16T09:17:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.102+0000 7f07977fe640 1 -- 172.21.15.112:0/1158084649 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f07a80047a0 con 0x55ed82bba850 2024-03-16T09:17:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.102+0000 7f07977fe640 1 -- 172.21.15.112:0/1158084649 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8020520 con 0x55ed82bba850 2024-03-16T09:17:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.102+0000 7f07b67db740 1 -- 172.21.15.112:0/1158084649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 msgr2=0x55ed82bb4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.102+0000 7f07b67db740 1 --2- 172.21.15.112:0/1158084649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 0x55ed82bb4750 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x55ed82bbb640 tx=0x7f07a8005f90 comp rx=0 tx=0).stop 2024-03-16T09:17:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.102+0000 7f07b67db740 1 -- 172.21.15.112:0/1158084649 shutdown_connections 2024-03-16T09:17:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.102+0000 7f07b67db740 1 --2- 172.21.15.112:0/1158084649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bc03b0 0x55ed82bb4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.102+0000 7f07b67db740 1 --2- 172.21.15.112:0/1158084649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bbb240 0x55ed82bcc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.102+0000 7f07b67db740 1 --2- 172.21.15.112:0/1158084649 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 0x55ed82bb4750 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.102+0000 7f07b67db740 1 -- 172.21.15.112:0/1158084649 >> 172.21.15.112:0/1158084649 conn(0x55ed82bb5dc0 msgr2=0x55ed82bbc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:18.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.102+0000 7f07b67db740 1 -- 172.21.15.112:0/1158084649 shutdown_connections 2024-03-16T09:17:18.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.103+0000 7f07b67db740 1 -- 172.21.15.112:0/1158084649 wait complete. 2024-03-16T09:17:18.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.103+0000 7f07b67db740 1 -- start start 2024-03-16T09:17:18.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.103+0000 7f07b67db740 1 Processor -- start 2024-03-16T09:17:18.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.103+0000 7f07b67db740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 0x55ed82bc8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:18.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.103+0000 7f07b67db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bbb240 0x55ed82bc9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:18.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.103+0000 7f07b67db740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bc03b0 0x55ed82c34a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:18.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.103+0000 7f07b67db740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac1a5010 con 0x55ed82bba850 2024-03-16T09:17:18.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.103+0000 7f07b67db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac1a6480 con 0x55ed82bbb240 2024-03-16T09:17:18.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.103+0000 7f07b67db740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f07ac1a56e0 con 0x55ed82bc03b0 2024-03-16T09:17:18.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.103+0000 7f07b67db740 10 client.0 _ll_register_callbacks cb 0x7f079c015cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:18.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.103+0000 7f07b18ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 0x55ed82bc8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:18.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.103+0000 7f07b18ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 0x55ed82bc8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42838/0 (socket says 172.21.15.112:42838) 2024-03-16T09:17:18.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.103+0000 7f07b18ac640 1 -- 172.21.15.112:0/2019854907 learned_addr learned my addr 172.21.15.112:0/2019854907 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:18.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.103+0000 7f07b10ab640 1 --2- 172.21.15.112:0/2019854907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bbb240 0x55ed82bc9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:18.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.103+0000 7f07b20ad640 1 --2- 172.21.15.112:0/2019854907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bc03b0 0x55ed82c34a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:18.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07b18ac640 1 -- 172.21.15.112:0/2019854907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bc03b0 msgr2=0x55ed82c34a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:18.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07b18ac640 1 --2- 172.21.15.112:0/2019854907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bc03b0 0x55ed82c34a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07b18ac640 1 -- 172.21.15.112:0/2019854907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bbb240 msgr2=0x55ed82bc9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:18.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07b18ac640 1 --2- 172.21.15.112:0/2019854907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bbb240 0x55ed82bc9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07b18ac640 1 -- 172.21.15.112:0/2019854907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed82bcc2e0 con 0x55ed82bba850 2024-03-16T09:17:18.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07b10ab640 1 --2- 172.21.15.112:0/2019854907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bbb240 0x55ed82bc9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:18.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07b20ad640 1 --2- 172.21.15.112:0/2019854907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bc03b0 0x55ed82c34a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:18.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07b18ac640 1 --2- 172.21.15.112:0/2019854907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 0x55ed82bc8cd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f07a8005890 tx=0x7f07a8012050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82cc687be88fb7e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:18.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07957fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:18.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07957fa640 1 -- 172.21.15.112:0/2019854907 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8024040 con 0x55ed82bba850 2024-03-16T09:17:18.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07957fa640 1 -- 172.21.15.112:0/2019854907 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f07a8020260 con 0x55ed82bba850 2024-03-16T09:17:18.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07b67db740 1 -- 172.21.15.112:0/2019854907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a80047a0 con 0x55ed82bba850 2024-03-16T09:17:18.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07957fa640 1 -- 172.21.15.112:0/2019854907 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f07a8020460 con 0x55ed82bba850 2024-03-16T09:17:18.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07b67db740 10 client.15837 fetch_fsmap learned FSMap version 23 2024-03-16T09:17:18.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07b67db740 1 -- 172.21.15.112:0/2019854907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ed82c35f20 con 0x55ed82bba850 2024-03-16T09:17:18.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07957fa640 1 -- 172.21.15.112:0/2019854907 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f07a80047a0 con 0x55ed82bba850 2024-03-16T09:17:18.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07957fa640 1 -- 172.21.15.112:0/2019854907 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f07a8010800 con 0x55ed82bba850 2024-03-16T09:17:18.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07b67db740 10 client.15837 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T09:17:18.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07b67db740 10 client.15837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:18.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07b67db740 20 client.15837 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:18.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07b67db740 1 --2- 172.21.15.112:0/2019854907 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55ed82c369e0 0x55ed82c38e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:18.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07b67db740 4 client.15837 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T09:17:18.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.104+0000 7f07b67db740 1 -- 172.21.15.112:0/2019854907 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed82aa46a0 con 0x55ed82c369e0 2024-03-16T09:17:18.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.105+0000 7f07b10ab640 1 --2- 172.21.15.112:0/2019854907 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55ed82c369e0 0x55ed82c38e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:18.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.105+0000 7f07b10ab640 1 --2- 172.21.15.112:0/2019854907 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55ed82c369e0 0x55ed82c38e00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bc612384296471a server_cookie=892470944a269eff in_seq=0 out_seq=0 2024-03-16T09:17:18.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.105+0000 7f07957fa640 10 client.15837 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:18.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.105+0000 7f07957fa640 1 -- 172.21.15.112:0/2019854907 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed82aa46a0 con 0x55ed82c369e0 2024-03-16T09:17:18.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.105+0000 7f07957fa640 10 client.15837 handle_command_reply: tid=0 2024-03-16T09:17:18.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.105+0000 7f07b67db740 1 client.15837 shutdown 2024-03-16T09:17:18.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.105+0000 7f07b67db740 1 -- 172.21.15.112:0/2019854907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 msgr2=0x55ed82bc8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:18.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.105+0000 7f07b67db740 1 --2- 172.21.15.112:0/2019854907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 0x55ed82bc8cd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f07a8005890 tx=0x7f07a8012050 comp rx=0 tx=0).stop 2024-03-16T09:17:18.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.105+0000 7f07b67db740 1 -- 172.21.15.112:0/2019854907 shutdown_connections 2024-03-16T09:17:18.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.105+0000 7f07b67db740 1 --2- 172.21.15.112:0/2019854907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ed82bc03b0 0x55ed82c34a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.105+0000 7f07b67db740 1 --2- 172.21.15.112:0/2019854907 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55ed82c369e0 0x55ed82c38e00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.105+0000 7f07b67db740 1 --2- 172.21.15.112:0/2019854907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed82bbb240 0x55ed82bc9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.105+0000 7f07b67db740 1 --2- 172.21.15.112:0/2019854907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ed82bba850 0x55ed82bc8cd0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.105+0000 7f07957fa640 0 client.15837 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:18.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.105+0000 7f07b67db740 1 -- 172.21.15.112:0/2019854907 >> 172.21.15.112:0/2019854907 conn(0x55ed82bb5dc0 msgr2=0x55ed82bb9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:18.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.105+0000 7f07b67db740 1 -- 172.21.15.112:0/2019854907 shutdown_connections 2024-03-16T09:17:18.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.105+0000 7f07b67db740 1 -- 172.21.15.112:0/2019854907 wait complete. 2024-03-16T09:17:18.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.105+0000 7f07b67db740 20 client.15837 trim_cache size 0 max 16384 2024-03-16T09:17:18.115 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-03-16T09:17:18.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.106+0000 7f07b3b10640 1 -- 172.21.15.112:0/645839589 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f07980a2800 msgr2=0x7f07980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:18.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.106+0000 7f07b3b10640 1 --2- 172.21.15.112:0/645839589 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f07980a2800 0x7f07980a4cc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55ed82bbbaa0 tx=0x7f079c005f90 comp rx=0 tx=0).stop 2024-03-16T09:17:18.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.106+0000 7f07b3b10640 1 -- 172.21.15.112:0/645839589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07ac102290 msgr2=0x7f07ac19ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:18.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.106+0000 7f07b3b10640 1 --2- 172.21.15.112:0/645839589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07ac102290 0x7f07ac19ef20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f07a402c700 tx=0x7f07a40346d0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.106+0000 7f07b3b10640 1 -- 172.21.15.112:0/645839589 shutdown_connections 2024-03-16T09:17:18.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.106+0000 7f07b3b10640 1 --2- 172.21.15.112:0/645839589 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f07980a2800 0x7f07980a4cc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.106+0000 7f07b3b10640 1 --2- 172.21.15.112:0/645839589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07ac102290 0x7f07ac19ef20 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.106+0000 7f07b3b10640 1 --2- 172.21.15.112:0/645839589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07ac1018d0 0x7f07ac19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.106+0000 7f07b3b10640 1 --2- 172.21.15.112:0/645839589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07ac1006e0 0x7f07ac0694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:18.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.106+0000 7f07b3b10640 1 -- 172.21.15.112:0/645839589 >> 172.21.15.112:0/645839589 conn(0x7f07ac0fbc70 msgr2=0x7f07ac104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:18.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.106+0000 7f07b3b10640 1 -- 172.21.15.112:0/645839589 shutdown_connections 2024-03-16T09:17:18.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:18.106+0000 7f07b3b10640 1 -- 172.21.15.112:0/645839589 wait complete. 2024-03-16T09:17:18.227 INFO:tasks.workunit.client.0.smithi112.stderr:+ tar cf doc.tar linux-4.0.5 2024-03-16T09:17:18.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:17 smithi159 ceph-mon[36739]: pgmap v400: 97 pgs: 97 active+clean; 85 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 7.5 MiB/s wr, 165 op/s 2024-03-16T09:17:18.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:17 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3765221262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:17:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:17 smithi112 ceph-mon[32307]: pgmap v400: 97 pgs: 97 active+clean; 85 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 7.5 MiB/s wr, 165 op/s 2024-03-16T09:17:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:17 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3765221262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:17:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:17 smithi194 ceph-mon[36729]: pgmap v400: 97 pgs: 97 active+clean; 85 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 7.5 MiB/s wr, 165 op/s 2024-03-16T09:17:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:17 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3765221262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:17:19.070 INFO:tasks.workunit.client.0.smithi112.stderr:+ tar xf doc.tar 2024-03-16T09:17:19.120 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-16T09:17:19.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:17:19.120 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-16T09:17:19.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.233+0000 7fcbbcbcd640 1 -- 172.21.15.112:0/121230799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb8102a00 msgr2=0x7fcbb8102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:19.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.233+0000 7fcbbcbcd640 1 --2- 172.21.15.112:0/121230799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb8102a00 0x7fcbb8102e80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009900 tx=0x7fcba002bc30 comp rx=0 tx=0).stop 2024-03-16T09:17:19.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.233+0000 7fcbbcbcd640 1 -- 172.21.15.112:0/121230799 shutdown_connections 2024-03-16T09:17:19.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.233+0000 7fcbbcbcd640 1 --2- 172.21.15.112:0/121230799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbb81033c0 0x7fcbb8109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.233+0000 7fcbbcbcd640 1 --2- 172.21.15.112:0/121230799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb8102a00 0x7fcbb8102e80 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.233+0000 7fcbbcbcd640 1 --2- 172.21.15.112:0/121230799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbb8101810 0x7fcbb8101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.233+0000 7fcbbcbcd640 1 -- 172.21.15.112:0/121230799 >> 172.21.15.112:0/121230799 conn(0x7fcbb80fcd80 msgr2=0x7fcbb80ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:19.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.233+0000 7fcbbcbcd640 1 -- 172.21.15.112:0/121230799 shutdown_connections 2024-03-16T09:17:19.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.233+0000 7fcbbcbcd640 1 -- 172.21.15.112:0/121230799 wait complete. 2024-03-16T09:17:19.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbbcbcd640 1 Processor -- start 2024-03-16T09:17:19.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbbcbcd640 1 -- start start 2024-03-16T09:17:19.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbbcbcd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbb8101810 0x7fcbb8197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbbcbcd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbb8102a00 0x7fcbb8198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbbcbcd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb81033c0 0x7fcbb819f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbbcbcd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb810d570 con 0x7fcbb81033c0 2024-03-16T09:17:19.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbbcbcd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb810d3f0 con 0x7fcbb8101810 2024-03-16T09:17:19.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbbcbcd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb810d6f0 con 0x7fcbb8102a00 2024-03-16T09:17:19.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbb6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb81033c0 0x7fcbb819f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbb6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb81033c0 0x7fcbb819f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42870/0 (socket says 172.21.15.112:42870) 2024-03-16T09:17:19.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbb6d9d640 1 -- 172.21.15.112:0/917288345 learned_addr learned my addr 172.21.15.112:0/917288345 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:19.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbaffff640 1 --2- 172.21.15.112:0/917288345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbb8102a00 0x7fcbb8198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbb659c640 1 --2- 172.21.15.112:0/917288345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbb8101810 0x7fcbb8197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbb6d9d640 1 -- 172.21.15.112:0/917288345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbb8102a00 msgr2=0x7fcbb8198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:19.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbb6d9d640 1 --2- 172.21.15.112:0/917288345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbb8102a00 0x7fcbb8198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbb6d9d640 1 -- 172.21.15.112:0/917288345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbb8101810 msgr2=0x7fcbb8197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:19.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbb6d9d640 1 --2- 172.21.15.112:0/917288345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbb8101810 0x7fcbb8197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbb6d9d640 1 -- 172.21.15.112:0/917288345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb819f7f0 con 0x7fcbb81033c0 2024-03-16T09:17:19.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbaffff640 1 --2- 172.21.15.112:0/917288345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbb8102a00 0x7fcbb8198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:19.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbb6d9d640 1 --2- 172.21.15.112:0/917288345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb81033c0 0x7fcbb819f180 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000eb10 tx=0x7fcbb000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:19.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.234+0000 7fcbaf7fe640 1 -- 172.21.15.112:0/917288345 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0004b20 con 0x7fcbb81033c0 2024-03-16T09:17:19.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.235+0000 7fcbaf7fe640 1 -- 172.21.15.112:0/917288345 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcbb0004cc0 con 0x7fcbb81033c0 2024-03-16T09:17:19.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.235+0000 7fcbbcbcd640 1 -- 172.21.15.112:0/917288345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb819fae0 con 0x7fcbb81033c0 2024-03-16T09:17:19.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.235+0000 7fcbaf7fe640 1 -- 172.21.15.112:0/917288345 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb00107f0 con 0x7fcbb81033c0 2024-03-16T09:17:19.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.235+0000 7fcbbcbcd640 1 -- 172.21.15.112:0/917288345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb819ffb0 con 0x7fcbb81033c0 2024-03-16T09:17:19.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.237+0000 7fcbaf7fe640 1 -- 172.21.15.112:0/917288345 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7fcbb0005020 con 0x7fcbb81033c0 2024-03-16T09:17:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.237+0000 7fcbbf7b9740 1 Processor -- start 2024-03-16T09:17:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbbf7b9740 1 -- start start 2024-03-16T09:17:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbaf7fe640 1 --2- 172.21.15.112:0/917288345 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcba40a2800 0x7fcba40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbbf7b9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59a990 0x559bae59adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbbf7b9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae59b380 0x559bae59fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbbf7b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae5a04f0 0x559bae5a2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbbf7b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559bae4ff0d0 con 0x559bae5a04f0 2024-03-16T09:17:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbbf7b9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559bae4a7c70 con 0x559bae59b380 2024-03-16T09:17:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbbf7b9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559bae4fb170 con 0x559bae59a990 2024-03-16T09:17:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbaffff640 1 --2- 172.21.15.112:0/917288345 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcba40a2800 0x7fcba40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbaf7fe640 1 -- 172.21.15.112:0/917288345 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcbb00cefe0 con 0x7fcbb81033c0 2024-03-16T09:17:19.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbb659c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59a990 0x559bae59adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbb659c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59a990 0x559bae59adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:34596/0 (socket says 172.21.15.112:34596) 2024-03-16T09:17:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbb659c640 1 -- 172.21.15.112:0/2990952545 learned_addr learned my addr 172.21.15.112:0/2990952545 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbb6d9d640 1 --2- 172.21.15.112:0/2990952545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae5a04f0 0x559bae5a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbaffff640 1 --2- 172.21.15.112:0/2990952545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae59b380 0x559bae59fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbb659c640 1 -- 172.21.15.112:0/2990952545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae59b380 msgr2=0x559bae59fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbb659c640 1 --2- 172.21.15.112:0/2990952545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae59b380 0x559bae59fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbb659c640 1 -- 172.21.15.112:0/2990952545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae5a04f0 msgr2=0x559bae5a2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbb659c640 1 --2- 172.21.15.112:0/2990952545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae5a04f0 0x559bae5a2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbb659c640 1 -- 172.21.15.112:0/2990952545 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bae2db7e0 con 0x559bae59a990 2024-03-16T09:17:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbb659c640 1 --2- 172.21.15.112:0/2990952545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59a990 0x559bae59adb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009ca0 tx=0x7fcba8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df2797c3441e5e67 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.238+0000 7fcbaffff640 1 --2- 172.21.15.112:0/917288345 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcba40a2800 0x7fcba40a4cc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcba002c900 tx=0x7fcba0004040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.239+0000 7fcbacff9640 1 -- 172.21.15.112:0/2990952545 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8004220 con 0x559bae59a990 2024-03-16T09:17:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.239+0000 7fcbacff9640 1 -- 172.21.15.112:0/2990952545 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcba80043c0 con 0x559bae59a990 2024-03-16T09:17:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.239+0000 7fcbacff9640 1 -- 172.21.15.112:0/2990952545 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba80189b0 con 0x559bae59a990 2024-03-16T09:17:19.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.239+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/2990952545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59a990 msgr2=0x559bae59adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.239+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/2990952545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59a990 0x559bae59adb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009ca0 tx=0x7fcba8011810 comp rx=0 tx=0).stop 2024-03-16T09:17:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.239+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/2990952545 shutdown_connections 2024-03-16T09:17:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.239+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/2990952545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae5a04f0 0x559bae5a2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.239+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/2990952545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae59b380 0x559bae59fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.239+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/2990952545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59a990 0x559bae59adb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.239+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/2990952545 >> 172.21.15.112:0/2990952545 conn(0x559bae562c50 msgr2=0x559bae596300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.239+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/2990952545 shutdown_connections 2024-03-16T09:17:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.239+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/2990952545 wait complete. 2024-03-16T09:17:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.240+0000 7fcbbf7b9740 1 -- start start 2024-03-16T09:17:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.240+0000 7fcbbf7b9740 1 Processor -- start 2024-03-16T09:17:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.240+0000 7fcbbf7b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae59a990 0x559bae5a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.240+0000 7fcbbf7b9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59b380 0x559bae5a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.240+0000 7fcbbf7b9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae5a04f0 0x559bae5ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.240+0000 7fcbbf7b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559bae5a34a0 con 0x559bae59a990 2024-03-16T09:17:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.240+0000 7fcbbf7b9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559bae4bb0d0 con 0x559bae5a04f0 2024-03-16T09:17:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.240+0000 7fcbbf7b9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559bae5a3620 con 0x559bae59b380 2024-03-16T09:17:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.240+0000 7fcbbf7b9740 10 client.0 _ll_register_callbacks cb 0x559bae484760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.240+0000 7fcbb659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae59a990 0x559bae5a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.240+0000 7fcbb659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae59a990 0x559bae5a4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42890/0 (socket says 172.21.15.112:42890) 2024-03-16T09:17:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.240+0000 7fcbb659c640 1 -- 172.21.15.112:0/3801276132 learned_addr learned my addr 172.21.15.112:0/3801276132 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.240+0000 7fcbaffff640 1 --2- 172.21.15.112:0/3801276132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59b380 0x559bae5a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.240+0000 7fcbb6d9d640 1 --2- 172.21.15.112:0/3801276132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae5a04f0 0x559bae5ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.240+0000 7fcbb659c640 1 -- 172.21.15.112:0/3801276132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59b380 msgr2=0x559bae5a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.240+0000 7fcbb659c640 1 --2- 172.21.15.112:0/3801276132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59b380 0x559bae5a53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.241+0000 7fcbb659c640 1 -- 172.21.15.112:0/3801276132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae5a04f0 msgr2=0x559bae5ac3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.241+0000 7fcbb659c640 1 --2- 172.21.15.112:0/3801276132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae5a04f0 0x559bae5ac3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.241+0000 7fcbb659c640 1 -- 172.21.15.112:0/3801276132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bae33bf70 con 0x559bae59a990 2024-03-16T09:17:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.241+0000 7fcbb659c640 1 --2- 172.21.15.112:0/3801276132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae59a990 0x559bae5a4e70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcba80058c0 tx=0x7fcba80052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbeee19f90aef49 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:19.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.241+0000 7fcb9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:19.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.241+0000 7fcb9dffb640 1 -- 172.21.15.112:0/3801276132 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8027020 con 0x559bae59a990 2024-03-16T09:17:19.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.241+0000 7fcb9dffb640 1 -- 172.21.15.112:0/3801276132 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcba80035f0 con 0x559bae59a990 2024-03-16T09:17:19.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.241+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/3801276132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcba80043c0 con 0x559bae59a990 2024-03-16T09:17:19.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.241+0000 7fcb9dffb640 1 -- 172.21.15.112:0/3801276132 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fcba801e040 con 0x559bae59a990 2024-03-16T09:17:19.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.241+0000 7fcbbf7b9740 10 client.15849 fetch_fsmap learned FSMap version 23 2024-03-16T09:17:19.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.241+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/3801276132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x559bae1d3170 con 0x559bae59a990 2024-03-16T09:17:19.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.241+0000 7fcb9dffb640 1 -- 172.21.15.112:0/3801276132 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcba80043c0 con 0x559bae59a990 2024-03-16T09:17:19.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.241+0000 7fcb9dffb640 1 -- 172.21.15.112:0/3801276132 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fcba8004c30 con 0x559bae59a990 2024-03-16T09:17:19.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.241+0000 7fcbbf7b9740 10 client.15849 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T09:17:19.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.241+0000 7fcbbf7b9740 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:19.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.241+0000 7fcbbf7b9740 20 client.15849 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:19.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.241+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/3801276132 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x559bae5b0770 0x559bae5b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.241+0000 7fcbbf7b9740 4 client.15849 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:17:19.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.241+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/3801276132 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bae50c610 con 0x559bae5b0770 2024-03-16T09:17:19.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.242+0000 7fcbaffff640 1 --2- 172.21.15.112:0/3801276132 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x559bae5b0770 0x559bae5b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.242+0000 7fcb9dffb640 10 client.15849 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:19.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.242+0000 7fcbaffff640 1 --2- 172.21.15.112:0/3801276132 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x559bae5b0770 0x559bae5b2c10 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41286d49859d7d2e server_cookie=2f75da2706b3b841 in_seq=0 out_seq=0 2024-03-16T09:17:19.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.243+0000 7fcb9dffb640 1 -- 172.21.15.112:0/3801276132 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559bae50c610 con 0x559bae5b0770 2024-03-16T09:17:19.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.243+0000 7fcb9dffb640 10 client.15849 handle_command_reply: tid=0 2024-03-16T09:17:19.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.243+0000 7fcbbf7b9740 1 client.15849 shutdown 2024-03-16T09:17:19.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.243+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/3801276132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae59a990 msgr2=0x559bae5a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:19.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.243+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/3801276132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae59a990 0x559bae5a4e70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcba80058c0 tx=0x7fcba80052f0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.243+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/3801276132 shutdown_connections 2024-03-16T09:17:19.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.243+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/3801276132 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x559bae5b0770 0x559bae5b2c10 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.243+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/3801276132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae5a04f0 0x559bae5ac3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.243+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/3801276132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59b380 0x559bae5a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.243+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/3801276132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae59a990 0x559bae5a4e70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.243+0000 7fcb9dffb640 0 client.15849 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:19.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.243+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/3801276132 >> 172.21.15.112:0/3801276132 conn(0x559bae562c50 msgr2=0x559bae5954e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:19.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.243+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/3801276132 shutdown_connections 2024-03-16T09:17:19.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.243+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/3801276132 wait complete. 2024-03-16T09:17:19.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.243+0000 7fcbbf7b9740 20 client.15849 trim_cache size 0 max 16384 2024-03-16T09:17:19.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.260+0000 7fcbbf7b9740 1 Processor -- start 2024-03-16T09:17:19.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.260+0000 7fcbbf7b9740 1 -- start start 2024-03-16T09:17:19.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.260+0000 7fcbbf7b9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae59a990 0x559bae599c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.260+0000 7fcbbf7b9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59b380 0x559bae59a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.260+0000 7fcbbf7b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae5a04f0 0x559bae5ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.260+0000 7fcbbf7b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb81ab680 con 0x559bae5a04f0 2024-03-16T09:17:19.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.260+0000 7fcbbf7b9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb81a8f50 con 0x559bae59a990 2024-03-16T09:17:19.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.260+0000 7fcbbf7b9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb81a6da0 con 0x559bae59b380 2024-03-16T09:17:19.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.261+0000 7fcbb6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae5a04f0 0x559bae5ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.261+0000 7fcbb6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae5a04f0 0x559bae5ac3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42894/0 (socket says 172.21.15.112:42894) 2024-03-16T09:17:19.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.261+0000 7fcbb6d9d640 1 -- 172.21.15.112:0/344652043 learned_addr learned my addr 172.21.15.112:0/344652043 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:19.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.261+0000 7fcbb659c640 1 --2- 172.21.15.112:0/344652043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae59a990 0x559bae599c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.261+0000 7fcbaffff640 1 --2- 172.21.15.112:0/344652043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59b380 0x559bae59a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.261+0000 7fcbb6d9d640 1 -- 172.21.15.112:0/344652043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59b380 msgr2=0x559bae59a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:19.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.261+0000 7fcbb6d9d640 1 --2- 172.21.15.112:0/344652043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59b380 0x559bae59a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.261+0000 7fcbb6d9d640 1 -- 172.21.15.112:0/344652043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae59a990 msgr2=0x559bae599c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:19.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.261+0000 7fcbb6d9d640 1 --2- 172.21.15.112:0/344652043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae59a990 0x559bae599c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.261+0000 7fcbb6d9d640 1 -- 172.21.15.112:0/344652043 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bae5b0550 con 0x559bae5a04f0 2024-03-16T09:17:19.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.261+0000 7fcbb6d9d640 1 --2- 172.21.15.112:0/344652043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae5a04f0 0x559bae5ac3b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0097df0 tx=0x7fcbb0086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d63ce89d194d2d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:19.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.261+0000 7fcb9ffff640 1 -- 172.21.15.112:0/344652043 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb009b990 con 0x559bae5a04f0 2024-03-16T09:17:19.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.261+0000 7fcb9ffff640 1 -- 172.21.15.112:0/344652043 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcbb009fae0 con 0x559bae5a04f0 2024-03-16T09:17:19.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.261+0000 7fcb9ffff640 1 -- 172.21.15.112:0/344652043 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb00860b0 con 0x559bae5a04f0 2024-03-16T09:17:19.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.262+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/344652043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae5a04f0 msgr2=0x559bae5ac3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:19.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.262+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/344652043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae5a04f0 0x559bae5ac3b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0097df0 tx=0x7fcbb0086040 comp rx=0 tx=0).stop 2024-03-16T09:17:19.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.262+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/344652043 shutdown_connections 2024-03-16T09:17:19.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.262+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/344652043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae5a04f0 0x559bae5ac3b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.262+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/344652043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59b380 0x559bae59a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.262+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/344652043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae59a990 0x559bae599c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.262+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/344652043 >> 172.21.15.112:0/344652043 conn(0x559bae562c50 msgr2=0x559bae599540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:19.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.262+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/344652043 shutdown_connections 2024-03-16T09:17:19.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.262+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/344652043 wait complete. 2024-03-16T09:17:19.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.262+0000 7fcbbf7b9740 1 -- start start 2024-03-16T09:17:19.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.262+0000 7fcbbf7b9740 1 Processor -- start 2024-03-16T09:17:19.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.262+0000 7fcbbf7b9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59a990 0x559bae5a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.262+0000 7fcbbf7b9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae59b380 0x559bae5aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.262+0000 7fcbbf7b9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae5a04f0 0x559bae5ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.262+0000 7fcbbf7b9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb81a7410 con 0x559bae59b380 2024-03-16T09:17:19.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.262+0000 7fcbbf7b9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb81a8880 con 0x559bae5a04f0 2024-03-16T09:17:19.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.262+0000 7fcbbf7b9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb81a7ae0 con 0x559bae59a990 2024-03-16T09:17:19.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.262+0000 7fcbbf7b9740 10 client.0 _ll_register_callbacks cb 0x7fcbb0004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:19.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.263+0000 7fcbaffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae59b380 0x559bae5aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.263+0000 7fcbb659c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59a990 0x559bae5a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.263+0000 7fcbaffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae59b380 0x559bae5aa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42910/0 (socket says 172.21.15.112:42910) 2024-03-16T09:17:19.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.263+0000 7fcbaffff640 1 -- 172.21.15.112:0/3140822639 learned_addr learned my addr 172.21.15.112:0/3140822639 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:19.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.263+0000 7fcbb6d9d640 1 --2- 172.21.15.112:0/3140822639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae5a04f0 0x559bae5ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.263+0000 7fcbaffff640 1 -- 172.21.15.112:0/3140822639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59a990 msgr2=0x559bae5a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:19.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.263+0000 7fcbaffff640 1 --2- 172.21.15.112:0/3140822639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59a990 0x559bae5a9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.263+0000 7fcbaffff640 1 -- 172.21.15.112:0/3140822639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae5a04f0 msgr2=0x559bae5ab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:19.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.263+0000 7fcbaffff640 1 --2- 172.21.15.112:0/3140822639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae5a04f0 0x559bae5ab4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.263+0000 7fcbaffff640 1 -- 172.21.15.112:0/3140822639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bae5abae0 con 0x559bae59b380 2024-03-16T09:17:19.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.263+0000 7fcbaffff640 1 --2- 172.21.15.112:0/3140822639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae59b380 0x559bae5aa4e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcba0036040 tx=0x7fcba002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8192eb7ba2fa94a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:19.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.263+0000 7fcb9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:19.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.263+0000 7fcb9dffb640 1 -- 172.21.15.112:0/3140822639 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0049070 con 0x559bae59b380 2024-03-16T09:17:19.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.263+0000 7fcb9dffb640 1 -- 172.21.15.112:0/3140822639 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7fcba0053340 con 0x559bae59b380 2024-03-16T09:17:19.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.263+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/3140822639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbb009fae0 con 0x559bae59b380 2024-03-16T09:17:19.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.263+0000 7fcb9dffb640 1 -- 172.21.15.112:0/3140822639 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fcba0053540 con 0x559bae59b380 2024-03-16T09:17:19.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.263+0000 7fcbbf7b9740 10 client.15855 fetch_fsmap learned FSMap version 23 2024-03-16T09:17:19.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.263+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/3140822639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x559bae617060 con 0x559bae59b380 2024-03-16T09:17:19.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.264+0000 7fcb9dffb640 1 -- 172.21.15.112:0/3140822639 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcbb009fae0 con 0x559bae59b380 2024-03-16T09:17:19.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.264+0000 7fcb9dffb640 1 -- 172.21.15.112:0/3140822639 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fcba0040040 con 0x559bae59b380 2024-03-16T09:17:19.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.264+0000 7fcbbf7b9740 10 client.15855 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T09:17:19.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.264+0000 7fcbbf7b9740 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:19.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.264+0000 7fcbbf7b9740 20 client.15855 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:19.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.264+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/3140822639 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x559bae617aa0 0x559bae619ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.264+0000 7fcbbf7b9740 4 client.15855 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-16T09:17:19.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.264+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/3140822639 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559bae506960 con 0x559bae617aa0 2024-03-16T09:17:19.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.264+0000 7fcbb659c640 1 --2- 172.21.15.112:0/3140822639 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x559bae617aa0 0x559bae619ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.264+0000 7fcb9dffb640 10 client.15855 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:19.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.264+0000 7fcbb659c640 1 --2- 172.21.15.112:0/3140822639 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x559bae617aa0 0x559bae619ec0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ab2cd9f16e9202e server_cookie=3fed4a1967833cbe in_seq=0 out_seq=0 2024-03-16T09:17:19.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.265+0000 7fcb9dffb640 1 -- 172.21.15.112:0/3140822639 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559bae506960 con 0x559bae617aa0 2024-03-16T09:17:19.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.265+0000 7fcb9dffb640 10 client.15855 handle_command_reply: tid=0 2024-03-16T09:17:19.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.265+0000 7fcbbf7b9740 1 client.15855 shutdown 2024-03-16T09:17:19.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.265+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/3140822639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae59b380 msgr2=0x559bae5aa4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:19.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.265+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/3140822639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae59b380 0x559bae5aa4e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcba0036040 tx=0x7fcba002bc30 comp rx=0 tx=0).stop 2024-03-16T09:17:19.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.265+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/3140822639 shutdown_connections 2024-03-16T09:17:19.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.265+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/3140822639 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x559bae617aa0 0x559bae619ec0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.265+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/3140822639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bae5a04f0 0x559bae5ab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.265+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/3140822639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559bae59b380 0x559bae5aa4e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.265+0000 7fcbbf7b9740 1 --2- 172.21.15.112:0/3140822639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559bae59a990 0x559bae5a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.265+0000 7fcb9dffb640 0 client.15855 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:19.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.265+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/3140822639 >> 172.21.15.112:0/3140822639 conn(0x559bae562c50 msgr2=0x559bae599dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:19.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.265+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/3140822639 shutdown_connections 2024-03-16T09:17:19.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.265+0000 7fcbbf7b9740 1 -- 172.21.15.112:0/3140822639 wait complete. 2024-03-16T09:17:19.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.265+0000 7fcbbf7b9740 20 client.15855 trim_cache size 0 max 16384 2024-03-16T09:17:19.274 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:17:19.274 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-03-16T09:17:19.274 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "3f778942-1662-48df-ad81-6d9cdf602b1e", 2024-03-16T09:17:19.275 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-03-16T09:17:19.275 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:17:19.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.266+0000 7fcbbcbcd640 1 -- 172.21.15.112:0/917288345 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcba40a2800 msgr2=0x7fcba40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:19.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.266+0000 7fcbbcbcd640 1 --2- 172.21.15.112:0/917288345 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcba40a2800 0x7fcba40a4cc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcba002c900 tx=0x7fcba0004040 comp rx=0 tx=0).stop 2024-03-16T09:17:19.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.266+0000 7fcbbcbcd640 1 -- 172.21.15.112:0/917288345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb81033c0 msgr2=0x7fcbb819f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:19.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.266+0000 7fcbbcbcd640 1 --2- 172.21.15.112:0/917288345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb81033c0 0x7fcbb819f180 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000eb10 tx=0x7fcbb000c2d0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.266+0000 7fcbbcbcd640 1 -- 172.21.15.112:0/917288345 shutdown_connections 2024-03-16T09:17:19.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.266+0000 7fcbbcbcd640 1 --2- 172.21.15.112:0/917288345 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fcba40a2800 0x7fcba40a4cc0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.266+0000 7fcbbcbcd640 1 --2- 172.21.15.112:0/917288345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcbb81033c0 0x7fcbb819f180 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.266+0000 7fcbbcbcd640 1 --2- 172.21.15.112:0/917288345 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbb8102a00 0x7fcbb8198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.266+0000 7fcbbcbcd640 1 --2- 172.21.15.112:0/917288345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbb8101810 0x7fcbb8197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.266+0000 7fcbbcbcd640 1 -- 172.21.15.112:0/917288345 >> 172.21.15.112:0/917288345 conn(0x7fcbb80fcd80 msgr2=0x7fcbb8105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:19.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.266+0000 7fcbbcbcd640 1 -- 172.21.15.112:0/917288345 shutdown_connections 2024-03-16T09:17:19.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.266+0000 7fcbbcbcd640 1 -- 172.21.15.112:0/917288345 wait complete. 2024-03-16T09:17:19.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T09:17:19.279 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T09:17:19.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.385+0000 7feb25b5b640 1 -- 172.21.15.112:0/1845111526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb20102290 msgr2=0x7feb20108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:19.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.385+0000 7feb25b5b640 1 --2- 172.21.15.112:0/1845111526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb20102290 0x7feb20108b20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7feb18009ca0 tx=0x7feb1802c0f0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.385+0000 7feb25b5b640 1 -- 172.21.15.112:0/1845111526 shutdown_connections 2024-03-16T09:17:19.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.385+0000 7feb25b5b640 1 --2- 172.21.15.112:0/1845111526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb20102290 0x7feb20108b20 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.385+0000 7feb25b5b640 1 --2- 172.21.15.112:0/1845111526 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb201018d0 0x7feb20101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.385+0000 7feb25b5b640 1 --2- 172.21.15.112:0/1845111526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb201006e0 0x7feb20100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.385+0000 7feb25b5b640 1 -- 172.21.15.112:0/1845111526 >> 172.21.15.112:0/1845111526 conn(0x7feb200fbc70 msgr2=0x7feb200fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:19.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.385+0000 7feb25b5b640 1 -- 172.21.15.112:0/1845111526 shutdown_connections 2024-03-16T09:17:19.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.385+0000 7feb25b5b640 1 -- 172.21.15.112:0/1845111526 wait complete. 2024-03-16T09:17:19.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.385+0000 7feb25b5b640 1 Processor -- start 2024-03-16T09:17:19.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.386+0000 7feb25b5b640 1 -- start start 2024-03-16T09:17:19.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.386+0000 7feb25b5b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb201006e0 0x7feb2010b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.386+0000 7feb25b5b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb201018d0 0x7feb201096a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.386+0000 7feb25b5b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb20102290 0x7feb20109be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.386+0000 7feb25b5b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb2010d5a0 con 0x7feb201018d0 2024-03-16T09:17:19.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.386+0000 7feb25b5b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feb2010d420 con 0x7feb201006e0 2024-03-16T09:17:19.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.386+0000 7feb25b5b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feb2010d720 con 0x7feb20102290 2024-03-16T09:17:19.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.386+0000 7feb1effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb201018d0 0x7feb201096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.386+0000 7feb1f7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb201006e0 0x7feb2010b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.386+0000 7feb1effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb201018d0 0x7feb201096a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42924/0 (socket says 172.21.15.112:42924) 2024-03-16T09:17:19.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.386+0000 7feb1effd640 1 -- 172.21.15.112:0/1144493267 learned_addr learned my addr 172.21.15.112:0/1144493267 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:19.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.386+0000 7feb1ffff640 1 --2- 172.21.15.112:0/1144493267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb20102290 0x7feb20109be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.386+0000 7feb1effd640 1 -- 172.21.15.112:0/1144493267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb20102290 msgr2=0x7feb20109be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:19.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.386+0000 7feb1effd640 1 --2- 172.21.15.112:0/1144493267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb20102290 0x7feb20109be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.386+0000 7feb1effd640 1 -- 172.21.15.112:0/1144493267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb201006e0 msgr2=0x7feb2010b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:19.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.386+0000 7feb1effd640 1 --2- 172.21.15.112:0/1144493267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb201006e0 0x7feb2010b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.386+0000 7feb1effd640 1 -- 172.21.15.112:0/1144493267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2010a470 con 0x7feb201018d0 2024-03-16T09:17:19.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.386+0000 7feb1f7fe640 1 --2- 172.21.15.112:0/1144493267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb201006e0 0x7feb2010b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:19.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.386+0000 7feb1effd640 1 --2- 172.21.15.112:0/1144493267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb201018d0 0x7feb201096a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7feb1400e9f0 tx=0x7feb1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:19.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.387+0000 7feb1cff9640 1 -- 172.21.15.112:0/1144493267 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1400cd80 con 0x7feb201018d0 2024-03-16T09:17:19.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.387+0000 7feb25b5b640 1 -- 172.21.15.112:0/1144493267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb201a1cd0 con 0x7feb201018d0 2024-03-16T09:17:19.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.387+0000 7feb1cff9640 1 -- 172.21.15.112:0/1144493267 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feb14005520 con 0x7feb201018d0 2024-03-16T09:17:19.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.387+0000 7feb1cff9640 1 -- 172.21.15.112:0/1144493267 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14010690 con 0x7feb201018d0 2024-03-16T09:17:19.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.387+0000 7feb25b5b640 1 -- 172.21.15.112:0/1144493267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb201a21d0 con 0x7feb201018d0 2024-03-16T09:17:19.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.389+0000 7feb28826740 1 Processor -- start 2024-03-16T09:17:19.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb1cff9640 1 -- 172.21.15.112:0/1144493267 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7feb140046c0 con 0x7feb201018d0 2024-03-16T09:17:19.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb28826740 1 -- start start 2024-03-16T09:17:19.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb28826740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabf9c170 0x555dabf9c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb28826740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabf9cb60 0x555dabfa1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb28826740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabfa1cd0 0x555dabfa4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb28826740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555dabf01500 con 0x555dabfa1cd0 2024-03-16T09:17:19.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb28826740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555dabea9be0 con 0x555dabf9c170 2024-03-16T09:17:19.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb28826740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555dabefd5a0 con 0x555dabf9cb60 2024-03-16T09:17:19.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb1effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabfa1cd0 0x555dabfa4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb1effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabfa1cd0 0x555dabfa4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42936/0 (socket says 172.21.15.112:42936) 2024-03-16T09:17:19.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb1ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabf9c170 0x555dabf9c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb1effd640 1 -- 172.21.15.112:0/4159302629 learned_addr learned my addr 172.21.15.112:0/4159302629 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:19.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb1f7fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabf9cb60 0x555dabfa1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb1effd640 1 -- 172.21.15.112:0/4159302629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabf9cb60 msgr2=0x555dabfa1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:19.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb1effd640 1 --2- 172.21.15.112:0/4159302629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabf9cb60 0x555dabfa1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb1effd640 1 -- 172.21.15.112:0/4159302629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabf9c170 msgr2=0x555dabf9c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:19.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb1effd640 1 --2- 172.21.15.112:0/4159302629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabf9c170 0x555dabf9c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb1effd640 1 -- 172.21.15.112:0/4159302629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dabebd4b0 con 0x555dabfa1cd0 2024-03-16T09:17:19.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb1f7fe640 1 --2- 172.21.15.112:0/4159302629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabf9cb60 0x555dabfa1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:19.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb1cff9640 1 --2- 172.21.15.112:0/1144493267 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feb080a2800 0x7feb080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb1effd640 1 --2- 172.21.15.112:0/4159302629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabfa1cd0 0x555dabfa4160 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7feb140d2dc0 tx=0x7feb140d2eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98aa1d4fae2ed144 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:19.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.390+0000 7feb1ffff640 1 --2- 172.21.15.112:0/1144493267 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feb080a2800 0x7feb080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.391+0000 7feb1cff9640 1 -- 172.21.15.112:0/1144493267 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7feb140cee40 con 0x7feb201018d0 2024-03-16T09:17:19.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.391+0000 7feb05ffb640 1 -- 172.21.15.112:0/4159302629 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14097a90 con 0x555dabfa1cd0 2024-03-16T09:17:19.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.391+0000 7feb05ffb640 1 -- 172.21.15.112:0/4159302629 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feb1409bef0 con 0x555dabfa1cd0 2024-03-16T09:17:19.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.391+0000 7feb05ffb640 1 -- 172.21.15.112:0/4159302629 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb140bba70 con 0x555dabfa1cd0 2024-03-16T09:17:19.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.391+0000 7feb28826740 1 -- 172.21.15.112:0/4159302629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabfa1cd0 msgr2=0x555dabfa4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:19.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.391+0000 7feb1ffff640 1 --2- 172.21.15.112:0/1144493267 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feb080a2800 0x7feb080a4cc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feb18005e30 tx=0x7feb18032670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:19.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.391+0000 7feb28826740 1 --2- 172.21.15.112:0/4159302629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabfa1cd0 0x555dabfa4160 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7feb140d2dc0 tx=0x7feb140d2eb0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.391+0000 7feb28826740 1 -- 172.21.15.112:0/4159302629 shutdown_connections 2024-03-16T09:17:19.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.391+0000 7feb28826740 1 --2- 172.21.15.112:0/4159302629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabfa1cd0 0x555dabfa4160 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.391+0000 7feb28826740 1 --2- 172.21.15.112:0/4159302629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabf9cb60 0x555dabfa1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.391+0000 7feb28826740 1 --2- 172.21.15.112:0/4159302629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabf9c170 0x555dabf9c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.391+0000 7feb28826740 1 -- 172.21.15.112:0/4159302629 >> 172.21.15.112:0/4159302629 conn(0x555dabf976e0 msgr2=0x555dabf97ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:19.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.391+0000 7feb28826740 1 -- 172.21.15.112:0/4159302629 shutdown_connections 2024-03-16T09:17:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.391+0000 7feb28826740 1 -- 172.21.15.112:0/4159302629 wait complete. 2024-03-16T09:17:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.392+0000 7feb28826740 1 -- start start 2024-03-16T09:17:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.392+0000 7feb28826740 1 Processor -- start 2024-03-16T09:17:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.392+0000 7feb28826740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabf9c170 0x555dabfa66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.392+0000 7feb28826740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9cb60 0x555dabfa6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.392+0000 7feb28826740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabfa1cd0 0x555dabfadc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.392+0000 7feb28826740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555dabfa4c80 con 0x555dabf9cb60 2024-03-16T09:17:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.392+0000 7feb28826740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555dabebd160 con 0x555dabfa1cd0 2024-03-16T09:17:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.392+0000 7feb28826740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555dabfa4e00 con 0x555dabf9c170 2024-03-16T09:17:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.392+0000 7feb28826740 10 client.0 _ll_register_callbacks cb 0x555dabea1150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.392+0000 7feb1effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9cb60 0x555dabfa6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.392+0000 7feb1effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9cb60 0x555dabfa6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42946/0 (socket says 172.21.15.112:42946) 2024-03-16T09:17:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.392+0000 7feb1effd640 1 -- 172.21.15.112:0/2297318880 learned_addr learned my addr 172.21.15.112:0/2297318880 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:19.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.392+0000 7feb1f7fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabf9c170 0x555dabfa66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.392+0000 7feb1ffff640 1 --2- 172.21.15.112:0/2297318880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabfa1cd0 0x555dabfadc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.393+0000 7feb1effd640 1 -- 172.21.15.112:0/2297318880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabf9c170 msgr2=0x555dabfa66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:19.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.393+0000 7feb1effd640 1 --2- 172.21.15.112:0/2297318880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabf9c170 0x555dabfa66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.393+0000 7feb1effd640 1 -- 172.21.15.112:0/2297318880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabfa1cd0 msgr2=0x555dabfadc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:19.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.393+0000 7feb1effd640 1 --2- 172.21.15.112:0/2297318880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabfa1cd0 0x555dabfadc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.393+0000 7feb1effd640 1 -- 172.21.15.112:0/2297318880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dabcdf7d0 con 0x555dabf9cb60 2024-03-16T09:17:19.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.393+0000 7feb1f7fe640 1 --2- 172.21.15.112:0/2297318880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabf9c170 0x555dabfa66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:19.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.393+0000 7feb1effd640 1 --2- 172.21.15.112:0/2297318880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9cb60 0x555dabfa6bf0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7feb14004ab0 tx=0x7feb1400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a193295f02dcaf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:19.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.393+0000 7feafeffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:19.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.393+0000 7feafeffd640 1 -- 172.21.15.112:0/2297318880 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb140b4060 con 0x555dabf9cb60 2024-03-16T09:17:19.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.393+0000 7feafeffd640 1 -- 172.21.15.112:0/2297318880 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7feb1409ba50 con 0x555dabf9cb60 2024-03-16T09:17:19.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.393+0000 7feb28826740 1 -- 172.21.15.112:0/2297318880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb1409bef0 con 0x555dabf9cb60 2024-03-16T09:17:19.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.393+0000 7feafeffd640 1 -- 172.21.15.112:0/2297318880 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7feb1409b150 con 0x555dabf9cb60 2024-03-16T09:17:19.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.393+0000 7feb28826740 10 client.15867 fetch_fsmap learned FSMap version 23 2024-03-16T09:17:19.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.393+0000 7feb28826740 1 -- 172.21.15.112:0/2297318880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x555dabc153f0 con 0x555dabf9cb60 2024-03-16T09:17:19.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.393+0000 7feafeffd640 1 -- 172.21.15.112:0/2297318880 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feb1409bef0 con 0x555dabf9cb60 2024-03-16T09:17:19.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.394+0000 7feafeffd640 1 -- 172.21.15.112:0/2297318880 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7feb140d0070 con 0x555dabf9cb60 2024-03-16T09:17:19.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.394+0000 7feb28826740 10 client.15867 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T09:17:19.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.394+0000 7feb28826740 10 client.15867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:19.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.394+0000 7feb28826740 20 client.15867 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:19.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.394+0000 7feb28826740 1 --2- 172.21.15.112:0/2297318880 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x555dabfb1d90 0x555dabfb4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.394+0000 7feb28826740 4 client.15867 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:17:19.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.394+0000 7feb28826740 1 -- 172.21.15.112:0/2297318880 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555dabf0ea40 con 0x555dabfb1d90 2024-03-16T09:17:19.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.394+0000 7feb1f7fe640 1 --2- 172.21.15.112:0/2297318880 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x555dabfb1d90 0x555dabfb4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.394+0000 7feb1f7fe640 1 --2- 172.21.15.112:0/2297318880 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x555dabfb1d90 0x555dabfb4180 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c63261df12a8c340 server_cookie=772cd19a623868a6 in_seq=0 out_seq=0 2024-03-16T09:17:19.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.394+0000 7feafeffd640 10 client.15867 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:19.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.395+0000 7feafeffd640 1 -- 172.21.15.112:0/2297318880 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555dabf0ea40 con 0x555dabfb1d90 2024-03-16T09:17:19.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.395+0000 7feafeffd640 10 client.15867 handle_command_reply: tid=0 2024-03-16T09:17:19.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.395+0000 7feb28826740 1 client.15867 shutdown 2024-03-16T09:17:19.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.395+0000 7feb28826740 1 -- 172.21.15.112:0/2297318880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9cb60 msgr2=0x555dabfa6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:19.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.395+0000 7feb28826740 1 --2- 172.21.15.112:0/2297318880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9cb60 0x555dabfa6bf0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7feb14004ab0 tx=0x7feb1400cf20 comp rx=0 tx=0).stop 2024-03-16T09:17:19.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.395+0000 7feb28826740 1 -- 172.21.15.112:0/2297318880 shutdown_connections 2024-03-16T09:17:19.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.395+0000 7feb28826740 1 --2- 172.21.15.112:0/2297318880 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x555dabfb1d90 0x555dabfb4180 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.395+0000 7feb28826740 1 --2- 172.21.15.112:0/2297318880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabfa1cd0 0x555dabfadc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.395+0000 7feb28826740 1 --2- 172.21.15.112:0/2297318880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9cb60 0x555dabfa6bf0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.395+0000 7feb28826740 1 --2- 172.21.15.112:0/2297318880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabf9c170 0x555dabfa66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.395+0000 7feb28826740 1 -- 172.21.15.112:0/2297318880 >> 172.21.15.112:0/2297318880 conn(0x555dabf976e0 msgr2=0x555dabf96900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:19.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.395+0000 7feafeffd640 0 client.15867 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:19.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.396+0000 7feb28826740 1 -- 172.21.15.112:0/2297318880 shutdown_connections 2024-03-16T09:17:19.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.396+0000 7feb28826740 1 -- 172.21.15.112:0/2297318880 wait complete. 2024-03-16T09:17:19.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.396+0000 7feb28826740 20 client.15867 trim_cache size 0 max 16384 2024-03-16T09:17:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.413+0000 7feb28826740 1 Processor -- start 2024-03-16T09:17:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.413+0000 7feb28826740 1 -- start start 2024-03-16T09:17:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.413+0000 7feb28826740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9c170 0x555dabf928e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb28826740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabf9cb60 0x555dabfadbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb28826740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabfa1cd0 0x555dabf96070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb28826740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb201ad890 con 0x555dabf9c170 2024-03-16T09:17:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb28826740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feb201ab160 con 0x555dabf9cb60 2024-03-16T09:17:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb28826740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feb201a8fb0 con 0x555dabfa1cd0 2024-03-16T09:17:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb1f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9c170 0x555dabf928e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb1f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9c170 0x555dabf928e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42950/0 (socket says 172.21.15.112:42950) 2024-03-16T09:17:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb1f7fe640 1 -- 172.21.15.112:0/992760868 learned_addr learned my addr 172.21.15.112:0/992760868 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb1effd640 1 --2- 172.21.15.112:0/992760868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabf9cb60 0x555dabfadbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb1ffff640 1 --2- 172.21.15.112:0/992760868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabfa1cd0 0x555dabf96070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb1f7fe640 1 -- 172.21.15.112:0/992760868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabfa1cd0 msgr2=0x555dabf96070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb1f7fe640 1 --2- 172.21.15.112:0/992760868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabfa1cd0 0x555dabf96070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb1f7fe640 1 -- 172.21.15.112:0/992760868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabf9cb60 msgr2=0x555dabfadbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb1f7fe640 1 --2- 172.21.15.112:0/992760868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabf9cb60 0x555dabfadbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb1f7fe640 1 -- 172.21.15.112:0/992760868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dabbd50f0 con 0x555dabf9c170 2024-03-16T09:17:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb1ffff640 1 --2- 172.21.15.112:0/992760868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabfa1cd0 0x555dabf96070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:17:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb1f7fe640 1 --2- 172.21.15.112:0/992760868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9c170 0x555dabf928e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x555dabf9d5d0 tx=0x7feb1000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57644a55952fd9b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb057fa640 1 -- 172.21.15.112:0/992760868 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10005220 con 0x555dabf9c170 2024-03-16T09:17:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb057fa640 1 -- 172.21.15.112:0/992760868 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feb10024030 con 0x555dabf9c170 2024-03-16T09:17:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.414+0000 7feb057fa640 1 -- 172.21.15.112:0/992760868 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10020520 con 0x555dabf9c170 2024-03-16T09:17:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.415+0000 7feb28826740 1 -- 172.21.15.112:0/992760868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9c170 msgr2=0x555dabf928e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.415+0000 7feb28826740 1 --2- 172.21.15.112:0/992760868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9c170 0x555dabf928e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x555dabf9d5d0 tx=0x7feb1000b410 comp rx=0 tx=0).stop 2024-03-16T09:17:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.415+0000 7feb28826740 1 -- 172.21.15.112:0/992760868 shutdown_connections 2024-03-16T09:17:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.415+0000 7feb28826740 1 --2- 172.21.15.112:0/992760868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabfa1cd0 0x555dabf96070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.415+0000 7feb28826740 1 --2- 172.21.15.112:0/992760868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabf9cb60 0x555dabfadbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.415+0000 7feb28826740 1 --2- 172.21.15.112:0/992760868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9c170 0x555dabf928e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.415+0000 7feb28826740 1 -- 172.21.15.112:0/992760868 >> 172.21.15.112:0/992760868 conn(0x555dabf976e0 msgr2=0x555dabf9e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.415+0000 7feb28826740 1 -- 172.21.15.112:0/992760868 shutdown_connections 2024-03-16T09:17:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.415+0000 7feb28826740 1 -- 172.21.15.112:0/992760868 wait complete. 2024-03-16T09:17:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.415+0000 7feb28826740 1 -- start start 2024-03-16T09:17:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb28826740 1 Processor -- start 2024-03-16T09:17:19.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb28826740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9c170 0x555dabfab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb28826740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabf9cb60 0x555dabfab620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb28826740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabfa1cd0 0x555dabfac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb28826740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb201a9620 con 0x555dabf9c170 2024-03-16T09:17:19.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb28826740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feb201aaa90 con 0x555dabfa1cd0 2024-03-16T09:17:19.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb28826740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feb201a9cf0 con 0x555dabf9cb60 2024-03-16T09:17:19.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb28826740 10 client.0 _ll_register_callbacks cb 0x7feb14005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:19.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb1f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9c170 0x555dabfab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb1f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9c170 0x555dabfab0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42954/0 (socket says 172.21.15.112:42954) 2024-03-16T09:17:19.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb1f7fe640 1 -- 172.21.15.112:0/2038576908 learned_addr learned my addr 172.21.15.112:0/2038576908 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:19.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb1effd640 1 --2- 172.21.15.112:0/2038576908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabf9cb60 0x555dabfab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb1ffff640 1 --2- 172.21.15.112:0/2038576908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabfa1cd0 0x555dabfac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb1f7fe640 1 -- 172.21.15.112:0/2038576908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabf9cb60 msgr2=0x555dabfab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:19.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb1f7fe640 1 --2- 172.21.15.112:0/2038576908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabf9cb60 0x555dabfab620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb1f7fe640 1 -- 172.21.15.112:0/2038576908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabfa1cd0 msgr2=0x555dabfac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:19.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb1f7fe640 1 --2- 172.21.15.112:0/2038576908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabfa1cd0 0x555dabfac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb1f7fe640 1 -- 172.21.15.112:0/2038576908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dabec89d0 con 0x555dabf9c170 2024-03-16T09:17:19.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb1f7fe640 1 --2- 172.21.15.112:0/2038576908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9c170 0x555dabfab0e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7feb10023040 tx=0x7feb10005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70e4960b9b1a898 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:19.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feafeffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:19.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feafeffd640 1 -- 172.21.15.112:0/2038576908 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10020260 con 0x555dabf9c170 2024-03-16T09:17:19.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feafeffd640 1 -- 172.21.15.112:0/2038576908 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7feb10020400 con 0x555dabf9c170 2024-03-16T09:17:19.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.416+0000 7feb28826740 1 -- 172.21.15.112:0/2038576908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb10024030 con 0x555dabf9c170 2024-03-16T09:17:19.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.417+0000 7feafeffd640 1 -- 172.21.15.112:0/2038576908 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7feb10020600 con 0x555dabf9c170 2024-03-16T09:17:19.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.417+0000 7feb28826740 10 client.15873 fetch_fsmap learned FSMap version 23 2024-03-16T09:17:19.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.417+0000 7feb28826740 1 -- 172.21.15.112:0/2038576908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x555dabfaddd0 con 0x555dabf9c170 2024-03-16T09:17:19.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.417+0000 7feafeffd640 1 -- 172.21.15.112:0/2038576908 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feb10024030 con 0x555dabf9c170 2024-03-16T09:17:19.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.417+0000 7feafeffd640 1 -- 172.21.15.112:0/2038576908 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7feb10003910 con 0x555dabf9c170 2024-03-16T09:17:19.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.417+0000 7feb28826740 10 client.15873 fetch_fsmap finished waiting for FSMap version 23 2024-03-16T09:17:19.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.417+0000 7feb28826740 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:19.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.417+0000 7feb28826740 20 client.15873 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:19.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.417+0000 7feb28826740 1 --2- 172.21.15.112:0/2038576908 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x555dac018510 0x555dac01a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:19.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.417+0000 7feb28826740 4 client.15873 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T09:17:19.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.417+0000 7feb28826740 1 -- 172.21.15.112:0/2038576908 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555dabf08d90 con 0x555dac018510 2024-03-16T09:17:19.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.417+0000 7feb1effd640 1 --2- 172.21.15.112:0/2038576908 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x555dac018510 0x555dac01a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:19.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.418+0000 7feb1effd640 1 --2- 172.21.15.112:0/2038576908 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x555dac018510 0x555dac01a930 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6acb0889c4108a08 server_cookie=f31aca45fb4415bc in_seq=0 out_seq=0 2024-03-16T09:17:19.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.418+0000 7feafeffd640 10 client.15873 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:19.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.419+0000 7feafeffd640 1 -- 172.21.15.112:0/2038576908 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x555dabf08d90 con 0x555dac018510 2024-03-16T09:17:19.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.419+0000 7feafeffd640 10 client.15873 handle_command_reply: tid=0 2024-03-16T09:17:19.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.419+0000 7feb28826740 1 client.15873 shutdown 2024-03-16T09:17:19.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.419+0000 7feb28826740 1 -- 172.21.15.112:0/2038576908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9c170 msgr2=0x555dabfab0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:19.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.419+0000 7feb28826740 1 --2- 172.21.15.112:0/2038576908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9c170 0x555dabfab0e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7feb10023040 tx=0x7feb10005020 comp rx=0 tx=0).stop 2024-03-16T09:17:19.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.419+0000 7feb28826740 1 -- 172.21.15.112:0/2038576908 shutdown_connections 2024-03-16T09:17:19.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.419+0000 7feb28826740 1 --2- 172.21.15.112:0/2038576908 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x555dac018510 0x555dac01a930 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.419+0000 7feb28826740 1 --2- 172.21.15.112:0/2038576908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dabfa1cd0 0x555dabfac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.419+0000 7feb28826740 1 --2- 172.21.15.112:0/2038576908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555dabf9cb60 0x555dabfab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.419+0000 7feafeffd640 0 client.15873 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:19.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.419+0000 7feb28826740 1 --2- 172.21.15.112:0/2038576908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555dabf9c170 0x555dabfab0e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.419+0000 7feb28826740 1 -- 172.21.15.112:0/2038576908 >> 172.21.15.112:0/2038576908 conn(0x555dabf976e0 msgr2=0x555dabf9a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:19.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.419+0000 7feb28826740 1 -- 172.21.15.112:0/2038576908 shutdown_connections 2024-03-16T09:17:19.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.419+0000 7feb28826740 1 -- 172.21.15.112:0/2038576908 wait complete. 2024-03-16T09:17:19.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.419+0000 7feb28826740 20 client.15873 trim_cache size 0 max 16384 2024-03-16T09:17:19.433 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:17:19.434 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "scrub active (1223 inodes in the stack)", 2024-03-16T09:17:19.434 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-03-16T09:17:19.434 INFO:teuthology.orchestra.run.smithi112.stdout: "3f778942-1662-48df-ad81-6d9cdf602b1e": { 2024-03-16T09:17:19.434 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-03-16T09:17:19.434 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "3f778942-1662-48df-ad81-6d9cdf602b1e", 2024-03-16T09:17:19.434 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-03-16T09:17:19.434 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:17:19.434 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:17:19.434 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:17:19.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.420+0000 7feb25b5b640 1 -- 172.21.15.112:0/1144493267 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feb080a2800 msgr2=0x7feb080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:19.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.420+0000 7feb25b5b640 1 --2- 172.21.15.112:0/1144493267 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feb080a2800 0x7feb080a4cc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feb18005e30 tx=0x7feb18032670 comp rx=0 tx=0).stop 2024-03-16T09:17:19.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.420+0000 7feb25b5b640 1 -- 172.21.15.112:0/1144493267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb201018d0 msgr2=0x7feb201096a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:19.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.420+0000 7feb25b5b640 1 --2- 172.21.15.112:0/1144493267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb201018d0 0x7feb201096a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7feb1400e9f0 tx=0x7feb1400edf0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.420+0000 7feb25b5b640 1 -- 172.21.15.112:0/1144493267 shutdown_connections 2024-03-16T09:17:19.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.420+0000 7feb25b5b640 1 --2- 172.21.15.112:0/1144493267 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feb080a2800 0x7feb080a4cc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.420+0000 7feb25b5b640 1 --2- 172.21.15.112:0/1144493267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb20102290 0x7feb20109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.420+0000 7feb25b5b640 1 --2- 172.21.15.112:0/1144493267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb201018d0 0x7feb201096a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.420+0000 7feb25b5b640 1 --2- 172.21.15.112:0/1144493267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb201006e0 0x7feb2010b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:19.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.420+0000 7feb25b5b640 1 -- 172.21.15.112:0/1144493267 >> 172.21.15.112:0/1144493267 conn(0x7feb200fbc70 msgr2=0x7feb20104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:19.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.420+0000 7feb25b5b640 1 -- 172.21.15.112:0/1144493267 shutdown_connections 2024-03-16T09:17:19.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:19.420+0000 7feb25b5b640 1 -- 172.21.15.112:0/1144493267 wait complete. 2024-03-16T09:17:19.437 INFO:tasks.cephfs.filesystem:scrub status for tag:3f778942-1662-48df-ad81-6d9cdf602b1e - {'path': '/', 'tag': '3f778942-1662-48df-ad81-6d9cdf602b1e', 'options': 'recursive,force'} 2024-03-16T09:17:19.737 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.439+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000104b [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Opti auth v18 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db5679a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000104b:[<0x10000001042/Opti v18>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x1000000104b:[<0x10000001042/Opti v18>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.738 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.439+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001046 [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ESS auth v8 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db5679ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001046:[<0x10000001042/ESS v8>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x10000001046:[<0x10000001042/ESS v8>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.738 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.439+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001048 [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Introduction auth v12 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db5679b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001048:[<0x10000001042/Introduction v12>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x10000001048:[<0x10000001042/Introduction v12>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.738 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.439+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000104f [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/README.OSS auth v26 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db5679bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000104f:[<0x10000001042/README.OSS v26>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x1000000104f:[<0x10000001042/README.OSS v26>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.738 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.439+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000105a [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ultrasound auth v48 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db5679c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000105a:[<0x10000001042/ultrasound v48>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x1000000105a:[<0x10000001042/ultrasound v48>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.738 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.439+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001055 [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/VIBRA16 auth v38 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db5679c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001055:[<0x10000001042/VIBRA16 v38>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x10000001055:[<0x10000001042/VIBRA16 v38>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.738 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.440+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000104c [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/PAS16 auth v20 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db5679cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000104c:[<0x10000001042/PAS16 v20>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x1000000104c:[<0x10000001042/PAS16 v20>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.738 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.440+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000104d [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/PSS auth v22 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db5679d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000104d:[<0x10000001042/PSS v22>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x1000000104d:[<0x10000001042/PSS v22>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.738 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.440+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001054 [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Tropez+ auth v36 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db5679d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001054:[<0x10000001042/Tropez+ v36>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x10000001054:[<0x10000001042/Tropez+ v36>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.738 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.440+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000104a [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/OPL3 auth v16 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db567e0000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000104a:[<0x10000001042/OPL3 v16>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x1000000104a:[<0x10000001042/OPL3 v16>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.738 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.441+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001050 [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/README.modules auth v28 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db567e0580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001050:[<0x10000001042/README.modules v28>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x10000001050:[<0x10000001042/README.modules v28>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.738 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.441+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001056 [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/WaveArtist auth v40 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db567e1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001056:[<0x10000001042/WaveArtist v40>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x10000001056:[<0x10000001042/WaveArtist v40>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.738 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.441+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001059 [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/oss-parameters.txt auth v46 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db567e1b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001059:[<0x10000001042/oss-parameters.txt v46>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x10000001059:[<0x10000001042/oss-parameters.txt v46>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.739 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.442+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001052 [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/SoundPro auth v32 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db567e1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001052:[<0x10000001042/SoundPro v32>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x10000001052:[<0x10000001042/SoundPro v32>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.740 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.442+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001053 [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Soundblaster auth v34 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db567e2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001053:[<0x10000001042/Soundblaster v34>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x10000001053:[<0x10000001042/Soundblaster v34>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.740 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.442+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001057 [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/btaudio auth v42 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db567e2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001057:[<0x10000001042/btaudio v42>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x10000001057:[<0x10000001042/btaudio v42>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.740 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.442+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001058 [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/mwave auth v44 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db567e0b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001058:[<0x10000001042/mwave v44>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x10000001058:[<0x10000001042/mwave v44>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.740 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.442+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000104e [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/PSS-updates auth v24 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db567e2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000104e:[<0x10000001042/PSS-updates v24>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x1000000104e:[<0x10000001042/PSS-updates v24>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.740 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.443+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001047 [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ESS1868 auth v10 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db567e3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001047:[<0x10000001042/ESS1868 v10>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x10000001047:[<0x10000001042/ESS1868 v10>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.740 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.443+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001049 [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/MultiSound auth v14 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db567e3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001049:[<0x10000001042/MultiSound v14>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x10000001049:[<0x10000001042/MultiSound v14>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.740 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.443+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001045 [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/CMI8330 auth v6 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db567e8b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001045:[<0x10000001042/CMI8330 v6>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x10000001045:[<0x10000001042/CMI8330 v6>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.740 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.444+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001051 [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/README.ymfsb auth v30 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db567e8000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001051:[<0x10000001042/README.ymfsb v30>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x10000001051:[<0x10000001042/README.ymfsb v30>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.740 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.444+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001044 [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/AudioExcelDSP16 auth v4 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db567e9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001044:[<0x10000001042/AudioExcelDSP16 v4>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x10000001044:[<0x10000001042/AudioExcelDSP16 v4>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:19.740 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:17:19.444+0000 7ff09fbb5700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001043 [2,head] /volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ALS auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={25111=0-4194304@1} caps={25111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=25111 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55db567e8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000001043:[<0x10000001042/ALS v2>,<0x1000000101c/oss v104>,<0x10000000203/sound v1840>,<0x10000000202/Documentation v525>,<0x10000000201/linux-4.0.5 v25>,<0x10000000006/untar_tar v43>,<0x10000000005/tmp v40>,<0x100000001fb/client.0 v28>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v43>,<0x10000000001/sv_1 v63>,<0x10000000000/qa v54>,<0x1/volumes v48>]//[]","memoryvalue":"(3)0x10000001043:[<0x10000001042/ALS v2>,<0x1000000101c/oss v109>,<0x10000000203/sound v1962>,<0x10000000202/Documentation v579>,<0x10000000201/linux-4.0.5 v30>,<0x10000000006/untar_tar v48>,<0x10000000005/tmp v44>,<0x100000001fb/client.0 v33>,<0x100000001fa/c3301382-2c5f-45a7-a72f-5326fd07baff v49>,<0x10000000001/sv_1 v65>,<0x10000000000/qa v29>,<0x1/volumes v48>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-16T09:17:20.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: pgmap v401: 97 pgs: 97 active+clean; 83 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 7.8 MiB/s wr, 414 op/s 2024-03-16T09:17:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub queued for path: / 2024-03-16T09:17:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: active paths [/] 2024-03-16T09:17:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: active paths [/] 2024-03-16T09:17:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: active paths [/] 2024-03-16T09:17:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: active paths [/] 2024-03-16T09:17:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: active paths [/] 2024-03-16T09:17:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: active paths [/] 2024-03-16T09:17:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: active paths [/] 2024-03-16T09:17:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: active paths [/] 2024-03-16T09:17:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: active paths [/] 2024-03-16T09:17:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x1000000104b (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Opti) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x10000001046 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ESS) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x10000001048 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Introduction) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x1000000104f (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/README.OSS) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x1000000105a (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ultrasound) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x10000001055 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/VIBRA16) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x1000000104c (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/PAS16) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x1000000104d (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/PSS) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x10000001054 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Tropez+) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x1000000104a (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/OPL3) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x10000001050 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/README.modules) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x10000001056 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/WaveArtist) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x10000001059 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/oss-parameters.txt) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x10000001052 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/SoundPro) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x10000001053 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Soundblaster) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x10000001057 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/btaudio) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x10000001058 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/mwave) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x1000000104e (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/PSS-updates) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x10000001047 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ESS1868) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x10000001049 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/MultiSound) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x10000001045 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/CMI8330) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x10000001051 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/README.ymfsb) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x10000001044 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/AudioExcelDSP16) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: Scrub error on inode 0x10000001043 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ALS) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: active paths [/] 2024-03-16T09:17:20.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: active paths [/] 2024-03-16T09:17:20.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: active paths [/] 2024-03-16T09:17:20.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:19 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: pgmap v401: 97 pgs: 97 active+clean; 83 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 7.8 MiB/s wr, 414 op/s 2024-03-16T09:17:20.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub queued for path: / 2024-03-16T09:17:20.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: active paths [/] 2024-03-16T09:17:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: active paths [/] 2024-03-16T09:17:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: active paths [/] 2024-03-16T09:17:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: active paths [/] 2024-03-16T09:17:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: active paths [/] 2024-03-16T09:17:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: active paths [/] 2024-03-16T09:17:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: active paths [/] 2024-03-16T09:17:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: active paths [/] 2024-03-16T09:17:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: active paths [/] 2024-03-16T09:17:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x1000000104b (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Opti) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x10000001046 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ESS) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x10000001048 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Introduction) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x1000000104f (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/README.OSS) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x1000000105a (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ultrasound) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x10000001055 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/VIBRA16) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x1000000104c (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/PAS16) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x1000000104d (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/PSS) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x10000001054 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Tropez+) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x1000000104a (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/OPL3) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x10000001050 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/README.modules) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x10000001056 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/WaveArtist) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x10000001059 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/oss-parameters.txt) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.405 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x10000001052 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/SoundPro) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.405 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x10000001053 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Soundblaster) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.405 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x10000001057 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/btaudio) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.405 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x10000001058 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/mwave) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.405 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x1000000104e (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/PSS-updates) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.405 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x10000001047 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ESS1868) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.405 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x10000001049 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/MultiSound) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.405 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x10000001045 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/CMI8330) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.405 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x10000001051 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/README.ymfsb) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.405 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x10000001044 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/AudioExcelDSP16) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.405 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: Scrub error on inode 0x10000001043 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ALS) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.405 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.405 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: active paths [/] 2024-03-16T09:17:20.405 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.405 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: active paths [/] 2024-03-16T09:17:20.406 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.406 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: active paths [/] 2024-03-16T09:17:20.406 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:19 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: pgmap v401: 97 pgs: 97 active+clean; 83 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 7.8 MiB/s wr, 414 op/s 2024-03-16T09:17:20.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub queued for path: / 2024-03-16T09:17:20.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-16T09:17:20.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-16T09:17:20.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-16T09:17:20.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-16T09:17:20.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-16T09:17:20.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-16T09:17:20.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-16T09:17:20.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-16T09:17:20.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-16T09:17:20.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x1000000104b (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Opti) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x10000001046 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ESS) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x10000001048 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Introduction) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x1000000104f (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/README.OSS) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x1000000105a (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ultrasound) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x10000001055 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/VIBRA16) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x1000000104c (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/PAS16) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x1000000104d (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/PSS) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x10000001054 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Tropez+) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x1000000104a (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/OPL3) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x10000001050 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/README.modules) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x10000001056 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/WaveArtist) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.489 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x10000001059 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/oss-parameters.txt) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x10000001052 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/SoundPro) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x10000001053 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Soundblaster) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x10000001057 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/btaudio) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x10000001058 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/mwave) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x1000000104e (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/PSS-updates) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x10000001047 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ESS1868) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x10000001049 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/MultiSound) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x10000001045 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/CMI8330) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x10000001051 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/README.ymfsb) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x10000001044 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/AudioExcelDSP16) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: Scrub error on inode 0x10000001043 (/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ALS) see mds.h log and `damage ls` output for details 2024-03-16T09:17:20.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-16T09:17:20.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.491 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-16T09:17:20.491 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:20.491 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-16T09:17:20.491 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:19 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:20 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-a[32303]: 2024-03-16T09:17:20.987+0000 7f723909a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-16T09:17:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:20 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:20 smithi112 ceph-mon[32307]: scrub summary: active paths [/] 2024-03-16T09:17:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:20 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:21.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:20 smithi112 ceph-mon[32307]: scrub summary: active paths [/] 2024-03-16T09:17:21.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:20 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:21.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:20 smithi112 ceph-mon[32307]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:21.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:20 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:21.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:20 smithi194 ceph-mon[36729]: scrub summary: active paths [/] 2024-03-16T09:17:21.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:20 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:21.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:20 smithi194 ceph-mon[36729]: scrub summary: active paths [/] 2024-03-16T09:17:21.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:20 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:21.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:20 smithi194 ceph-mon[36729]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:21.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:20 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:21.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:20 smithi159 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-16T09:17:21.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:20 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:21.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:20 smithi159 ceph-mon[36739]: scrub summary: active paths [/] 2024-03-16T09:17:21.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:20 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:21.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:20 smithi159 ceph-mon[36739]: scrub summary: idle+waiting paths [/] 2024-03-16T09:17:22.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:21 smithi112 ceph-mon[32307]: pgmap v402: 97 pgs: 97 active+clean; 84 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.9 MiB/s wr, 513 op/s 2024-03-16T09:17:22.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:21 smithi112 ceph-mon[32307]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-16T09:17:22.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:21 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] up:active 2024-03-16T09:17:22.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:21 smithi112 ceph-mon[32307]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:17:22.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:21 smithi194 ceph-mon[36729]: pgmap v402: 97 pgs: 97 active+clean; 84 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.9 MiB/s wr, 513 op/s 2024-03-16T09:17:22.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:21 smithi194 ceph-mon[36729]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-16T09:17:22.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:21 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] up:active 2024-03-16T09:17:22.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:21 smithi194 ceph-mon[36729]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:17:22.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:21 smithi159 ceph-mon[36739]: pgmap v402: 97 pgs: 97 active+clean; 84 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.9 MiB/s wr, 513 op/s 2024-03-16T09:17:22.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:21 smithi159 ceph-mon[36739]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-16T09:17:22.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:21 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] up:active 2024-03-16T09:17:22.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:21 smithi159 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:17:24.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:23 smithi112 ceph-mon[32307]: pgmap v403: 97 pgs: 97 active+clean; 88 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 597 op/s 2024-03-16T09:17:24.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:23 smithi194 ceph-mon[36729]: pgmap v403: 97 pgs: 97 active+clean; 88 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 597 op/s 2024-03-16T09:17:24.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:23 smithi159 ceph-mon[36739]: pgmap v403: 97 pgs: 97 active+clean; 88 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 597 op/s 2024-03-16T09:17:26.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:25 smithi112 ceph-mon[32307]: pgmap v404: 97 pgs: 97 active+clean; 89 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 1022 KiB/s wr, 682 op/s 2024-03-16T09:17:26.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:25 smithi194 ceph-mon[36729]: pgmap v404: 97 pgs: 97 active+clean; 89 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 1022 KiB/s wr, 682 op/s 2024-03-16T09:17:26.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:25 smithi159 ceph-mon[36739]: pgmap v404: 97 pgs: 97 active+clean; 89 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 1022 KiB/s wr, 682 op/s 2024-03-16T09:17:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.702+0000 7f9416e63640 1 -- 172.21.15.112:0/2633896912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9410104c10 msgr2=0x7f9410107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.702+0000 7f9416e63640 1 --2- 172.21.15.112:0/2633896912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9410104c10 0x7f9410107000 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9400009900 tx=0x7f940002bc30 comp rx=0 tx=0).stop 2024-03-16T09:17:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.702+0000 7f9416e63640 1 -- 172.21.15.112:0/2633896912 shutdown_connections 2024-03-16T09:17:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.702+0000 7f9416e63640 1 --2- 172.21.15.112:0/2633896912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9410107540 0x7f9410109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.702+0000 7f9416e63640 1 --2- 172.21.15.112:0/2633896912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9410104c10 0x7f9410107000 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.702+0000 7f9416e63640 1 --2- 172.21.15.112:0/2633896912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94101022e0 0x7f94101046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.702+0000 7f9416e63640 1 -- 172.21.15.112:0/2633896912 >> 172.21.15.112:0/2633896912 conn(0x7f94100fbc70 msgr2=0x7f94100fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.702+0000 7f9416e63640 1 -- 172.21.15.112:0/2633896912 shutdown_connections 2024-03-16T09:17:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.702+0000 7f9416e63640 1 -- 172.21.15.112:0/2633896912 wait complete. 2024-03-16T09:17:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.703+0000 7f9416e63640 1 Processor -- start 2024-03-16T09:17:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.703+0000 7f9416e63640 1 -- start start 2024-03-16T09:17:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.703+0000 7f9416e63640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94101022e0 0x7f9410197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.703+0000 7f9416e63640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9410104c10 0x7f94101980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.703+0000 7f9416e63640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9410107540 0x7f941019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.703+0000 7f9416e63640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f941010d560 con 0x7f94101022e0 2024-03-16T09:17:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.703+0000 7f9416e63640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f941010d3e0 con 0x7f9410107540 2024-03-16T09:17:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.703+0000 7f9416e63640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f941010d6e0 con 0x7f9410104c10 2024-03-16T09:17:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.703+0000 7f9414bff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94101022e0 0x7f9410197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.703+0000 7f9414bff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94101022e0 0x7f9410197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45550/0 (socket says 172.21.15.112:45550) 2024-03-16T09:17:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.703+0000 7f9414bff640 1 -- 172.21.15.112:0/2666720658 learned_addr learned my addr 172.21.15.112:0/2666720658 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.703+0000 7f9407fff640 1 --2- 172.21.15.112:0/2666720658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9410104c10 0x7f94101980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.703+0000 7f9415400640 1 --2- 172.21.15.112:0/2666720658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9410107540 0x7f941019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.703+0000 7f9414bff640 1 -- 172.21.15.112:0/2666720658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9410104c10 msgr2=0x7f94101980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.703+0000 7f9414bff640 1 --2- 172.21.15.112:0/2666720658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9410104c10 0x7f94101980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.703+0000 7f9414bff640 1 -- 172.21.15.112:0/2666720658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9410107540 msgr2=0x7f941019f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.703+0000 7f9414bff640 1 --2- 172.21.15.112:0/2666720658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9410107540 0x7f941019f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.703+0000 7f9414bff640 1 -- 172.21.15.112:0/2666720658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f941019f7c0 con 0x7f94101022e0 2024-03-16T09:17:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.703+0000 7f9414bff640 1 --2- 172.21.15.112:0/2666720658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94101022e0 0x7f9410197ba0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f940c00e9f0 tx=0x7f940c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.704+0000 7f9405ffb640 1 -- 172.21.15.112:0/2666720658 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c00cd80 con 0x7f94101022e0 2024-03-16T09:17:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.704+0000 7f9416e63640 1 -- 172.21.15.112:0/2666720658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f941019fab0 con 0x7f94101022e0 2024-03-16T09:17:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.704+0000 7f9405ffb640 1 -- 172.21.15.112:0/2666720658 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f940c010930 con 0x7f94101022e0 2024-03-16T09:17:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.704+0000 7f9405ffb640 1 -- 172.21.15.112:0/2666720658 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c018b00 con 0x7f94101022e0 2024-03-16T09:17:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.704+0000 7f9416e63640 1 -- 172.21.15.112:0/2666720658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94101a0020 con 0x7f94101022e0 2024-03-16T09:17:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.706+0000 7f9405ffb640 1 -- 172.21.15.112:0/2666720658 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7f940c018ca0 con 0x7f94101022e0 2024-03-16T09:17:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9419a4a740 1 Processor -- start 2024-03-16T09:17:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9405ffb640 1 --2- 172.21.15.112:0/2666720658 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f93f80a28a0 0x7f93f80a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9419a4a740 1 -- start start 2024-03-16T09:17:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9419a4a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf83340 0x556a2bf83760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9419a4a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf83d30 0x556a2bf88830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9415400640 1 --2- 172.21.15.112:0/2666720658 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f93f80a28a0 0x7f93f80a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9419a4a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf88ea0 0x556a2bf8b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9405ffb640 1 -- 172.21.15.112:0/2666720658 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f940c0ceda0 con 0x7f94101022e0 2024-03-16T09:17:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9419a4a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556a2bee8a40 con 0x556a2bf83d30 2024-03-16T09:17:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9419a4a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556a2be91920 con 0x556a2bf88ea0 2024-03-16T09:17:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9419a4a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556a2bee4ae0 con 0x556a2bf83340 2024-03-16T09:17:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9407fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf83d30 0x556a2bf88830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9407fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf83d30 0x556a2bf88830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45558/0 (socket says 172.21.15.112:45558) 2024-03-16T09:17:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9414bff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf83340 0x556a2bf83760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9407fff640 1 -- 172.21.15.112:0/4159186947 learned_addr learned my addr 172.21.15.112:0/4159186947 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9415400640 1 --2- 172.21.15.112:0/4159186947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf88ea0 0x556a2bf8b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9407fff640 1 -- 172.21.15.112:0/4159186947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf83340 msgr2=0x556a2bf83760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9407fff640 1 --2- 172.21.15.112:0/4159186947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf83340 0x556a2bf83760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9407fff640 1 -- 172.21.15.112:0/4159186947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf88ea0 msgr2=0x556a2bf8b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9407fff640 1 --2- 172.21.15.112:0/4159186947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf88ea0 0x556a2bf8b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9407fff640 1 -- 172.21.15.112:0/4159186947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a2bea9460 con 0x556a2bf83d30 2024-03-16T09:17:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.707+0000 7f9414bff640 1 --2- 172.21.15.112:0/4159186947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf83340 0x556a2bf83760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.708+0000 7f9407fff640 1 --2- 172.21.15.112:0/4159186947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf83d30 0x556a2bf88830 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9410069960 tx=0x7f9400009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eaf63a55eb141f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.708+0000 7f9415400640 1 --2- 172.21.15.112:0/2666720658 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f93f80a28a0 0x7f93f80a4d60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9408005fd0 tx=0x7f9408009e00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.708+0000 7f93f6ffd640 1 -- 172.21.15.112:0/4159186947 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940002c5d0 con 0x556a2bf83d30 2024-03-16T09:17:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.708+0000 7f93f6ffd640 1 -- 172.21.15.112:0/4159186947 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9400004040 con 0x556a2bf83d30 2024-03-16T09:17:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.708+0000 7f93f6ffd640 1 -- 172.21.15.112:0/4159186947 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94000046c0 con 0x556a2bf83d30 2024-03-16T09:17:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.708+0000 7f9419a4a740 1 -- 172.21.15.112:0/4159186947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf83d30 msgr2=0x556a2bf88830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.708+0000 7f9419a4a740 1 --2- 172.21.15.112:0/4159186947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf83d30 0x556a2bf88830 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9410069960 tx=0x7f9400009670 comp rx=0 tx=0).stop 2024-03-16T09:17:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.708+0000 7f9419a4a740 1 -- 172.21.15.112:0/4159186947 shutdown_connections 2024-03-16T09:17:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.708+0000 7f9419a4a740 1 --2- 172.21.15.112:0/4159186947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf88ea0 0x556a2bf8b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.708+0000 7f9419a4a740 1 --2- 172.21.15.112:0/4159186947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf83d30 0x556a2bf88830 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.708+0000 7f9419a4a740 1 --2- 172.21.15.112:0/4159186947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf83340 0x556a2bf83760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.708+0000 7f9419a4a740 1 -- 172.21.15.112:0/4159186947 >> 172.21.15.112:0/4159186947 conn(0x556a2bf7e8b0 msgr2=0x556a2bf7ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.709+0000 7f9419a4a740 1 -- 172.21.15.112:0/4159186947 shutdown_connections 2024-03-16T09:17:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.709+0000 7f9419a4a740 1 -- 172.21.15.112:0/4159186947 wait complete. 2024-03-16T09:17:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.709+0000 7f9419a4a740 1 -- start start 2024-03-16T09:17:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.709+0000 7f9419a4a740 1 Processor -- start 2024-03-16T09:17:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.709+0000 7f9419a4a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf83340 0x556a2bf8d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.709+0000 7f9419a4a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf83d30 0x556a2bf8dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.709+0000 7f9419a4a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf88ea0 0x556a2bf94b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.709+0000 7f9419a4a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556a2bf8bdf0 con 0x556a2bf88ea0 2024-03-16T09:17:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.709+0000 7f9419a4a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556a2bea4d70 con 0x556a2bf83d30 2024-03-16T09:17:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.709+0000 7f9419a4a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556a2bf8bf70 con 0x556a2bf83340 2024-03-16T09:17:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.709+0000 7f9419a4a740 10 client.0 _ll_register_callbacks cb 0x556a2bbfca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.709+0000 7f9415400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf88ea0 0x556a2bf94b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.709+0000 7f9415400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf88ea0 0x556a2bf94b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45572/0 (socket says 172.21.15.112:45572) 2024-03-16T09:17:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.709+0000 7f9414bff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf83340 0x556a2bf8d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.709+0000 7f9415400640 1 -- 172.21.15.112:0/2292710318 learned_addr learned my addr 172.21.15.112:0/2292710318 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:27.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.709+0000 7f9407fff640 1 --2- 172.21.15.112:0/2292710318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf83d30 0x556a2bf8dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.710+0000 7f9414bff640 1 -- 172.21.15.112:0/2292710318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf83d30 msgr2=0x556a2bf8dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.710+0000 7f9414bff640 1 --2- 172.21.15.112:0/2292710318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf83d30 0x556a2bf8dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.710+0000 7f9414bff640 1 -- 172.21.15.112:0/2292710318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf88ea0 msgr2=0x556a2bf94b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.710+0000 7f9414bff640 1 --2- 172.21.15.112:0/2292710318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf88ea0 0x556a2bf94b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.710+0000 7f9414bff640 1 -- 172.21.15.112:0/2292710318 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a2bbfd210 con 0x556a2bf83340 2024-03-16T09:17:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.710+0000 7f9415400640 1 --2- 172.21.15.112:0/2292710318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf88ea0 0x556a2bf94b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.710+0000 7f9414bff640 1 --2- 172.21.15.112:0/2292710318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf83340 0x556a2bf8d590 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f940c00c6a0 tx=0x7f940c09b800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2815096cd9e7eb46 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.710+0000 7f93effff640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T09:17:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.710+0000 7f93effff640 1 -- 172.21.15.112:0/2292710318 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c0b4060 con 0x556a2bf83340 2024-03-16T09:17:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.710+0000 7f9419a4a740 1 -- 172.21.15.112:0/2292710318 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9400004040 con 0x556a2bf83340 2024-03-16T09:17:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.710+0000 7f93effff640 1 -- 172.21.15.112:0/2292710318 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f940c010ae0 con 0x556a2bf83340 2024-03-16T09:17:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.710+0000 7f93effff640 1 -- 172.21.15.112:0/2292710318 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f940c010ce0 con 0x556a2bf83340 2024-03-16T09:17:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.710+0000 7f9419a4a740 10 client.25744 fetch_fsmap learned FSMap version 24 2024-03-16T09:17:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.710+0000 7f9419a4a740 1 -- 172.21.15.112:0/2292710318 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556a2bcc9620 con 0x556a2bf83340 2024-03-16T09:17:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.710+0000 7f93effff640 1 -- 172.21.15.112:0/2292710318 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9400004040 con 0x556a2bf83340 2024-03-16T09:17:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.711+0000 7f93effff640 1 -- 172.21.15.112:0/2292710318 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f940c097550 con 0x556a2bf83340 2024-03-16T09:17:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.711+0000 7f9419a4a740 10 client.25744 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T09:17:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.711+0000 7f9419a4a740 10 client.25744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.711+0000 7f9419a4a740 20 client.25744 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.711+0000 7f9419a4a740 1 --2- 172.21.15.112:0/2292710318 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x556a2bf99440 0x556a2bf9b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.711+0000 7f9419a4a740 4 client.25744 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:17:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.711+0000 7f9419a4a740 1 -- 172.21.15.112:0/2292710318 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a2bef5f80 con 0x556a2bf99440 2024-03-16T09:17:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.711+0000 7f9407fff640 1 --2- 172.21.15.112:0/2292710318 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x556a2bf99440 0x556a2bf9b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.711+0000 7f93effff640 10 client.25744 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:27.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.711+0000 7f9407fff640 1 --2- 172.21.15.112:0/2292710318 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x556a2bf99440 0x556a2bf9b8d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a043b532f953cfa server_cookie=b549a449965de0e4 in_seq=0 out_seq=0 2024-03-16T09:17:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.715+0000 7f93effff640 1 -- 172.21.15.112:0/2292710318 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556a2bef5f80 con 0x556a2bf99440 2024-03-16T09:17:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.715+0000 7f93effff640 10 client.25744 handle_command_reply: tid=0 2024-03-16T09:17:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.715+0000 7f9419a4a740 1 client.25744 shutdown 2024-03-16T09:17:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.716+0000 7f9419a4a740 1 -- 172.21.15.112:0/2292710318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf83340 msgr2=0x556a2bf8d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.716+0000 7f9419a4a740 1 --2- 172.21.15.112:0/2292710318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf83340 0x556a2bf8d590 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f940c00c6a0 tx=0x7f940c09b800 comp rx=0 tx=0).stop 2024-03-16T09:17:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.716+0000 7f9419a4a740 1 -- 172.21.15.112:0/2292710318 shutdown_connections 2024-03-16T09:17:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.716+0000 7f9419a4a740 1 --2- 172.21.15.112:0/2292710318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf88ea0 0x556a2bf94b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.716+0000 7f9419a4a740 1 --2- 172.21.15.112:0/2292710318 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x556a2bf99440 0x556a2bf9b8d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.716+0000 7f9419a4a740 1 --2- 172.21.15.112:0/2292710318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf83d30 0x556a2bf8dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.716+0000 7f9419a4a740 1 --2- 172.21.15.112:0/2292710318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf83340 0x556a2bf8d590 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.716+0000 7f93effff640 0 client.25744 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.716+0000 7f9419a4a740 1 -- 172.21.15.112:0/2292710318 >> 172.21.15.112:0/2292710318 conn(0x556a2bf7e8b0 msgr2=0x556a2bf7dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:27.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.716+0000 7f9419a4a740 1 -- 172.21.15.112:0/2292710318 shutdown_connections 2024-03-16T09:17:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.716+0000 7f9419a4a740 1 -- 172.21.15.112:0/2292710318 wait complete. 2024-03-16T09:17:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.716+0000 7f9419a4a740 20 client.25744 trim_cache size 0 max 16384 2024-03-16T09:17:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.731+0000 7f9419a4a740 1 Processor -- start 2024-03-16T09:17:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.731+0000 7f9419a4a740 1 -- start start 2024-03-16T09:17:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.731+0000 7f9419a4a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf83340 0x556a2bfa11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.731+0000 7f9419a4a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf83d30 0x556a2bf8d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.731+0000 7f9419a4a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf88ea0 0x556a2bf8d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:27.737 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.731+0000 7f9419a4a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f94101ab510 con 0x556a2bf88ea0 2024-03-16T09:17:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.731+0000 7f9419a4a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f94101a8de0 con 0x556a2bf83d30 2024-03-16T09:17:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.731+0000 7f9419a4a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f94101a6c30 con 0x556a2bf83340 2024-03-16T09:17:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.731+0000 7f9415400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf88ea0 0x556a2bf8d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.731+0000 7f9415400640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf88ea0 0x556a2bf8d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45574/0 (socket says 172.21.15.112:45574) 2024-03-16T09:17:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.731+0000 7f9415400640 1 -- 172.21.15.112:0/2573087547 learned_addr learned my addr 172.21.15.112:0/2573087547 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.731+0000 7f9407fff640 1 --2- 172.21.15.112:0/2573087547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf83d30 0x556a2bf8d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.731+0000 7f9414bff640 1 --2- 172.21.15.112:0/2573087547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf83340 0x556a2bfa11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.731+0000 7f9415400640 1 -- 172.21.15.112:0/2573087547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf83340 msgr2=0x556a2bfa11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.731+0000 7f9415400640 1 --2- 172.21.15.112:0/2573087547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf83340 0x556a2bfa11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.731+0000 7f9415400640 1 -- 172.21.15.112:0/2573087547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf83d30 msgr2=0x556a2bf8d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.731+0000 7f9415400640 1 --2- 172.21.15.112:0/2573087547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf83d30 0x556a2bf8d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.731+0000 7f9415400640 1 -- 172.21.15.112:0/2573087547 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a2bf7a310 con 0x556a2bf88ea0 2024-03-16T09:17:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.731+0000 7f9415400640 1 --2- 172.21.15.112:0/2573087547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf88ea0 0x556a2bf8d6e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f94080135d0 tx=0x7f9408009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3141c0e9e2ba3777 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.732+0000 7f93f67fc640 1 -- 172.21.15.112:0/2573087547 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940801fe30 con 0x556a2bf88ea0 2024-03-16T09:17:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.732+0000 7f93f67fc640 1 -- 172.21.15.112:0/2573087547 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9408013170 con 0x556a2bf88ea0 2024-03-16T09:17:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.732+0000 7f93f67fc640 1 -- 172.21.15.112:0/2573087547 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9408004780 con 0x556a2bf88ea0 2024-03-16T09:17:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.732+0000 7f9419a4a740 1 -- 172.21.15.112:0/2573087547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf88ea0 msgr2=0x556a2bf8d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.732+0000 7f9419a4a740 1 --2- 172.21.15.112:0/2573087547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf88ea0 0x556a2bf8d6e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f94080135d0 tx=0x7f9408009500 comp rx=0 tx=0).stop 2024-03-16T09:17:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.732+0000 7f9419a4a740 1 -- 172.21.15.112:0/2573087547 shutdown_connections 2024-03-16T09:17:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.732+0000 7f9419a4a740 1 --2- 172.21.15.112:0/2573087547 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf88ea0 0x556a2bf8d6e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.732+0000 7f9419a4a740 1 --2- 172.21.15.112:0/2573087547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf83d30 0x556a2bf8d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.732+0000 7f9419a4a740 1 --2- 172.21.15.112:0/2573087547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf83340 0x556a2bfa11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.732+0000 7f9419a4a740 1 -- 172.21.15.112:0/2573087547 >> 172.21.15.112:0/2573087547 conn(0x556a2bf7e8b0 msgr2=0x556a2bf7ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.732+0000 7f9419a4a740 1 -- 172.21.15.112:0/2573087547 shutdown_connections 2024-03-16T09:17:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.732+0000 7f9419a4a740 1 -- 172.21.15.112:0/2573087547 wait complete. 2024-03-16T09:17:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.733+0000 7f9419a4a740 1 -- start start 2024-03-16T09:17:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.733+0000 7f9419a4a740 1 Processor -- start 2024-03-16T09:17:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.733+0000 7f9419a4a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf83340 0x556a2bf94630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.733+0000 7f9419a4a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf83d30 0x556a2bf92c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.733+0000 7f9419a4a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf88ea0 0x556a2bf93180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.733+0000 7f9419a4a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f94101a72a0 con 0x556a2bf83d30 2024-03-16T09:17:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.733+0000 7f9419a4a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f94101a8710 con 0x556a2bf83340 2024-03-16T09:17:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.733+0000 7f9419a4a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f94101a7970 con 0x556a2bf88ea0 2024-03-16T09:17:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.733+0000 7f9419a4a740 10 client.0 _ll_register_callbacks cb 0x7f940c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.733+0000 7f9407fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf83d30 0x556a2bf92c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.733+0000 7f9407fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf83d30 0x556a2bf92c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45584/0 (socket says 172.21.15.112:45584) 2024-03-16T09:17:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.733+0000 7f9407fff640 1 -- 172.21.15.112:0/3072854720 learned_addr learned my addr 172.21.15.112:0/3072854720 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.733+0000 7f9414bff640 1 --2- 172.21.15.112:0/3072854720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf83340 0x556a2bf94630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.733+0000 7f9415400640 1 --2- 172.21.15.112:0/3072854720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf88ea0 0x556a2bf93180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.733+0000 7f9407fff640 1 -- 172.21.15.112:0/3072854720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf88ea0 msgr2=0x556a2bf93180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f9407fff640 1 --2- 172.21.15.112:0/3072854720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf88ea0 0x556a2bf93180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f9407fff640 1 -- 172.21.15.112:0/3072854720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf83340 msgr2=0x556a2bf94630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f9407fff640 1 --2- 172.21.15.112:0/3072854720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf83340 0x556a2bf94630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f9407fff640 1 -- 172.21.15.112:0/3072854720 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a2bf938f0 con 0x556a2bf83d30 2024-03-16T09:17:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f9407fff640 1 --2- 172.21.15.112:0/3072854720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf83d30 0x556a2bf92c40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9400009900 tx=0x7f940002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cef5052f3486262 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f93effff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f93effff640 1 -- 172.21.15.112:0/3072854720 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940002c6f0 con 0x556a2bf83d30 2024-03-16T09:17:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f93effff640 1 -- 172.21.15.112:0/3072854720 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8211+0+0 (secure 0 0 0) 0x7f940002cad0 con 0x556a2bf83d30 2024-03-16T09:17:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f9419a4a740 1 -- 172.21.15.112:0/3072854720 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9408013170 con 0x556a2bf83d30 2024-03-16T09:17:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f93effff640 1 -- 172.21.15.112:0/3072854720 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f940002ccf0 con 0x556a2bf83d30 2024-03-16T09:17:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f9419a4a740 10 client.15891 fetch_fsmap learned FSMap version 24 2024-03-16T09:17:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f9419a4a740 1 -- 172.21.15.112:0/3072854720 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556a2c000840 con 0x556a2bf83d30 2024-03-16T09:17:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f93effff640 1 -- 172.21.15.112:0/3072854720 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9408013170 con 0x556a2bf83d30 2024-03-16T09:17:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f93effff640 1 -- 172.21.15.112:0/3072854720 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f94000035f0 con 0x556a2bf83d30 2024-03-16T09:17:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f9419a4a740 10 client.15891 fetch_fsmap finished waiting for FSMap version 24 2024-03-16T09:17:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f9419a4a740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f9419a4a740 20 client.15891 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f9419a4a740 1 --2- 172.21.15.112:0/3072854720 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x556a2c000d90 0x556a2c0031b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f9419a4a740 4 client.15891 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T09:17:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.734+0000 7f9419a4a740 1 -- 172.21.15.112:0/3072854720 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a2bef02d0 con 0x556a2c000d90 2024-03-16T09:17:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.735+0000 7f9414bff640 1 --2- 172.21.15.112:0/3072854720 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x556a2c000d90 0x556a2c0031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.735+0000 7f93effff640 10 client.15891 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:27.735+0000 7f9414bff640 1 --2- 172.21.15.112:0/3072854720 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x556a2c000d90 0x556a2c0031b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b40ebc3f96bcd0a server_cookie=b23236feb4aae50a in_seq=0 out_seq=0 2024-03-16T09:17:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.200+0000 7f93effff640 1 -- 172.21.15.112:0/3072854720 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556a2bef02d0 con 0x556a2c000d90 2024-03-16T09:17:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.200+0000 7f93effff640 10 client.15891 handle_command_reply: tid=0 2024-03-16T09:17:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.200+0000 7f9419a4a740 1 client.15891 shutdown 2024-03-16T09:17:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.200+0000 7f9419a4a740 1 -- 172.21.15.112:0/3072854720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf83d30 msgr2=0x556a2bf92c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.200+0000 7f9419a4a740 1 --2- 172.21.15.112:0/3072854720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf83d30 0x556a2bf92c40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9400009900 tx=0x7f940002bc30 comp rx=0 tx=0).stop 2024-03-16T09:17:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.201+0000 7f9419a4a740 1 -- 172.21.15.112:0/3072854720 shutdown_connections 2024-03-16T09:17:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.201+0000 7f9419a4a740 1 --2- 172.21.15.112:0/3072854720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556a2bf88ea0 0x556a2bf93180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.201+0000 7f9419a4a740 1 --2- 172.21.15.112:0/3072854720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a2bf83d30 0x556a2bf92c40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.201+0000 7f9419a4a740 1 --2- 172.21.15.112:0/3072854720 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x556a2c000d90 0x556a2c0031b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.201+0000 7f9419a4a740 1 --2- 172.21.15.112:0/3072854720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a2bf83340 0x556a2bf94630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.201+0000 7f93effff640 0 client.15891 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.201+0000 7f9419a4a740 1 -- 172.21.15.112:0/3072854720 >> 172.21.15.112:0/3072854720 conn(0x556a2bf7e8b0 msgr2=0x556a2bf8d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.201+0000 7f9419a4a740 1 -- 172.21.15.112:0/3072854720 shutdown_connections 2024-03-16T09:17:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.201+0000 7f9419a4a740 1 -- 172.21.15.112:0/3072854720 wait complete. 2024-03-16T09:17:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.201+0000 7f9419a4a740 20 client.15891 trim_cache size 0 max 16384 2024-03-16T09:17:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-03-16T09:17:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-03-16T09:17:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-03-16T09:17:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-03-16T09:17:28.205 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.201+0000 7f9416e63640 1 -- 172.21.15.112:0/2666720658 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f93f80a28a0 msgr2=0x7f93f80a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.201+0000 7f9416e63640 1 --2- 172.21.15.112:0/2666720658 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f93f80a28a0 0x7f93f80a4d60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9408005fd0 tx=0x7f9408009e00 comp rx=0 tx=0).stop 2024-03-16T09:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.201+0000 7f9416e63640 1 -- 172.21.15.112:0/2666720658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94101022e0 msgr2=0x7f9410197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.201+0000 7f9416e63640 1 --2- 172.21.15.112:0/2666720658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94101022e0 0x7f9410197ba0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f940c00e9f0 tx=0x7f940c00eeb0 comp rx=0 tx=0).stop 2024-03-16T09:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.202+0000 7f9416e63640 1 -- 172.21.15.112:0/2666720658 shutdown_connections 2024-03-16T09:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.202+0000 7f9416e63640 1 --2- 172.21.15.112:0/2666720658 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f93f80a28a0 0x7f93f80a4d60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.202+0000 7f9416e63640 1 --2- 172.21.15.112:0/2666720658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9410107540 0x7f941019f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.202+0000 7f9416e63640 1 --2- 172.21.15.112:0/2666720658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9410104c10 0x7f94101980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.202+0000 7f9416e63640 1 --2- 172.21.15.112:0/2666720658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94101022e0 0x7f9410197ba0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.202+0000 7f9416e63640 1 -- 172.21.15.112:0/2666720658 >> 172.21.15.112:0/2666720658 conn(0x7f94100fbc70 msgr2=0x7f9410102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.202+0000 7f9416e63640 1 -- 172.21.15.112:0/2666720658 shutdown_connections 2024-03-16T09:17:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:28.202+0000 7f9416e63640 1 -- 172.21.15.112:0/2666720658 wait complete. 2024-03-16T09:17:28.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:27 smithi112 ceph-mon[32307]: pgmap v405: 97 pgs: 97 active+clean; 89 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 842 KiB/s wr, 605 op/s 2024-03-16T09:17:28.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:27 smithi194 ceph-mon[36729]: pgmap v405: 97 pgs: 97 active+clean; 89 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 842 KiB/s wr, 605 op/s 2024-03-16T09:17:28.402 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:17:28 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-c[67892]: ignoring --setuser ceph since I am not root 2024-03-16T09:17:28.402 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:17:28 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-c[67892]: ignoring --setgroup ceph since I am not root 2024-03-16T09:17:28.402 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:17:28 smithi194 ceph-mds[67898]: -- 172.21.15.194:0/1842670656 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555f1cd71a40 con 0x555f1cd44c00 2024-03-16T09:17:28.402 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:17:28 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-c[67892]: starting mds.c at 2024-03-16T09:17:28.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:27 smithi159 ceph-mon[36739]: pgmap v405: 97 pgs: 97 active+clean; 89 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 842 KiB/s wr, 605 op/s 2024-03-16T09:17:29.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:29 smithi112 ceph-mon[32307]: daemon mds.c restarted 2024-03-16T09:17:29.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:29 smithi194 ceph-mon[36729]: daemon mds.c restarted 2024-03-16T09:17:29.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:29 smithi159 ceph-mon[36739]: daemon mds.c restarted 2024-03-16T09:17:30.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:30 smithi112 ceph-mon[32307]: pgmap v406: 97 pgs: 97 active+clean; 96 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 585 op/s 2024-03-16T09:17:30.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:30 smithi112 ceph-mon[32307]: osdmap e88: 12 total, 12 up, 12 in 2024-03-16T09:17:30.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:30 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.194:6836/2561048853,v1:172.21.15.194:6837/2561048853] up:boot 2024-03-16T09:17:30.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:30 smithi112 ceph-mon[32307]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-03-16T09:17:30.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:30 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T09:17:30.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:30 smithi112 ceph-mon[32307]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:17:30.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:30 smithi194 ceph-mon[36729]: pgmap v406: 97 pgs: 97 active+clean; 96 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 585 op/s 2024-03-16T09:17:30.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:30 smithi194 ceph-mon[36729]: osdmap e88: 12 total, 12 up, 12 in 2024-03-16T09:17:30.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:30 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.194:6836/2561048853,v1:172.21.15.194:6837/2561048853] up:boot 2024-03-16T09:17:30.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:30 smithi194 ceph-mon[36729]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-03-16T09:17:30.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:30 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T09:17:30.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:30 smithi194 ceph-mon[36729]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:17:30.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:30 smithi159 ceph-mon[36739]: pgmap v406: 97 pgs: 97 active+clean; 96 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 585 op/s 2024-03-16T09:17:30.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:30 smithi159 ceph-mon[36739]: osdmap e88: 12 total, 12 up, 12 in 2024-03-16T09:17:30.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:30 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.194:6836/2561048853,v1:172.21.15.194:6837/2561048853] up:boot 2024-03-16T09:17:30.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:30 smithi159 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-03-16T09:17:30.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:30 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T09:17:30.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:30 smithi159 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:17:32.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:32 smithi112 ceph-mon[32307]: pgmap v408: 97 pgs: 97 active+clean; 93 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 319 op/s 2024-03-16T09:17:32.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:32 smithi112 ceph-mon[32307]: scrub summary: idle 2024-03-16T09:17:32.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:32 smithi112 ceph-mon[32307]: scrub summary: idle 2024-03-16T09:17:32.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:32 smithi112 ceph-mon[32307]: scrub summary: idle 2024-03-16T09:17:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:32 smithi194 ceph-mon[36729]: pgmap v408: 97 pgs: 97 active+clean; 93 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 319 op/s 2024-03-16T09:17:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:32 smithi194 ceph-mon[36729]: scrub summary: idle 2024-03-16T09:17:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:32 smithi194 ceph-mon[36729]: scrub summary: idle 2024-03-16T09:17:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:32 smithi194 ceph-mon[36729]: scrub summary: idle 2024-03-16T09:17:32.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:32 smithi159 ceph-mon[36739]: pgmap v408: 97 pgs: 97 active+clean; 93 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 319 op/s 2024-03-16T09:17:32.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:32 smithi159 ceph-mon[36739]: scrub summary: idle 2024-03-16T09:17:32.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:32 smithi159 ceph-mon[36739]: scrub summary: idle 2024-03-16T09:17:32.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:32 smithi159 ceph-mon[36739]: scrub summary: idle 2024-03-16T09:17:34.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:34 smithi112 ceph-mon[32307]: pgmap v409: 97 pgs: 97 active+clean; 112 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 383 op/s 2024-03-16T09:17:34.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:34 smithi194 ceph-mon[36729]: pgmap v409: 97 pgs: 97 active+clean; 112 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 383 op/s 2024-03-16T09:17:34.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:34 smithi159 ceph-mon[36739]: pgmap v409: 97 pgs: 97 active+clean; 112 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 383 op/s 2024-03-16T09:17:36.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:36 smithi112 ceph-mon[32307]: pgmap v410: 97 pgs: 97 active+clean; 138 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.7 MiB/s wr, 647 op/s 2024-03-16T09:17:36.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:36 smithi194 ceph-mon[36729]: pgmap v410: 97 pgs: 97 active+clean; 138 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.7 MiB/s wr, 647 op/s 2024-03-16T09:17:36.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:36 smithi159 ceph-mon[36739]: pgmap v410: 97 pgs: 97 active+clean; 138 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.7 MiB/s wr, 647 op/s 2024-03-16T09:17:38.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:38 smithi112 ceph-mon[32307]: pgmap v411: 97 pgs: 97 active+clean; 147 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.7 MiB/s wr, 741 op/s 2024-03-16T09:17:38.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:38 smithi194 ceph-mon[36729]: pgmap v411: 97 pgs: 97 active+clean; 147 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.7 MiB/s wr, 741 op/s 2024-03-16T09:17:38.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:38 smithi159 ceph-mon[36739]: pgmap v411: 97 pgs: 97 active+clean; 147 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.7 MiB/s wr, 741 op/s 2024-03-16T09:17:40.106 INFO:tasks.workunit.client.0.smithi112.stderr:+ sync 2024-03-16T09:17:40.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:40 smithi112 ceph-mon[32307]: pgmap v412: 97 pgs: 97 active+clean; 158 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.0 MiB/s wr, 973 op/s 2024-03-16T09:17:40.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:40 smithi194 ceph-mon[36729]: pgmap v412: 97 pgs: 97 active+clean; 158 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.0 MiB/s wr, 973 op/s 2024-03-16T09:17:40.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:40 smithi159 ceph-mon[36739]: pgmap v412: 97 pgs: 97 active+clean; 158 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.0 MiB/s wr, 973 op/s 2024-03-16T09:17:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.299+0000 7f2756713640 1 -- 172.21.15.112:0/2145774086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2750101910 msgr2=0x7f2750101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.299+0000 7f2756713640 1 --2- 172.21.15.112:0/2145774086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2750101910 0x7f2750101d90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2744009900 tx=0x7f274402bc30 comp rx=0 tx=0).stop 2024-03-16T09:17:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.299+0000 7f2756713640 1 -- 172.21.15.112:0/2145774086 shutdown_connections 2024-03-16T09:17:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.299+0000 7f2756713640 1 --2- 172.21.15.112:0/2145774086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27501022d0 0x7f2750108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.299+0000 7f2756713640 1 --2- 172.21.15.112:0/2145774086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2750101910 0x7f2750101d90 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.299+0000 7f2756713640 1 --2- 172.21.15.112:0/2145774086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2750100720 0x7f2750100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.299+0000 7f2756713640 1 -- 172.21.15.112:0/2145774086 >> 172.21.15.112:0/2145774086 conn(0x7f27500fbc90 msgr2=0x7f27500fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.299+0000 7f2756713640 1 -- 172.21.15.112:0/2145774086 shutdown_connections 2024-03-16T09:17:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.299+0000 7f2756713640 1 -- 172.21.15.112:0/2145774086 wait complete. 2024-03-16T09:17:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.300+0000 7f2756713640 1 Processor -- start 2024-03-16T09:17:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.300+0000 7f2756713640 1 -- start start 2024-03-16T09:17:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.300+0000 7f2756713640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2750100720 0x7f275006ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.300+0000 7f2756713640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2750101910 0x7f275019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.300+0000 7f2756713640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27501022d0 0x7f275019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.300+0000 7f2756713640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f275010c480 con 0x7f2750101910 2024-03-16T09:17:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.300+0000 7f2756713640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f275010c300 con 0x7f2750100720 2024-03-16T09:17:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.300+0000 7f2756713640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f275010c600 con 0x7f27501022d0 2024-03-16T09:17:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.300+0000 7f274f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2750101910 0x7f275019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.300+0000 7f274f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2750101910 0x7f275019c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45576/0 (socket says 172.21.15.112:45576) 2024-03-16T09:17:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.300+0000 7f274f7fe640 1 -- 172.21.15.112:0/115383694 learned_addr learned my addr 172.21.15.112:0/115383694 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.300+0000 7f274ffff640 1 --2- 172.21.15.112:0/115383694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2750100720 0x7f275006ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.301+0000 7f2754cb0640 1 --2- 172.21.15.112:0/115383694 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27501022d0 0x7f275019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.301+0000 7f274f7fe640 1 -- 172.21.15.112:0/115383694 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27501022d0 msgr2=0x7f275019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.301+0000 7f274f7fe640 1 --2- 172.21.15.112:0/115383694 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27501022d0 0x7f275019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.301+0000 7f274f7fe640 1 -- 172.21.15.112:0/115383694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2750100720 msgr2=0x7f275006ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.301+0000 7f274f7fe640 1 --2- 172.21.15.112:0/115383694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2750100720 0x7f275006ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.301+0000 7f274f7fe640 1 -- 172.21.15.112:0/115383694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275019f7c0 con 0x7f2750101910 2024-03-16T09:17:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.301+0000 7f274ffff640 1 --2- 172.21.15.112:0/115383694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2750100720 0x7f275006ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.301+0000 7f274f7fe640 1 --2- 172.21.15.112:0/115383694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2750101910 0x7f275019c890 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2744009230 tx=0x7f2744004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.301+0000 7f274d7fa640 1 -- 172.21.15.112:0/115383694 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744043020 con 0x7f2750101910 2024-03-16T09:17:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.301+0000 7f274d7fa640 1 -- 172.21.15.112:0/115383694 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2744004250 con 0x7f2750101910 2024-03-16T09:17:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.301+0000 7f2756713640 1 -- 172.21.15.112:0/115383694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f275019f9f0 con 0x7f2750101910 2024-03-16T09:17:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.301+0000 7f274d7fa640 1 -- 172.21.15.112:0/115383694 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744003b50 con 0x7f2750101910 2024-03-16T09:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.301+0000 7f2756713640 1 -- 172.21.15.112:0/115383694 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f275019ff40 con 0x7f2750101910 2024-03-16T09:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.304+0000 7f274d7fa640 1 -- 172.21.15.112:0/115383694 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7f2744046020 con 0x7f2750101910 2024-03-16T09:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.304+0000 7f27593cf740 1 Processor -- start 2024-03-16T09:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.304+0000 7f27593cf740 1 -- start start 2024-03-16T09:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.304+0000 7f274d7fa640 1 --2- 172.21.15.112:0/115383694 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f27380a28a0 0x7f27380a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.304+0000 7f27593cf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e968369340 0x55e968369760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.304+0000 7f27593cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369d30 0x55e96836e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.304+0000 7f27593cf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96836eea0 0x55e968371330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.304+0000 7f27593cf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e9682cea40 con 0x55e968369340 2024-03-16T09:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.304+0000 7f27593cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e968277920 con 0x55e968369d30 2024-03-16T09:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.304+0000 7f27593cf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e9682caae0 con 0x55e96836eea0 2024-03-16T09:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.304+0000 7f2754cb0640 1 --2- 172.21.15.112:0/115383694 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f27380a28a0 0x7f27380a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.304+0000 7f274ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e968369340 0x55e968369760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.304+0000 7f274ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e968369340 0x55e968369760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45586/0 (socket says 172.21.15.112:45586) 2024-03-16T09:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.304+0000 7f274ffff640 1 -- 172.21.15.112:0/3690050213 learned_addr learned my addr 172.21.15.112:0/3690050213 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.304+0000 7f274d7fa640 1 -- 172.21.15.112:0/115383694 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f27440f0b60 con 0x7f2750101910 2024-03-16T09:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.304+0000 7f2754cb0640 1 --2- 172.21.15.112:0/3690050213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96836eea0 0x55e968371330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.304+0000 7f274f7fe640 1 --2- 172.21.15.112:0/3690050213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369d30 0x55e96836e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.305+0000 7f274f7fe640 1 -- 172.21.15.112:0/3690050213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96836eea0 msgr2=0x55e968371330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.305+0000 7f274f7fe640 1 --2- 172.21.15.112:0/3690050213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96836eea0 0x55e968371330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.305+0000 7f274f7fe640 1 -- 172.21.15.112:0/3690050213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e968369340 msgr2=0x55e968369760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.305+0000 7f274f7fe640 1 --2- 172.21.15.112:0/3690050213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e968369340 0x55e968369760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.305+0000 7f274f7fe640 1 -- 172.21.15.112:0/3690050213 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e96828f460 con 0x55e968369d30 2024-03-16T09:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.305+0000 7f274ffff640 1 --2- 172.21.15.112:0/3690050213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e968369340 0x55e968369760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.305+0000 7f2754cb0640 1 --2- 172.21.15.112:0/115383694 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f27380a28a0 0x7f27380a4d60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2748005fd0 tx=0x7f2748009b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.305+0000 7f274f7fe640 1 --2- 172.21.15.112:0/3690050213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369d30 0x55e96836e830 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f27440ee9e0 tx=0x7f2744009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bd7a94be7f9322e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.305+0000 7f27367fc640 1 -- 172.21.15.112:0/3690050213 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27440bce70 con 0x55e968369d30 2024-03-16T09:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.305+0000 7f27367fc640 1 -- 172.21.15.112:0/3690050213 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f27440bdc10 con 0x55e968369d30 2024-03-16T09:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.305+0000 7f27367fc640 1 -- 172.21.15.112:0/3690050213 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744003cf0 con 0x55e968369d30 2024-03-16T09:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.306+0000 7f27593cf740 1 -- 172.21.15.112:0/3690050213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369d30 msgr2=0x55e96836e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.306+0000 7f27593cf740 1 --2- 172.21.15.112:0/3690050213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369d30 0x55e96836e830 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f27440ee9e0 tx=0x7f2744009620 comp rx=0 tx=0).stop 2024-03-16T09:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.306+0000 7f27593cf740 1 -- 172.21.15.112:0/3690050213 shutdown_connections 2024-03-16T09:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.306+0000 7f27593cf740 1 --2- 172.21.15.112:0/3690050213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96836eea0 0x55e968371330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.306+0000 7f27593cf740 1 --2- 172.21.15.112:0/3690050213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369d30 0x55e96836e830 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.306+0000 7f27593cf740 1 --2- 172.21.15.112:0/3690050213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e968369340 0x55e968369760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.306+0000 7f27593cf740 1 -- 172.21.15.112:0/3690050213 >> 172.21.15.112:0/3690050213 conn(0x55e9683648b0 msgr2=0x55e968364cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.306+0000 7f27593cf740 1 -- 172.21.15.112:0/3690050213 shutdown_connections 2024-03-16T09:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.306+0000 7f27593cf740 1 -- 172.21.15.112:0/3690050213 wait complete. 2024-03-16T09:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.306+0000 7f27593cf740 1 -- start start 2024-03-16T09:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f27593cf740 1 Processor -- start 2024-03-16T09:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f27593cf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e968369340 0x55e968373320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f27593cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369d30 0x55e968373860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f27593cf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96836eea0 0x55e96837a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f27593cf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e968277920 con 0x55e968369340 2024-03-16T09:17:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f27593cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e96828ad70 con 0x55e968369d30 2024-03-16T09:17:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f27593cf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e968371c10 con 0x55e96836eea0 2024-03-16T09:17:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f27593cf740 10 client.0 _ll_register_callbacks cb 0x55e967fe2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f274ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e968369340 0x55e968373320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f274ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e968369340 0x55e968373320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45592/0 (socket says 172.21.15.112:45592) 2024-03-16T09:17:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f274ffff640 1 -- 172.21.15.112:0/2286798241 learned_addr learned my addr 172.21.15.112:0/2286798241 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f274f7fe640 1 --2- 172.21.15.112:0/2286798241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369d30 0x55e968373860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f2754cb0640 1 --2- 172.21.15.112:0/2286798241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96836eea0 0x55e96837a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f274ffff640 1 -- 172.21.15.112:0/2286798241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96836eea0 msgr2=0x55e96837a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f274ffff640 1 --2- 172.21.15.112:0/2286798241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96836eea0 0x55e96837a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f274ffff640 1 -- 172.21.15.112:0/2286798241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369d30 msgr2=0x55e968373860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f274ffff640 1 --2- 172.21.15.112:0/2286798241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369d30 0x55e968373860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f274ffff640 1 -- 172.21.15.112:0/2286798241 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e967fe3210 con 0x55e968369340 2024-03-16T09:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f274ffff640 1 --2- 172.21.15.112:0/2286798241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e968369340 0x55e968373320 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2740012d20 tx=0x7f274000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7e37be73065d49 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f272f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.307+0000 7f272f7fe640 1 -- 172.21.15.112:0/2286798241 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27400120d0 con 0x55e968369340 2024-03-16T09:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.308+0000 7f272f7fe640 1 -- 172.21.15.112:0/2286798241 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f2740010430 con 0x55e968369340 2024-03-16T09:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.308+0000 7f27593cf740 1 -- 172.21.15.112:0/2286798241 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27440bdc10 con 0x55e968369340 2024-03-16T09:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.308+0000 7f272f7fe640 1 -- 172.21.15.112:0/2286798241 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2740012270 con 0x55e968369340 2024-03-16T09:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.308+0000 7f27593cf740 10 client.15909 fetch_fsmap learned FSMap version 26 2024-03-16T09:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.308+0000 7f27593cf740 1 -- 172.21.15.112:0/2286798241 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e9680af620 con 0x55e968369340 2024-03-16T09:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.308+0000 7f272f7fe640 1 -- 172.21.15.112:0/2286798241 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f27440bdc10 con 0x55e968369340 2024-03-16T09:17:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.308+0000 7f272f7fe640 1 -- 172.21.15.112:0/2286798241 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f274001c670 con 0x55e968369340 2024-03-16T09:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.308+0000 7f27593cf740 10 client.15909 fetch_fsmap finished waiting for FSMap version 26 2024-03-16T09:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.308+0000 7f27593cf740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.308+0000 7f27593cf740 20 client.15909 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.308+0000 7f27593cf740 1 --2- 172.21.15.112:0/2286798241 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55e96837f250 0x55e9683816e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.308+0000 7f27593cf740 4 client.15909 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.308+0000 7f27593cf740 1 -- 172.21.15.112:0/2286798241 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9682dbf80 con 0x55e96837f250 2024-03-16T09:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.308+0000 7f274f7fe640 1 --2- 172.21.15.112:0/2286798241 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55e96837f250 0x55e9683816e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.309+0000 7f274f7fe640 1 --2- 172.21.15.112:0/2286798241 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55e96837f250 0x55e9683816e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428b137d672de3fc server_cookie=92f52cc8f0cc2f63 in_seq=0 out_seq=0 2024-03-16T09:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.309+0000 7f272f7fe640 10 client.15909 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.310+0000 7f272f7fe640 1 -- 172.21.15.112:0/2286798241 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e9682dbf80 con 0x55e96837f250 2024-03-16T09:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.310+0000 7f272f7fe640 10 client.15909 handle_command_reply: tid=0 2024-03-16T09:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.310+0000 7f27593cf740 1 client.15909 shutdown 2024-03-16T09:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.310+0000 7f27593cf740 1 -- 172.21.15.112:0/2286798241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e968369340 msgr2=0x55e968373320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.310+0000 7f27593cf740 1 --2- 172.21.15.112:0/2286798241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e968369340 0x55e968373320 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2740012d20 tx=0x7f274000f790 comp rx=0 tx=0).stop 2024-03-16T09:17:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.310+0000 7f27593cf740 1 -- 172.21.15.112:0/2286798241 shutdown_connections 2024-03-16T09:17:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.310+0000 7f27593cf740 1 --2- 172.21.15.112:0/2286798241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e96836eea0 0x55e96837a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.310+0000 7f27593cf740 1 --2- 172.21.15.112:0/2286798241 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55e96837f250 0x55e9683816e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.310+0000 7f27593cf740 1 --2- 172.21.15.112:0/2286798241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369d30 0x55e968373860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.310+0000 7f27593cf740 1 --2- 172.21.15.112:0/2286798241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e968369340 0x55e968373320 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.310+0000 7f272f7fe640 0 client.15909 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.310+0000 7f27593cf740 1 -- 172.21.15.112:0/2286798241 >> 172.21.15.112:0/2286798241 conn(0x55e9683648b0 msgr2=0x55e968363bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.310+0000 7f27593cf740 1 -- 172.21.15.112:0/2286798241 shutdown_connections 2024-03-16T09:17:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.310+0000 7f27593cf740 1 -- 172.21.15.112:0/2286798241 wait complete. 2024-03-16T09:17:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.310+0000 7f27593cf740 20 client.15909 trim_cache size 0 max 16384 2024-03-16T09:17:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.325+0000 7f27593cf740 1 Processor -- start 2024-03-16T09:17:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.325+0000 7f27593cf740 1 -- start start 2024-03-16T09:17:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.325+0000 7f27593cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369340 0x55e968372610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.325+0000 7f27593cf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e968369d30 0x55e968372b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.325+0000 7f27593cf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e96836eea0 0x55e968373090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.325+0000 7f27593cf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27501ab550 con 0x55e96836eea0 2024-03-16T09:17:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.325+0000 7f27593cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f27501a8dc0 con 0x55e968369340 2024-03-16T09:17:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.325+0000 7f27593cf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f27501a6c10 con 0x55e968369d30 2024-03-16T09:17:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.326+0000 7f2754cb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e96836eea0 0x55e968373090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.326+0000 7f2754cb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e96836eea0 0x55e968373090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45596/0 (socket says 172.21.15.112:45596) 2024-03-16T09:17:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.326+0000 7f2754cb0640 1 -- 172.21.15.112:0/851698443 learned_addr learned my addr 172.21.15.112:0/851698443 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.326+0000 7f274ffff640 1 --2- 172.21.15.112:0/851698443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369340 0x55e968372610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.326+0000 7f274f7fe640 1 --2- 172.21.15.112:0/851698443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e968369d30 0x55e968372b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.326+0000 7f2754cb0640 1 -- 172.21.15.112:0/851698443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e968369d30 msgr2=0x55e968372b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.326+0000 7f2754cb0640 1 --2- 172.21.15.112:0/851698443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e968369d30 0x55e968372b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.326+0000 7f2754cb0640 1 -- 172.21.15.112:0/851698443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369340 msgr2=0x55e968372610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.326+0000 7f2754cb0640 1 --2- 172.21.15.112:0/851698443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369340 0x55e968372610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.326+0000 7f2754cb0640 1 -- 172.21.15.112:0/851698443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e968365620 con 0x55e96836eea0 2024-03-16T09:17:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.326+0000 7f2754cb0640 1 --2- 172.21.15.112:0/851698443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e96836eea0 0x55e968373090 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2748013de0 tx=0x7f2748009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9dba6b07ff6021 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.326+0000 7f2735ffb640 1 -- 172.21.15.112:0/851698443 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2748003a30 con 0x55e96836eea0 2024-03-16T09:17:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.326+0000 7f2735ffb640 1 -- 172.21.15.112:0/851698443 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2748010db0 con 0x55e96836eea0 2024-03-16T09:17:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.326+0000 7f2735ffb640 1 -- 172.21.15.112:0/851698443 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2748004390 con 0x55e96836eea0 2024-03-16T09:17:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.327+0000 7f27593cf740 1 -- 172.21.15.112:0/851698443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e96836eea0 msgr2=0x55e968373090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.327+0000 7f27593cf740 1 --2- 172.21.15.112:0/851698443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e96836eea0 0x55e968373090 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2748013de0 tx=0x7f2748009500 comp rx=0 tx=0).stop 2024-03-16T09:17:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.327+0000 7f27593cf740 1 -- 172.21.15.112:0/851698443 shutdown_connections 2024-03-16T09:17:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.327+0000 7f27593cf740 1 --2- 172.21.15.112:0/851698443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e96836eea0 0x55e968373090 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.327+0000 7f27593cf740 1 --2- 172.21.15.112:0/851698443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e968369d30 0x55e968372b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.327+0000 7f27593cf740 1 --2- 172.21.15.112:0/851698443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369340 0x55e968372610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.327+0000 7f27593cf740 1 -- 172.21.15.112:0/851698443 >> 172.21.15.112:0/851698443 conn(0x55e9683648b0 msgr2=0x55e96835fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.327+0000 7f27593cf740 1 -- 172.21.15.112:0/851698443 shutdown_connections 2024-03-16T09:17:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.327+0000 7f27593cf740 1 -- 172.21.15.112:0/851698443 wait complete. 2024-03-16T09:17:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.327+0000 7f27593cf740 1 -- start start 2024-03-16T09:17:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.327+0000 7f27593cf740 1 Processor -- start 2024-03-16T09:17:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f27593cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369340 0x55e968378150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f27593cf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e968369d30 0x55e968378690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f27593cf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e96836eea0 0x55e9683796a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f27593cf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27501a7280 con 0x55e96836eea0 2024-03-16T09:17:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f27593cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f27501a86f0 con 0x55e968369340 2024-03-16T09:17:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f27593cf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f27501a7950 con 0x55e968369d30 2024-03-16T09:17:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f27593cf740 10 client.0 _ll_register_callbacks cb 0x7f2740009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f2754cb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e96836eea0 0x55e9683796a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f274f7fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e968369d30 0x55e968378690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f274ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369340 0x55e968378150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f2754cb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e96836eea0 0x55e9683796a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45610/0 (socket says 172.21.15.112:45610) 2024-03-16T09:17:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f274f7fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e968369d30 0x55e968378690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:43856/0 (socket says 172.21.15.112:43856) 2024-03-16T09:17:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f2754cb0640 1 -- 172.21.15.112:0/3464221750 learned_addr learned my addr 172.21.15.112:0/3464221750 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f2754cb0640 1 -- 172.21.15.112:0/3464221750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e968369d30 msgr2=0x55e968378690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f2754cb0640 1 --2- 172.21.15.112:0/3464221750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e968369d30 0x55e968378690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f2754cb0640 1 -- 172.21.15.112:0/3464221750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369340 msgr2=0x55e968378150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f2754cb0640 1 --2- 172.21.15.112:0/3464221750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369340 0x55e968378150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f2754cb0640 1 -- 172.21.15.112:0/3464221750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e968371d50 con 0x55e96836eea0 2024-03-16T09:17:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f2754cb0640 1 --2- 172.21.15.112:0/3464221750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e96836eea0 0x55e9683796a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2748012010 tx=0x7f2748010ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff01286b4e1e971 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f272f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f272f7fe640 1 -- 172.21.15.112:0/3464221750 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2748003a30 con 0x55e96836eea0 2024-03-16T09:17:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f272f7fe640 1 -- 172.21.15.112:0/3464221750 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f2748009050 con 0x55e96836eea0 2024-03-16T09:17:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.328+0000 7f27593cf740 1 -- 172.21.15.112:0/3464221750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2748010db0 con 0x55e96836eea0 2024-03-16T09:17:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.329+0000 7f272f7fe640 1 -- 172.21.15.112:0/3464221750 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2748003bd0 con 0x55e96836eea0 2024-03-16T09:17:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.329+0000 7f27593cf740 10 client.15915 fetch_fsmap learned FSMap version 26 2024-03-16T09:17:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.329+0000 7f27593cf740 1 -- 172.21.15.112:0/3464221750 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e9683e5950 con 0x55e96836eea0 2024-03-16T09:17:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.329+0000 7f272f7fe640 1 -- 172.21.15.112:0/3464221750 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2748010db0 con 0x55e96836eea0 2024-03-16T09:17:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.329+0000 7f272f7fe640 1 -- 172.21.15.112:0/3464221750 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f2748020050 con 0x55e96836eea0 2024-03-16T09:17:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.329+0000 7f27593cf740 10 client.15915 fetch_fsmap finished waiting for FSMap version 26 2024-03-16T09:17:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.329+0000 7f27593cf740 10 client.15915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.329+0000 7f27593cf740 20 client.15915 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.329+0000 7f27593cf740 1 --2- 172.21.15.112:0/3464221750 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55e9683e63e0 0x55e9683e8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.329+0000 7f27593cf740 4 client.15915 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T09:17:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.329+0000 7f27593cf740 1 -- 172.21.15.112:0/3464221750 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e9682d62d0 con 0x55e9683e63e0 2024-03-16T09:17:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.329+0000 7f274ffff640 1 --2- 172.21.15.112:0/3464221750 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55e9683e63e0 0x55e9683e8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.329+0000 7f272f7fe640 10 client.15915 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:41.329+0000 7f274ffff640 1 --2- 172.21.15.112:0/3464221750 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55e9683e63e0 0x55e9683e8800 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9caffda1d9aba318 server_cookie=96f6c3ac69783bc2 in_seq=0 out_seq=0 2024-03-16T09:17:41.527 INFO:tasks.workunit.client.0.smithi112.stderr:+ tar c linux-4.0.5 2024-03-16T09:17:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.112+0000 7f272f7fe640 1 -- 172.21.15.112:0/3464221750 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e9682d62d0 con 0x55e9683e63e0 2024-03-16T09:17:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.112+0000 7f272f7fe640 10 client.15915 handle_command_reply: tid=0 2024-03-16T09:17:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.112+0000 7f27593cf740 1 client.15915 shutdown 2024-03-16T09:17:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.112+0000 7f27593cf740 1 -- 172.21.15.112:0/3464221750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e96836eea0 msgr2=0x55e9683796a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.112+0000 7f27593cf740 1 --2- 172.21.15.112:0/3464221750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e96836eea0 0x55e9683796a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2748012010 tx=0x7f2748010ae0 comp rx=0 tx=0).stop 2024-03-16T09:17:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.113+0000 7f27593cf740 1 -- 172.21.15.112:0/3464221750 shutdown_connections 2024-03-16T09:17:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.113+0000 7f27593cf740 1 --2- 172.21.15.112:0/3464221750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e96836eea0 0x55e9683796a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.113+0000 7f27593cf740 1 --2- 172.21.15.112:0/3464221750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e968369d30 0x55e968378690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.113+0000 7f27593cf740 1 --2- 172.21.15.112:0/3464221750 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55e9683e63e0 0x55e9683e8800 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.113+0000 7f272f7fe640 0 client.15915 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.113+0000 7f27593cf740 1 --2- 172.21.15.112:0/3464221750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e968369340 0x55e968378150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.113+0000 7f27593cf740 1 -- 172.21.15.112:0/3464221750 >> 172.21.15.112:0/3464221750 conn(0x55e9683648b0 msgr2=0x55e968363320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.113+0000 7f27593cf740 1 -- 172.21.15.112:0/3464221750 shutdown_connections 2024-03-16T09:17:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.113+0000 7f27593cf740 1 -- 172.21.15.112:0/3464221750 wait complete. 2024-03-16T09:17:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.113+0000 7f27593cf740 20 client.15915 trim_cache size 0 max 16384 2024-03-16T09:17:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-03-16T09:17:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-03-16T09:17:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-03-16T09:17:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-03-16T09:17:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.114+0000 7f2756713640 1 -- 172.21.15.112:0/115383694 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f27380a28a0 msgr2=0x7f27380a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.114+0000 7f2756713640 1 --2- 172.21.15.112:0/115383694 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f27380a28a0 0x7f27380a4d60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2748005fd0 tx=0x7f2748009b80 comp rx=0 tx=0).stop 2024-03-16T09:17:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.114+0000 7f2756713640 1 -- 172.21.15.112:0/115383694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2750101910 msgr2=0x7f275019c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.114+0000 7f2756713640 1 --2- 172.21.15.112:0/115383694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2750101910 0x7f275019c890 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2744009230 tx=0x7f2744004ab0 comp rx=0 tx=0).stop 2024-03-16T09:17:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.114+0000 7f2756713640 1 -- 172.21.15.112:0/115383694 shutdown_connections 2024-03-16T09:17:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.114+0000 7f2756713640 1 --2- 172.21.15.112:0/115383694 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f27380a28a0 0x7f27380a4d60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.114+0000 7f2756713640 1 --2- 172.21.15.112:0/115383694 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27501022d0 0x7f275019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.114+0000 7f2756713640 1 --2- 172.21.15.112:0/115383694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2750101910 0x7f275019c890 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.114+0000 7f2756713640 1 --2- 172.21.15.112:0/115383694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2750100720 0x7f275006ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.114+0000 7f2756713640 1 -- 172.21.15.112:0/115383694 >> 172.21.15.112:0/115383694 conn(0x7f27500fbc90 msgr2=0x7f2750104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.114+0000 7f2756713640 1 -- 172.21.15.112:0/115383694 shutdown_connections 2024-03-16T09:17:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:42.114+0000 7f2756713640 1 -- 172.21.15.112:0/115383694 wait complete. 2024-03-16T09:17:42.288 INFO:tasks.workunit.client.0.smithi112.stderr:+ rm -rf linux-4.0.5 2024-03-16T09:17:42.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:42 smithi112 ceph-mon[32307]: pgmap v413: 97 pgs: 97 active+clean; 156 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 893 op/s 2024-03-16T09:17:42.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:42 smithi194 ceph-mon[36729]: pgmap v413: 97 pgs: 97 active+clean; 156 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 893 op/s 2024-03-16T09:17:42.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:42 smithi159 ceph-mon[36739]: pgmap v413: 97 pgs: 97 active+clean; 156 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 893 op/s 2024-03-16T09:17:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:44 smithi112 ceph-mon[32307]: pgmap v414: 97 pgs: 97 active+clean; 153 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.1 MiB/s wr, 1.04k op/s 2024-03-16T09:17:44.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:44 smithi194 ceph-mon[36729]: pgmap v414: 97 pgs: 97 active+clean; 153 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.1 MiB/s wr, 1.04k op/s 2024-03-16T09:17:44.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:44 smithi159 ceph-mon[36739]: pgmap v414: 97 pgs: 97 active+clean; 153 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.1 MiB/s wr, 1.04k op/s 2024-03-16T09:17:46.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:46 smithi112 ceph-mon[32307]: pgmap v415: 97 pgs: 97 active+clean; 156 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.7 MiB/s wr, 1.22k op/s 2024-03-16T09:17:46.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:46 smithi112 ceph-mon[32307]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:17:46.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:46 smithi194 ceph-mon[36729]: pgmap v415: 97 pgs: 97 active+clean; 156 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.7 MiB/s wr, 1.22k op/s 2024-03-16T09:17:46.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:46 smithi194 ceph-mon[36729]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:17:46.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:46 smithi159 ceph-mon[36739]: pgmap v415: 97 pgs: 97 active+clean; 156 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.7 MiB/s wr, 1.22k op/s 2024-03-16T09:17:46.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:46 smithi159 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:17:48.288 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:48 smithi159 ceph-mon[36739]: pgmap v416: 97 pgs: 97 active+clean; 148 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 957 op/s 2024-03-16T09:17:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:48 smithi112 ceph-mon[32307]: pgmap v416: 97 pgs: 97 active+clean; 148 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 957 op/s 2024-03-16T09:17:48.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:48 smithi194 ceph-mon[36729]: pgmap v416: 97 pgs: 97 active+clean; 148 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 957 op/s 2024-03-16T09:17:49.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:49 smithi112 ceph-mon[32307]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:17:49.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:49 smithi194 ceph-mon[36729]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:17:49.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-16T09:17:49.439 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-16T09:17:49.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:49 smithi159 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:17:49.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.552+0000 7fea57a52640 1 -- 172.21.15.112:0/2572435294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea501006e0 msgr2=0x7fea50100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:49.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.552+0000 7fea57a52640 1 --2- 172.21.15.112:0/2572435294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea501006e0 0x7fea50100b00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009900 tx=0x7fea4c02bc30 comp rx=0 tx=0).stop 2024-03-16T09:17:49.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.552+0000 7fea57a52640 1 -- 172.21.15.112:0/2572435294 shutdown_connections 2024-03-16T09:17:49.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.552+0000 7fea57a52640 1 --2- 172.21.15.112:0/2572435294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea50102290 0x7fea50108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.552+0000 7fea57a52640 1 --2- 172.21.15.112:0/2572435294 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea501018d0 0x7fea50101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.552+0000 7fea57a52640 1 --2- 172.21.15.112:0/2572435294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea501006e0 0x7fea50100b00 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.552+0000 7fea57a52640 1 -- 172.21.15.112:0/2572435294 >> 172.21.15.112:0/2572435294 conn(0x7fea500fbc70 msgr2=0x7fea500fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:49.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.552+0000 7fea57a52640 1 -- 172.21.15.112:0/2572435294 shutdown_connections 2024-03-16T09:17:49.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.552+0000 7fea57a52640 1 -- 172.21.15.112:0/2572435294 wait complete. 2024-03-16T09:17:49.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.553+0000 7fea57a52640 1 Processor -- start 2024-03-16T09:17:49.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.553+0000 7fea57a52640 1 -- start start 2024-03-16T09:17:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.553+0000 7fea57a52640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea501006e0 0x7fea500694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.553+0000 7fea57a52640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea501018d0 0x7fea5019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.553+0000 7fea57a52640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea50102290 0x7fea5019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.553+0000 7fea57a52640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea5010c430 con 0x7fea501018d0 2024-03-16T09:17:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.553+0000 7fea57a52640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea5010c2b0 con 0x7fea50102290 2024-03-16T09:17:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.553+0000 7fea57a52640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fea5010c5b0 con 0x7fea501006e0 2024-03-16T09:17:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.553+0000 7fea54fed640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea501018d0 0x7fea5019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.553+0000 7fea54fed640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea501018d0 0x7fea5019c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52938/0 (socket says 172.21.15.112:52938) 2024-03-16T09:17:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.553+0000 7fea557ee640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea501006e0 0x7fea500694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.553+0000 7fea54fed640 1 -- 172.21.15.112:0/1893928238 learned_addr learned my addr 172.21.15.112:0/1893928238 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.553+0000 7fea55fef640 1 --2- 172.21.15.112:0/1893928238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea50102290 0x7fea5019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.554+0000 7fea54fed640 1 -- 172.21.15.112:0/1893928238 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea501006e0 msgr2=0x7fea500694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:49.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.554+0000 7fea54fed640 1 --2- 172.21.15.112:0/1893928238 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea501006e0 0x7fea500694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.554+0000 7fea54fed640 1 -- 172.21.15.112:0/1893928238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea50102290 msgr2=0x7fea5019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:49.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.554+0000 7fea54fed640 1 --2- 172.21.15.112:0/1893928238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea50102290 0x7fea5019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.554+0000 7fea54fed640 1 -- 172.21.15.112:0/1893928238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5019f8b0 con 0x7fea501018d0 2024-03-16T09:17:49.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.554+0000 7fea557ee640 1 --2- 172.21.15.112:0/1893928238 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea501006e0 0x7fea500694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:49.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.554+0000 7fea54fed640 1 --2- 172.21.15.112:0/1893928238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea501018d0 0x7fea5019c980 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fea4000e9f0 tx=0x7fea4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:49.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.554+0000 7fea467fc640 1 -- 172.21.15.112:0/1893928238 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4000cd80 con 0x7fea501018d0 2024-03-16T09:17:49.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.554+0000 7fea467fc640 1 -- 172.21.15.112:0/1893928238 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fea40010930 con 0x7fea501018d0 2024-03-16T09:17:49.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.554+0000 7fea57a52640 1 -- 172.21.15.112:0/1893928238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5019fb40 con 0x7fea501018d0 2024-03-16T09:17:49.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.554+0000 7fea467fc640 1 -- 172.21.15.112:0/1893928238 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40018b00 con 0x7fea501018d0 2024-03-16T09:17:49.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.554+0000 7fea57a52640 1 -- 172.21.15.112:0/1893928238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea501a0090 con 0x7fea501018d0 2024-03-16T09:17:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.556+0000 7fea467fc640 1 -- 172.21.15.112:0/1893928238 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7fea40018ca0 con 0x7fea501018d0 2024-03-16T09:17:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.556+0000 7fea5a63e740 1 Processor -- start 2024-03-16T09:17:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea5a63e740 1 -- start start 2024-03-16T09:17:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea467fc640 1 --2- 172.21.15.112:0/1893928238 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fea3c0a2800 0x7fea3c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea5a63e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 0x563834810590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea5a63e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834810b60 0x563834815660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea5a63e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834815cd0 0x563834818160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea5a63e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563834775500 con 0x563834810170 2024-03-16T09:17:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea5a63e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56383471dbe0 con 0x563834810b60 2024-03-16T09:17:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea5a63e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5638347715a0 con 0x563834815cd0 2024-03-16T09:17:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea55fef640 1 --2- 172.21.15.112:0/1893928238 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fea3c0a2800 0x7fea3c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea467fc640 1 -- 172.21.15.112:0/1893928238 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fea400ceda0 con 0x7fea501018d0 2024-03-16T09:17:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea557ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 0x563834810590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea557ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 0x563834810590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52954/0 (socket says 172.21.15.112:52954) 2024-03-16T09:17:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea557ee640 1 -- 172.21.15.112:0/3634935372 learned_addr learned my addr 172.21.15.112:0/3634935372 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea54fed640 1 --2- 172.21.15.112:0/3634935372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834810b60 0x563834815660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea55fef640 1 --2- 172.21.15.112:0/3634935372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834815cd0 0x563834818160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea557ee640 1 -- 172.21.15.112:0/3634935372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834815cd0 msgr2=0x563834818160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea557ee640 1 --2- 172.21.15.112:0/3634935372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834815cd0 0x563834818160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea557ee640 1 -- 172.21.15.112:0/3634935372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834810b60 msgr2=0x563834815660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea557ee640 1 --2- 172.21.15.112:0/3634935372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834810b60 0x563834815660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea557ee640 1 -- 172.21.15.112:0/3634935372 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638347314b0 con 0x563834810170 2024-03-16T09:17:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea54fed640 1 --2- 172.21.15.112:0/3634935372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834810b60 0x563834815660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:49.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea55fef640 1 --2- 172.21.15.112:0/1893928238 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fea3c0a2800 0x7fea3c0a4cc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fea48005fd0 tx=0x7fea48011bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.557+0000 7fea557ee640 1 --2- 172.21.15.112:0/3634935372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 0x563834810590 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009900 tx=0x7fea4c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4aa5a4d458a79a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.558+0000 7fea3b7fe640 1 -- 172.21.15.112:0/3634935372 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c02c5d0 con 0x563834810170 2024-03-16T09:17:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.558+0000 7fea3b7fe640 1 -- 172.21.15.112:0/3634935372 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fea4c005040 con 0x563834810170 2024-03-16T09:17:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.558+0000 7fea3b7fe640 1 -- 172.21.15.112:0/3634935372 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c0049f0 con 0x563834810170 2024-03-16T09:17:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.558+0000 7fea5a63e740 1 -- 172.21.15.112:0/3634935372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 msgr2=0x563834810590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.558+0000 7fea5a63e740 1 --2- 172.21.15.112:0/3634935372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 0x563834810590 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009900 tx=0x7fea4c009560 comp rx=0 tx=0).stop 2024-03-16T09:17:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.558+0000 7fea5a63e740 1 -- 172.21.15.112:0/3634935372 shutdown_connections 2024-03-16T09:17:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.558+0000 7fea5a63e740 1 --2- 172.21.15.112:0/3634935372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834815cd0 0x563834818160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.558+0000 7fea5a63e740 1 --2- 172.21.15.112:0/3634935372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834810b60 0x563834815660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.558+0000 7fea5a63e740 1 --2- 172.21.15.112:0/3634935372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 0x563834810590 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.558+0000 7fea5a63e740 1 -- 172.21.15.112:0/3634935372 >> 172.21.15.112:0/3634935372 conn(0x56383480b6e0 msgr2=0x56383480bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.558+0000 7fea5a63e740 1 -- 172.21.15.112:0/3634935372 shutdown_connections 2024-03-16T09:17:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.558+0000 7fea5a63e740 1 -- 172.21.15.112:0/3634935372 wait complete. 2024-03-16T09:17:49.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.559+0000 7fea5a63e740 1 -- start start 2024-03-16T09:17:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.559+0000 7fea5a63e740 1 Processor -- start 2024-03-16T09:17:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.559+0000 7fea5a63e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 0x56383481e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.559+0000 7fea5a63e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834810b60 0x56383481eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.559+0000 7fea5a63e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834815cd0 0x563834825f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.559+0000 7fea5a63e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56383481d030 con 0x563834810170 2024-03-16T09:17:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.559+0000 7fea5a63e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563834731160 con 0x563834815cd0 2024-03-16T09:17:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.559+0000 7fea5a63e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56383481d1b0 con 0x563834810b60 2024-03-16T09:17:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.559+0000 7fea5a63e740 10 client.0 _ll_register_callbacks cb 0x563834715150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.559+0000 7fea557ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 0x56383481e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.559+0000 7fea557ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 0x56383481e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52962/0 (socket says 172.21.15.112:52962) 2024-03-16T09:17:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.559+0000 7fea557ee640 1 -- 172.21.15.112:0/2925151341 learned_addr learned my addr 172.21.15.112:0/2925151341 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.559+0000 7fea54fed640 1 --2- 172.21.15.112:0/2925151341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834810b60 0x56383481eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.559+0000 7fea55fef640 1 --2- 172.21.15.112:0/2925151341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834815cd0 0x563834825f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.559+0000 7fea557ee640 1 -- 172.21.15.112:0/2925151341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834810b60 msgr2=0x56383481eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.559+0000 7fea557ee640 1 --2- 172.21.15.112:0/2925151341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834810b60 0x56383481eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.559+0000 7fea557ee640 1 -- 172.21.15.112:0/2925151341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834815cd0 msgr2=0x563834825f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.559+0000 7fea557ee640 1 --2- 172.21.15.112:0/2925151341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834815cd0 0x563834825f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.559+0000 7fea557ee640 1 -- 172.21.15.112:0/2925151341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638345537d0 con 0x563834810170 2024-03-16T09:17:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.560+0000 7fea557ee640 1 --2- 172.21.15.112:0/2925151341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 0x56383481e9b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fea4c02c170 tx=0x7fea4c0049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7270260277e001da server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.560+0000 7fea38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.560+0000 7fea38ff9640 1 -- 172.21.15.112:0/2925151341 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c02c5d0 con 0x563834810170 2024-03-16T09:17:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.560+0000 7fea38ff9640 1 -- 172.21.15.112:0/2925151341 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fea4c0344e0 con 0x563834810170 2024-03-16T09:17:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.560+0000 7fea5a63e740 1 -- 172.21.15.112:0/2925151341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea4c005040 con 0x563834810170 2024-03-16T09:17:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.560+0000 7fea38ff9640 1 -- 172.21.15.112:0/2925151341 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea4c05a050 con 0x563834810170 2024-03-16T09:17:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.560+0000 7fea5a63e740 10 client.15927 fetch_fsmap learned FSMap version 28 2024-03-16T09:17:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.560+0000 7fea5a63e740 1 -- 172.21.15.112:0/2925151341 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638344893f0 con 0x563834810170 2024-03-16T09:17:49.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.560+0000 7fea38ff9640 1 -- 172.21.15.112:0/2925151341 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fea4c005040 con 0x563834810170 2024-03-16T09:17:49.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.560+0000 7fea38ff9640 1 -- 172.21.15.112:0/2925151341 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fea4c041890 con 0x563834810170 2024-03-16T09:17:49.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.560+0000 7fea5a63e740 10 client.15927 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T09:17:49.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.560+0000 7fea5a63e740 10 client.15927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:49.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.560+0000 7fea5a63e740 20 client.15927 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:49.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.560+0000 7fea5a63e740 1 --2- 172.21.15.112:0/2925151341 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x56383482a090 0x56383482c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.560+0000 7fea5a63e740 4 client.15927 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:17:49.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.560+0000 7fea5a63e740 1 -- 172.21.15.112:0/2925151341 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563834782a40 con 0x56383482a090 2024-03-16T09:17:49.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.561+0000 7fea54fed640 1 --2- 172.21.15.112:0/2925151341 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x56383482a090 0x56383482c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.561+0000 7fea38ff9640 10 client.15927 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:49.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.561+0000 7fea54fed640 1 --2- 172.21.15.112:0/2925151341 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x56383482a090 0x56383482c480 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fba79beb83460d2 server_cookie=d787d3954e8c1749 in_seq=0 out_seq=0 2024-03-16T09:17:49.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.563+0000 7fea38ff9640 1 -- 172.21.15.112:0/2925151341 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563834782a40 con 0x56383482a090 2024-03-16T09:17:49.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.563+0000 7fea38ff9640 10 client.15927 handle_command_reply: tid=0 2024-03-16T09:17:49.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.563+0000 7fea5a63e740 1 client.15927 shutdown 2024-03-16T09:17:49.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.563+0000 7fea5a63e740 1 -- 172.21.15.112:0/2925151341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 msgr2=0x56383481e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:49.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.563+0000 7fea5a63e740 1 --2- 172.21.15.112:0/2925151341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 0x56383481e9b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fea4c02c170 tx=0x7fea4c0049e0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.563+0000 7fea5a63e740 1 -- 172.21.15.112:0/2925151341 shutdown_connections 2024-03-16T09:17:49.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.563+0000 7fea5a63e740 1 --2- 172.21.15.112:0/2925151341 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x56383482a090 0x56383482c480 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.563+0000 7fea5a63e740 1 --2- 172.21.15.112:0/2925151341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834815cd0 0x563834825f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.563+0000 7fea38ff9640 0 client.15927 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:49.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.563+0000 7fea5a63e740 1 --2- 172.21.15.112:0/2925151341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834810b60 0x56383481eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.563+0000 7fea5a63e740 1 --2- 172.21.15.112:0/2925151341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 0x56383481e9b0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.563+0000 7fea5a63e740 1 -- 172.21.15.112:0/2925151341 >> 172.21.15.112:0/2925151341 conn(0x56383480b6e0 msgr2=0x56383480a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:49.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.563+0000 7fea5a63e740 1 -- 172.21.15.112:0/2925151341 shutdown_connections 2024-03-16T09:17:49.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.563+0000 7fea5a63e740 1 -- 172.21.15.112:0/2925151341 wait complete. 2024-03-16T09:17:49.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.563+0000 7fea5a63e740 20 client.15927 trim_cache size 0 max 16384 2024-03-16T09:17:49.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.581+0000 7fea5a63e740 1 Processor -- start 2024-03-16T09:17:49.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.581+0000 7fea5a63e740 1 -- start start 2024-03-16T09:17:49.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.581+0000 7fea5a63e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 0x56383480f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.581+0000 7fea5a63e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834810b60 0x563834825ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.581+0000 7fea5a63e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834815cd0 0x56383480a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.581+0000 7fea5a63e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea501a9480 con 0x563834810170 2024-03-16T09:17:49.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.581+0000 7fea5a63e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea501a6cc0 con 0x563834815cd0 2024-03-16T09:17:49.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.581+0000 7fea5a63e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fea501a4b10 con 0x563834810b60 2024-03-16T09:17:49.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.581+0000 7fea557ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 0x56383480f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.581+0000 7fea557ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 0x56383480f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52978/0 (socket says 172.21.15.112:52978) 2024-03-16T09:17:49.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.581+0000 7fea557ee640 1 -- 172.21.15.112:0/469703095 learned_addr learned my addr 172.21.15.112:0/469703095 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:49.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.581+0000 7fea54fed640 1 --2- 172.21.15.112:0/469703095 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834810b60 0x563834825ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.581+0000 7fea55fef640 1 --2- 172.21.15.112:0/469703095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834815cd0 0x56383480a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.582+0000 7fea557ee640 1 -- 172.21.15.112:0/469703095 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834810b60 msgr2=0x563834825ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:49.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.582+0000 7fea557ee640 1 --2- 172.21.15.112:0/469703095 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834810b60 0x563834825ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.582+0000 7fea557ee640 1 -- 172.21.15.112:0/469703095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834815cd0 msgr2=0x56383480a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:49.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.582+0000 7fea557ee640 1 --2- 172.21.15.112:0/469703095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834815cd0 0x56383480a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.582+0000 7fea557ee640 1 -- 172.21.15.112:0/469703095 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638344490f0 con 0x563834810170 2024-03-16T09:17:49.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.582+0000 7fea557ee640 1 --2- 172.21.15.112:0/469703095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 0x56383480f1d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fea4c041ef0 tx=0x7fea4c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12a9f0f85b9abe6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:49.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.582+0000 7fea3affd640 1 -- 172.21.15.112:0/469703095 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c02c5d0 con 0x563834810170 2024-03-16T09:17:49.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.582+0000 7fea3affd640 1 -- 172.21.15.112:0/469703095 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fea4c004de0 con 0x563834810170 2024-03-16T09:17:49.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.582+0000 7fea3affd640 1 -- 172.21.15.112:0/469703095 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c041920 con 0x563834810170 2024-03-16T09:17:49.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.582+0000 7fea5a63e740 1 -- 172.21.15.112:0/469703095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 msgr2=0x56383480f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:49.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.582+0000 7fea5a63e740 1 --2- 172.21.15.112:0/469703095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 0x56383480f1d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fea4c041ef0 tx=0x7fea4c005fd0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea5a63e740 1 -- 172.21.15.112:0/469703095 shutdown_connections 2024-03-16T09:17:49.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea5a63e740 1 --2- 172.21.15.112:0/469703095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834815cd0 0x56383480a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea5a63e740 1 --2- 172.21.15.112:0/469703095 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834810b60 0x563834825ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea5a63e740 1 --2- 172.21.15.112:0/469703095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834810170 0x56383480f1d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea5a63e740 1 -- 172.21.15.112:0/469703095 >> 172.21.15.112:0/469703095 conn(0x56383480b6e0 msgr2=0x56383480a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea5a63e740 1 -- 172.21.15.112:0/469703095 shutdown_connections 2024-03-16T09:17:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea5a63e740 1 -- 172.21.15.112:0/469703095 wait complete. 2024-03-16T09:17:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea5a63e740 1 -- start start 2024-03-16T09:17:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea5a63e740 1 Processor -- start 2024-03-16T09:17:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea5a63e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834810170 0x563834823ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea5a63e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834810b60 0x563834824400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea5a63e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834815cd0 0x563834825410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea5a63e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea501a5180 con 0x563834815cd0 2024-03-16T09:17:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea5a63e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea501a65f0 con 0x563834810b60 2024-03-16T09:17:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea5a63e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fea501a5850 con 0x563834810170 2024-03-16T09:17:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea5a63e740 10 client.0 _ll_register_callbacks cb 0x7fea40009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:49.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea55fef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834815cd0 0x563834825410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea55fef640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834815cd0 0x563834825410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52990/0 (socket says 172.21.15.112:52990) 2024-03-16T09:17:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea55fef640 1 -- 172.21.15.112:0/3983592467 learned_addr learned my addr 172.21.15.112:0/3983592467 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea557ee640 1 --2- 172.21.15.112:0/3983592467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834810170 0x563834823ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.583+0000 7fea54fed640 1 --2- 172.21.15.112:0/3983592467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834810b60 0x563834824400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.584+0000 7fea55fef640 1 -- 172.21.15.112:0/3983592467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834810170 msgr2=0x563834823ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.584+0000 7fea55fef640 1 --2- 172.21.15.112:0/3983592467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834810170 0x563834823ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.584+0000 7fea55fef640 1 -- 172.21.15.112:0/3983592467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834810b60 msgr2=0x563834824400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.584+0000 7fea55fef640 1 --2- 172.21.15.112:0/3983592467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834810b60 0x563834824400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.584+0000 7fea55fef640 1 -- 172.21.15.112:0/3983592467 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56383473c9d0 con 0x563834815cd0 2024-03-16T09:17:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.584+0000 7fea557ee640 1 --2- 172.21.15.112:0/3983592467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834810170 0x563834823ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.584+0000 7fea55fef640 1 --2- 172.21.15.112:0/3983592467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834815cd0 0x563834825410 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fea4800fce0 tx=0x7fea48004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb58def8a1f530d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.584+0000 7fea38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.584+0000 7fea38ff9640 1 -- 172.21.15.112:0/3983592467 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4801a040 con 0x563834815cd0 2024-03-16T09:17:49.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.584+0000 7fea38ff9640 1 -- 172.21.15.112:0/3983592467 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fea48027070 con 0x563834815cd0 2024-03-16T09:17:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.584+0000 7fea5a63e740 1 -- 172.21.15.112:0/3983592467 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea4c004de0 con 0x563834815cd0 2024-03-16T09:17:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.584+0000 7fea38ff9640 1 -- 172.21.15.112:0/3983592467 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea48004b20 con 0x563834815cd0 2024-03-16T09:17:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.584+0000 7fea5a63e740 10 client.15933 fetch_fsmap learned FSMap version 28 2024-03-16T09:17:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.584+0000 7fea5a63e740 1 -- 172.21.15.112:0/3983592467 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563834890a40 con 0x563834815cd0 2024-03-16T09:17:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.584+0000 7fea38ff9640 1 -- 172.21.15.112:0/3983592467 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fea4c004de0 con 0x563834815cd0 2024-03-16T09:17:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.584+0000 7fea38ff9640 1 -- 172.21.15.112:0/3983592467 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fea4802a480 con 0x563834815cd0 2024-03-16T09:17:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.585+0000 7fea5a63e740 10 client.15933 fetch_fsmap finished waiting for FSMap version 28 2024-03-16T09:17:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.585+0000 7fea5a63e740 10 client.15933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.585+0000 7fea5a63e740 20 client.15933 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.585+0000 7fea5a63e740 1 --2- 172.21.15.112:0/3983592467 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5638348914f0 0x563834893910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.585+0000 7fea5a63e740 4 client.15933 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-16T09:17:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.585+0000 7fea5a63e740 1 -- 172.21.15.112:0/3983592467 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56383477cd90 con 0x5638348914f0 2024-03-16T09:17:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.585+0000 7fea557ee640 1 --2- 172.21.15.112:0/3983592467 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5638348914f0 0x563834893910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.585+0000 7fea38ff9640 10 client.15933 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:49.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.585+0000 7fea557ee640 1 --2- 172.21.15.112:0/3983592467 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5638348914f0 0x563834893910 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0e75245a1faaef3 server_cookie=4a9ad1e5ca2a6ff6 in_seq=0 out_seq=0 2024-03-16T09:17:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.588+0000 7fea38ff9640 1 -- 172.21.15.112:0/3983592467 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56383477cd90 con 0x5638348914f0 2024-03-16T09:17:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.588+0000 7fea38ff9640 10 client.15933 handle_command_reply: tid=0 2024-03-16T09:17:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.588+0000 7fea5a63e740 1 client.15933 shutdown 2024-03-16T09:17:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.588+0000 7fea5a63e740 1 -- 172.21.15.112:0/3983592467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834815cd0 msgr2=0x563834825410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.588+0000 7fea5a63e740 1 --2- 172.21.15.112:0/3983592467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834815cd0 0x563834825410 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fea4800fce0 tx=0x7fea48004ab0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea5a63e740 1 -- 172.21.15.112:0/3983592467 shutdown_connections 2024-03-16T09:17:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea5a63e740 1 --2- 172.21.15.112:0/3983592467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563834815cd0 0x563834825410 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea5a63e740 1 --2- 172.21.15.112:0/3983592467 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x5638348914f0 0x563834893910 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea5a63e740 1 --2- 172.21.15.112:0/3983592467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563834810b60 0x563834824400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea38ff9640 0 client.15933 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea5a63e740 1 --2- 172.21.15.112:0/3983592467 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563834810170 0x563834823ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea5a63e740 1 -- 172.21.15.112:0/3983592467 >> 172.21.15.112:0/3983592467 conn(0x56383480b6e0 msgr2=0x56383480a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:49.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea5a63e740 1 -- 172.21.15.112:0/3983592467 shutdown_connections 2024-03-16T09:17:49.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea5a63e740 1 -- 172.21.15.112:0/3983592467 wait complete. 2024-03-16T09:17:49.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea5a63e740 20 client.15933 trim_cache size 0 max 16384 2024-03-16T09:17:49.597 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:17:49.597 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-03-16T09:17:49.597 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-03-16T09:17:49.597 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:17:49.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea57a52640 1 -- 172.21.15.112:0/1893928238 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fea3c0a2800 msgr2=0x7fea3c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:49.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea57a52640 1 --2- 172.21.15.112:0/1893928238 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fea3c0a2800 0x7fea3c0a4cc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fea48005fd0 tx=0x7fea48011bc0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea57a52640 1 -- 172.21.15.112:0/1893928238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea501018d0 msgr2=0x7fea5019c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:49.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea57a52640 1 --2- 172.21.15.112:0/1893928238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea501018d0 0x7fea5019c980 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fea4000e9f0 tx=0x7fea4000eeb0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea57a52640 1 -- 172.21.15.112:0/1893928238 shutdown_connections 2024-03-16T09:17:49.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea57a52640 1 --2- 172.21.15.112:0/1893928238 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fea3c0a2800 0x7fea3c0a4cc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea57a52640 1 --2- 172.21.15.112:0/1893928238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea50102290 0x7fea5019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea57a52640 1 --2- 172.21.15.112:0/1893928238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea501018d0 0x7fea5019c980 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea57a52640 1 --2- 172.21.15.112:0/1893928238 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea501006e0 0x7fea500694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea57a52640 1 -- 172.21.15.112:0/1893928238 >> 172.21.15.112:0/1893928238 conn(0x7fea500fbc70 msgr2=0x7fea50104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:49.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea57a52640 1 -- 172.21.15.112:0/1893928238 shutdown_connections 2024-03-16T09:17:49.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.589+0000 7fea57a52640 1 -- 172.21.15.112:0/1893928238 wait complete. 2024-03-16T09:17:49.603 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-16T09:17:49.603 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:17:49.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.705+0000 7f5f9d685640 1 -- 172.21.15.112:0/3061919409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f981022b0 msgr2=0x7f5f98108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:49.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.705+0000 7f5f9d685640 1 --2- 172.21.15.112:0/3061919409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f981022b0 0x7f5f98108b40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009ca0 tx=0x7f5f9002c0f0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.705+0000 7f5f9d685640 1 -- 172.21.15.112:0/3061919409 shutdown_connections 2024-03-16T09:17:49.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.705+0000 7f5f9d685640 1 --2- 172.21.15.112:0/3061919409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f981022b0 0x7f5f98108b40 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.705+0000 7f5f9d685640 1 --2- 172.21.15.112:0/3061919409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f981018f0 0x7f5f98101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.705+0000 7f5f9d685640 1 --2- 172.21.15.112:0/3061919409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f98100700 0x7f5f98100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.705+0000 7f5f9d685640 1 -- 172.21.15.112:0/3061919409 >> 172.21.15.112:0/3061919409 conn(0x7f5f980fbc70 msgr2=0x7f5f980fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:49.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.705+0000 7f5f9d685640 1 -- 172.21.15.112:0/3061919409 shutdown_connections 2024-03-16T09:17:49.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.705+0000 7f5f9d685640 1 -- 172.21.15.112:0/3061919409 wait complete. 2024-03-16T09:17:49.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.705+0000 7f5f9d685640 1 Processor -- start 2024-03-16T09:17:49.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.705+0000 7f5f9d685640 1 -- start start 2024-03-16T09:17:49.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.705+0000 7f5f9d685640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f98100700 0x7f5f98196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f9d685640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f981018f0 0x7f5f98196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f9d685640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f981022b0 0x7f5f9819e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f9d685640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9810c4b0 con 0x7f5f98100700 2024-03-16T09:17:49.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f9d685640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9810c330 con 0x7f5f981022b0 2024-03-16T09:17:49.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f9d685640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9810c630 con 0x7f5f981018f0 2024-03-16T09:17:49.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f96ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f98100700 0x7f5f98196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f96ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f98100700 0x7f5f98196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52996/0 (socket says 172.21.15.112:52996) 2024-03-16T09:17:49.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f96ffd640 1 -- 172.21.15.112:0/2756152397 learned_addr learned my addr 172.21.15.112:0/2756152397 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:49.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f967fc640 1 --2- 172.21.15.112:0/2756152397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f981018f0 0x7f5f98196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f977fe640 1 --2- 172.21.15.112:0/2756152397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f981022b0 0x7f5f9819e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f96ffd640 1 -- 172.21.15.112:0/2756152397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f981018f0 msgr2=0x7f5f98196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:49.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f96ffd640 1 --2- 172.21.15.112:0/2756152397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f981018f0 0x7f5f98196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f96ffd640 1 -- 172.21.15.112:0/2756152397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f981022b0 msgr2=0x7f5f9819e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:49.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f96ffd640 1 --2- 172.21.15.112:0/2756152397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f981022b0 0x7f5f9819e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f96ffd640 1 -- 172.21.15.112:0/2756152397 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9819e6c0 con 0x7f5f98100700 2024-03-16T09:17:49.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f96ffd640 1 --2- 172.21.15.112:0/2756152397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f98100700 0x7f5f98196aa0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800e9c0 tx=0x7f5f8800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:49.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f7ffff640 1 -- 172.21.15.112:0/2756152397 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8800cd80 con 0x7f5f98100700 2024-03-16T09:17:49.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f9d685640 1 -- 172.21.15.112:0/2756152397 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f9819e9b0 con 0x7f5f98100700 2024-03-16T09:17:49.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f7ffff640 1 -- 172.21.15.112:0/2756152397 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5f880035f0 con 0x7f5f98100700 2024-03-16T09:17:49.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f7ffff640 1 -- 172.21.15.112:0/2756152397 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88010690 con 0x7f5f98100700 2024-03-16T09:17:49.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.706+0000 7f5f9d685640 1 -- 172.21.15.112:0/2756152397 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f9819ef00 con 0x7f5f98100700 2024-03-16T09:17:49.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.707+0000 7f5f9d685640 1 -- 172.21.15.112:0/2756152397 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f980685d0 con 0x7f5f98100700 2024-03-16T09:17:49.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.708+0000 7f5f7ffff640 1 -- 172.21.15.112:0/2756152397 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7f5f880037a0 con 0x7f5f98100700 2024-03-16T09:17:49.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.709+0000 7f5f7ffff640 1 --2- 172.21.15.112:0/2756152397 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5f800a2800 0x7f5f800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:49.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.709+0000 7f5f967fc640 1 --2- 172.21.15.112:0/2756152397 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5f800a2800 0x7f5f800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:49.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.709+0000 7f5f7ffff640 1 -- 172.21.15.112:0/2756152397 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5f880cee70 con 0x7f5f98100700 2024-03-16T09:17:49.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.709+0000 7f5f967fc640 1 --2- 172.21.15.112:0/2756152397 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5f800a2800 0x7f5f800a4cc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5f98197fb0 tx=0x7f5f8c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:49.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.713+0000 7f5f7ffff640 1 -- 172.21.15.112:0/2756152397 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5f8809bf20 con 0x7f5f98100700 2024-03-16T09:17:49.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.935+0000 7f5f9d685640 1 -- 172.21.15.112:0/2756152397 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f981079c0 con 0x7f5f98100700 2024-03-16T09:17:49.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.936+0000 7f5f7ffff640 1 -- 172.21.15.112:0/2756152397 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11896 (secure 0 0 0) 0x7f5f8809b820 con 0x7f5f98100700 2024-03-16T09:17:49.939 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:17:49.940 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3939748854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3939748854},{"type":"v1","addr":"172.21.15.112:6839","nonce":3939748854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3118523258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3118523258},{"type":"v1","addr":"172.21.15.112:6841","nonce":3118523258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T09:13:36.450216+0000","modified":"2024-03-16T09:17:47.663904+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":62,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.194:6839/2528943309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2528943309},{"type":"v1","addr":"172.21.15.194:6839","nonce":2528943309}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/272895965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":272895965},{"type":"v1","addr":"172.21.15.159:6839","nonce":272895965}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25756":{"gid":25756,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/2561048853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":2561048853},{"type":"v1","addr":"172.21.15.194:6837","nonce":2561048853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24452,24466]},"id":1}]} 2024-03-16T09:17:49.940 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 28 2024-03-16T09:17:49.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.938+0000 7f5f9d685640 1 -- 172.21.15.112:0/2756152397 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5f800a2800 msgr2=0x7f5f800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:49.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.938+0000 7f5f9d685640 1 --2- 172.21.15.112:0/2756152397 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5f800a2800 0x7f5f800a4cc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5f98197fb0 tx=0x7f5f8c005f90 comp rx=0 tx=0).stop 2024-03-16T09:17:49.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.938+0000 7f5f9d685640 1 -- 172.21.15.112:0/2756152397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f98100700 msgr2=0x7f5f98196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:49.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.938+0000 7f5f9d685640 1 --2- 172.21.15.112:0/2756152397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f98100700 0x7f5f98196aa0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800e9c0 tx=0x7f5f8800ee80 comp rx=0 tx=0).stop 2024-03-16T09:17:49.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.938+0000 7f5f9d685640 1 -- 172.21.15.112:0/2756152397 shutdown_connections 2024-03-16T09:17:49.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.938+0000 7f5f9d685640 1 --2- 172.21.15.112:0/2756152397 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5f800a2800 0x7f5f800a4cc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.938+0000 7f5f9d685640 1 --2- 172.21.15.112:0/2756152397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f981022b0 0x7f5f9819e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.938+0000 7f5f9d685640 1 --2- 172.21.15.112:0/2756152397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f981018f0 0x7f5f98196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.938+0000 7f5f9d685640 1 --2- 172.21.15.112:0/2756152397 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f98100700 0x7f5f98196aa0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:49.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.938+0000 7f5f9d685640 1 -- 172.21.15.112:0/2756152397 >> 172.21.15.112:0/2756152397 conn(0x7f5f980fbc70 msgr2=0x7f5f98104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:49.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.938+0000 7f5f9d685640 1 -- 172.21.15.112:0/2756152397 shutdown_connections 2024-03-16T09:17:49.942 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:49.938+0000 7f5f9d685640 1 -- 172.21.15.112:0/2756152397 wait complete. 2024-03-16T09:17:49.952 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-16T09:17:50.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.058+0000 7f66d20f1640 1 -- 172.21.15.112:0/1439408322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66cc102290 msgr2=0x7f66cc108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.059+0000 7f66d20f1640 1 --2- 172.21.15.112:0/1439408322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66cc102290 0x7f66cc108b20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009ca0 tx=0x7f66c402c0f0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.059+0000 7f66d20f1640 1 -- 172.21.15.112:0/1439408322 shutdown_connections 2024-03-16T09:17:50.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.059+0000 7f66d20f1640 1 --2- 172.21.15.112:0/1439408322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66cc102290 0x7f66cc108b20 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.059+0000 7f66d20f1640 1 --2- 172.21.15.112:0/1439408322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66cc1018d0 0x7f66cc101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.059+0000 7f66d20f1640 1 --2- 172.21.15.112:0/1439408322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66cc1006e0 0x7f66cc100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.059+0000 7f66d20f1640 1 -- 172.21.15.112:0/1439408322 >> 172.21.15.112:0/1439408322 conn(0x7f66cc0fbc70 msgr2=0x7f66cc0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:50.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.059+0000 7f66d20f1640 1 -- 172.21.15.112:0/1439408322 shutdown_connections 2024-03-16T09:17:50.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.059+0000 7f66d20f1640 1 -- 172.21.15.112:0/1439408322 wait complete. 2024-03-16T09:17:50.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.059+0000 7f66d20f1640 1 Processor -- start 2024-03-16T09:17:50.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.059+0000 7f66d20f1640 1 -- start start 2024-03-16T09:17:50.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66d20f1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66cc1006e0 0x7f66cc197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66d20f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66cc1018d0 0x7f66cc198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66d20f1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66cc102290 0x7f66cc19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66d20f1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc10c480 con 0x7f66cc1018d0 2024-03-16T09:17:50.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66d20f1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc10c300 con 0x7f66cc102290 2024-03-16T09:17:50.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66d20f1640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc10c600 con 0x7f66cc1006e0 2024-03-16T09:17:50.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66caffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66cc1018d0 0x7f66cc198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66cb7fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66cc1006e0 0x7f66cc197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66caffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66cc1018d0 0x7f66cc198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53016/0 (socket says 172.21.15.112:53016) 2024-03-16T09:17:50.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66caffd640 1 -- 172.21.15.112:0/2870214168 learned_addr learned my addr 172.21.15.112:0/2870214168 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:50.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66cb7fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66cc1006e0 0x7f66cc197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:37454/0 (socket says 172.21.15.112:37454) 2024-03-16T09:17:50.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66cbfff640 1 --2- 172.21.15.112:0/2870214168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66cc102290 0x7f66cc19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66caffd640 1 -- 172.21.15.112:0/2870214168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66cc1006e0 msgr2=0x7f66cc197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66caffd640 1 --2- 172.21.15.112:0/2870214168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66cc1006e0 0x7f66cc197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66caffd640 1 -- 172.21.15.112:0/2870214168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66cc102290 msgr2=0x7f66cc19f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66caffd640 1 --2- 172.21.15.112:0/2870214168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66cc102290 0x7f66cc19f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66caffd640 1 -- 172.21.15.112:0/2870214168 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66cc19f7e0 con 0x7f66cc1018d0 2024-03-16T09:17:50.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66cb7fe640 1 --2- 172.21.15.112:0/2870214168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66cc1006e0 0x7f66cc197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:50.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66caffd640 1 --2- 172.21.15.112:0/2870214168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66cc1018d0 0x7f66cc198100 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00e9f0 tx=0x7f66bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:50.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66c8ff9640 1 -- 172.21.15.112:0/2870214168 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc00cd80 con 0x7f66cc1018d0 2024-03-16T09:17:50.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66c8ff9640 1 -- 172.21.15.112:0/2870214168 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f66bc010930 con 0x7f66cc1018d0 2024-03-16T09:17:50.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66c8ff9640 1 -- 172.21.15.112:0/2870214168 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc018b00 con 0x7f66cc1018d0 2024-03-16T09:17:50.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66d20f1640 1 -- 172.21.15.112:0/2870214168 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66cc19fad0 con 0x7f66cc1018d0 2024-03-16T09:17:50.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.060+0000 7f66d20f1640 1 -- 172.21.15.112:0/2870214168 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66cc1a0070 con 0x7f66cc1018d0 2024-03-16T09:17:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.063+0000 7f66c8ff9640 1 -- 172.21.15.112:0/2870214168 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7f66bc018ca0 con 0x7f66cc1018d0 2024-03-16T09:17:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.063+0000 7f66d4dbb740 1 Processor -- start 2024-03-16T09:17:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.063+0000 7f66d4dbb740 1 -- start start 2024-03-16T09:17:50.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.063+0000 7f66d4dbb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d93850 0x5636a4d93c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.063+0000 7f66d4dbb740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d94240 0x5636a4d98d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.063+0000 7f66d4dbb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d993b0 0x5636a4d9b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.063+0000 7f66d4dbb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5636a4cf8490 con 0x5636a4d993b0 2024-03-16T09:17:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.063+0000 7f66d4dbb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5636a4ca0c90 con 0x5636a4d93850 2024-03-16T09:17:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.063+0000 7f66d4dbb740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5636a4cf4530 con 0x5636a4d94240 2024-03-16T09:17:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66caffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d993b0 0x5636a4d9b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66cbfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d93850 0x5636a4d93c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66cb7fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d94240 0x5636a4d98d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66cbfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d93850 0x5636a4d93c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:58122/0 (socket says 172.21.15.112:58122) 2024-03-16T09:17:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66cbfff640 1 -- 172.21.15.112:0/3161493136 learned_addr learned my addr 172.21.15.112:0/3161493136 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66caffd640 1 --2- 172.21.15.112:0/3161493136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d993b0 0x5636a4d9b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53028/0 (socket says 172.21.15.112:53028) 2024-03-16T09:17:50.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66c8ff9640 1 --2- 172.21.15.112:0/2870214168 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f66b80a2880 0x7f66b80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66cbfff640 1 -- 172.21.15.112:0/3161493136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d94240 msgr2=0x5636a4d98d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66cbfff640 1 --2- 172.21.15.112:0/3161493136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d94240 0x5636a4d98d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66cbfff640 1 -- 172.21.15.112:0/3161493136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d993b0 msgr2=0x5636a4d9b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66cbfff640 1 --2- 172.21.15.112:0/3161493136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d993b0 0x5636a4d9b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66cbfff640 1 -- 172.21.15.112:0/3161493136 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636a4d5b500 con 0x5636a4d93850 2024-03-16T09:17:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66caffd640 1 --2- 172.21.15.112:0/3161493136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d993b0 0x5636a4d9b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66cbfff640 1 --2- 172.21.15.112:0/2870214168 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f66b80a2880 0x7f66b80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66c8ff9640 1 -- 172.21.15.112:0/2870214168 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f66bc0ced40 con 0x7f66cc1018d0 2024-03-16T09:17:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66cbfff640 1 --2- 172.21.15.112:0/3161493136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d93850 0x5636a4d93c70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009ca0 tx=0x7f66c4032340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a447dd22430798a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66c1ffb640 1 -- 172.21.15.112:0/3161493136 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c4039040 con 0x5636a4d93850 2024-03-16T09:17:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66cbfff640 1 --2- 172.21.15.112:0/2870214168 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f66b80a2880 0x7f66b80a4d40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f66c404c000 tx=0x7f66c4045000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66c1ffb640 1 -- 172.21.15.112:0/3161493136 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f66c4003cf0 con 0x5636a4d93850 2024-03-16T09:17:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.064+0000 7f66c1ffb640 1 -- 172.21.15.112:0/3161493136 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c404a780 con 0x5636a4d93850 2024-03-16T09:17:50.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.065+0000 7f66d4dbb740 1 -- 172.21.15.112:0/3161493136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d93850 msgr2=0x5636a4d93c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.065+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/3161493136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d93850 0x5636a4d93c70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009ca0 tx=0x7f66c4032340 comp rx=0 tx=0).stop 2024-03-16T09:17:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.065+0000 7f66d4dbb740 1 -- 172.21.15.112:0/3161493136 shutdown_connections 2024-03-16T09:17:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.065+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/3161493136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d993b0 0x5636a4d9b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.065+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/3161493136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d94240 0x5636a4d98d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.065+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/3161493136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d93850 0x5636a4d93c70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.065+0000 7f66d4dbb740 1 -- 172.21.15.112:0/3161493136 >> 172.21.15.112:0/3161493136 conn(0x5636a4d8edc0 msgr2=0x5636a4d8f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.065+0000 7f66d4dbb740 1 -- 172.21.15.112:0/3161493136 shutdown_connections 2024-03-16T09:17:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.065+0000 7f66d4dbb740 1 -- 172.21.15.112:0/3161493136 wait complete. 2024-03-16T09:17:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.065+0000 7f66d4dbb740 1 -- start start 2024-03-16T09:17:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66d4dbb740 1 Processor -- start 2024-03-16T09:17:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66d4dbb740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d93850 0x5636a4d9dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66d4dbb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d94240 0x5636a4d9e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66d4dbb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d993b0 0x5636a4da5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66d4dbb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5636a4d9c360 con 0x5636a4d993b0 2024-03-16T09:17:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66d4dbb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5636a4cb4230 con 0x5636a4d94240 2024-03-16T09:17:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66d4dbb740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5636a4d9c4e0 con 0x5636a4d93850 2024-03-16T09:17:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66d4dbb740 10 client.0 _ll_register_callbacks cb 0x5636a4cf0010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66cbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d993b0 0x5636a4da5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66caffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d94240 0x5636a4d9e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66cb7fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d93850 0x5636a4d9dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66cbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d993b0 0x5636a4da5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53040/0 (socket says 172.21.15.112:53040) 2024-03-16T09:17:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66cb7fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d93850 0x5636a4d9dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:37482/0 (socket says 172.21.15.112:37482) 2024-03-16T09:17:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66caffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d94240 0x5636a4d9e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:58124/0 (socket says 172.21.15.112:58124) 2024-03-16T09:17:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66cbfff640 1 -- 172.21.15.112:0/715128074 learned_addr learned my addr 172.21.15.112:0/715128074 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66cbfff640 1 -- 172.21.15.112:0/715128074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d93850 msgr2=0x5636a4d9dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66cbfff640 1 --2- 172.21.15.112:0/715128074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d93850 0x5636a4d9dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66cbfff640 1 -- 172.21.15.112:0/715128074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d94240 msgr2=0x5636a4d9e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66cbfff640 1 --2- 172.21.15.112:0/715128074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d94240 0x5636a4d9e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66cbfff640 1 -- 172.21.15.112:0/715128074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636a4cc3100 con 0x5636a4d993b0 2024-03-16T09:17:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66cb7fe640 1 --2- 172.21.15.112:0/715128074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d93850 0x5636a4d9dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66caffd640 1 --2- 172.21.15.112:0/715128074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d94240 0x5636a4d9e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66cbfff640 1 --2- 172.21.15.112:0/715128074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d993b0 0x5636a4da5320 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f66c4032c60 tx=0x7f66c4003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d68140942570710 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.066+0000 7f66aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.067+0000 7f66aaffd640 1 -- 172.21.15.112:0/715128074 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c4039070 con 0x5636a4d993b0 2024-03-16T09:17:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.067+0000 7f66aaffd640 1 -- 172.21.15.112:0/715128074 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f66c4004450 con 0x5636a4d993b0 2024-03-16T09:17:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.067+0000 7f66d4dbb740 1 -- 172.21.15.112:0/715128074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66c4003cf0 con 0x5636a4d993b0 2024-03-16T09:17:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.067+0000 7f66aaffd640 1 -- 172.21.15.112:0/715128074 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f66c4004650 con 0x5636a4d993b0 2024-03-16T09:17:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.067+0000 7f66d4dbb740 10 client.15951 fetch_fsmap learned FSMap version 29 2024-03-16T09:17:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.067+0000 7f66d4dbb740 1 -- 172.21.15.112:0/715128074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636a4a0c3f0 con 0x5636a4d993b0 2024-03-16T09:17:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.067+0000 7f66aaffd640 1 -- 172.21.15.112:0/715128074 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f66c4003cf0 con 0x5636a4d993b0 2024-03-16T09:17:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.067+0000 7f66aaffd640 1 -- 172.21.15.112:0/715128074 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f66c403f040 con 0x5636a4d993b0 2024-03-16T09:17:50.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.067+0000 7f66d4dbb740 10 client.15951 fetch_fsmap finished waiting for FSMap version 29 2024-03-16T09:17:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.067+0000 7f66d4dbb740 10 client.15951 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T09:17:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.067+0000 7f66d4dbb740 20 client.15951 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.067+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/715128074 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x5636a4da9810 0x5636a4dabc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.067+0000 7f66d4dbb740 4 client.15951 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:17:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.067+0000 7f66d4dbb740 1 -- 172.21.15.112:0/715128074 --> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636a4d059d0 con 0x5636a4da9810 2024-03-16T09:17:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.067+0000 7f66cb7fe640 1 --2- 172.21.15.112:0/715128074 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x5636a4da9810 0x5636a4dabc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.068+0000 7f66aaffd640 10 client.15951 ms_handle_connect on v2:172.21.15.159:6836/4023942203 2024-03-16T09:17:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.068+0000 7f66cb7fe640 1 --2- 172.21.15.112:0/715128074 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x5636a4da9810 0x5636a4dabc00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74921a3bf5aa0929 server_cookie=ab5fcb738ecfa6dd in_seq=0 out_seq=0 2024-03-16T09:17:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.068+0000 7f66aaffd640 1 -- 172.21.15.112:0/715128074 <== mds.2 v2:172.21.15.159:6836/4023942203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5636a4d059d0 con 0x5636a4da9810 2024-03-16T09:17:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.068+0000 7f66aaffd640 10 client.15951 handle_command_reply: tid=0 2024-03-16T09:17:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.068+0000 7f66d4dbb740 1 client.15951 shutdown 2024-03-16T09:17:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.069+0000 7f66d4dbb740 1 -- 172.21.15.112:0/715128074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d993b0 msgr2=0x5636a4da5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.069+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/715128074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d993b0 0x5636a4da5320 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f66c4032c60 tx=0x7f66c4003ea0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.069+0000 7f66d4dbb740 1 -- 172.21.15.112:0/715128074 shutdown_connections 2024-03-16T09:17:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.069+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/715128074 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x5636a4da9810 0x5636a4dabc00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.069+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/715128074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d993b0 0x5636a4da5320 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.069+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/715128074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d94240 0x5636a4d9e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.069+0000 7f66aaffd640 0 client.15951 ms_handle_reset on v2:172.21.15.159:6836/4023942203 2024-03-16T09:17:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.069+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/715128074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d93850 0x5636a4d9dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.069+0000 7f66d4dbb740 1 -- 172.21.15.112:0/715128074 >> 172.21.15.112:0/715128074 conn(0x5636a4d8edc0 msgr2=0x5636a4d8dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.069+0000 7f66d4dbb740 1 -- 172.21.15.112:0/715128074 shutdown_connections 2024-03-16T09:17:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.069+0000 7f66d4dbb740 1 -- 172.21.15.112:0/715128074 wait complete. 2024-03-16T09:17:50.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.069+0000 7f66d4dbb740 20 client.15951 trim_cache size 0 max 16384 2024-03-16T09:17:50.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66d4dbb740 1 Processor -- start 2024-03-16T09:17:50.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66d4dbb740 1 -- start start 2024-03-16T09:17:50.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66d4dbb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d93850 0x5636a4d95740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66d4dbb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d94240 0x5636a4d95c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66d4dbb740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d993b0 0x5636a4d8d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66d4dbb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc1ab620 con 0x5636a4d94240 2024-03-16T09:17:50.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66d4dbb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc1a8e60 con 0x5636a4d93850 2024-03-16T09:17:50.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66d4dbb740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc1a6cb0 con 0x5636a4d993b0 2024-03-16T09:17:50.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66caffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d94240 0x5636a4d95c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66caffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d94240 0x5636a4d95c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53048/0 (socket says 172.21.15.112:53048) 2024-03-16T09:17:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66cb7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d93850 0x5636a4d95740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66caffd640 1 -- 172.21.15.112:0/497897592 learned_addr learned my addr 172.21.15.112:0/497897592 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66cbfff640 1 --2- 172.21.15.112:0/497897592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d993b0 0x5636a4d8d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66caffd640 1 -- 172.21.15.112:0/497897592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d993b0 msgr2=0x5636a4d8d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66caffd640 1 --2- 172.21.15.112:0/497897592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d993b0 0x5636a4d8d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66caffd640 1 -- 172.21.15.112:0/497897592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d93850 msgr2=0x5636a4d95740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66caffd640 1 --2- 172.21.15.112:0/497897592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d93850 0x5636a4d95740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66caffd640 1 -- 172.21.15.112:0/497897592 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636a4ad67d0 con 0x5636a4d94240 2024-03-16T09:17:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66cb7fe640 1 --2- 172.21.15.112:0/497897592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d93850 0x5636a4d95740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66caffd640 1 --2- 172.21.15.112:0/497897592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d94240 0x5636a4d95c80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f66bc09b4a0 tx=0x7f66bc097b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2030adc6373718f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.082+0000 7f66c17fa640 1 -- 172.21.15.112:0/497897592 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc09b570 con 0x5636a4d94240 2024-03-16T09:17:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.083+0000 7f66c17fa640 1 -- 172.21.15.112:0/497897592 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f66bc09b7a0 con 0x5636a4d94240 2024-03-16T09:17:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.083+0000 7f66c17fa640 1 -- 172.21.15.112:0/497897592 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc09abf0 con 0x5636a4d94240 2024-03-16T09:17:50.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.083+0000 7f66d4dbb740 1 -- 172.21.15.112:0/497897592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d94240 msgr2=0x5636a4d95c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.083+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/497897592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d94240 0x5636a4d95c80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f66bc09b4a0 tx=0x7f66bc097b70 comp rx=0 tx=0).stop 2024-03-16T09:17:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.083+0000 7f66d4dbb740 1 -- 172.21.15.112:0/497897592 shutdown_connections 2024-03-16T09:17:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.083+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/497897592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d993b0 0x5636a4d8d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.083+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/497897592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d94240 0x5636a4d95c80 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.083+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/497897592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d93850 0x5636a4d95740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.083+0000 7f66d4dbb740 1 -- 172.21.15.112:0/497897592 >> 172.21.15.112:0/497897592 conn(0x5636a4d8edc0 msgr2=0x5636a4d8f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.083+0000 7f66d4dbb740 1 -- 172.21.15.112:0/497897592 shutdown_connections 2024-03-16T09:17:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.083+0000 7f66d4dbb740 1 -- 172.21.15.112:0/497897592 wait complete. 2024-03-16T09:17:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.083+0000 7f66d4dbb740 1 -- start start 2024-03-16T09:17:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66d4dbb740 1 Processor -- start 2024-03-16T09:17:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66d4dbb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d93850 0x5636a4e0a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66d4dbb740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d94240 0x5636a4e0aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66d4dbb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d993b0 0x5636a4e11f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66d4dbb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc1a7320 con 0x5636a4d93850 2024-03-16T09:17:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66d4dbb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc1a8790 con 0x5636a4d993b0 2024-03-16T09:17:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66d4dbb740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc1a79f0 con 0x5636a4d94240 2024-03-16T09:17:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66d4dbb740 10 client.0 _ll_register_callbacks cb 0x7f66bc097a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d93850 0x5636a4e0a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66cb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d93850 0x5636a4e0a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53062/0 (socket says 172.21.15.112:53062) 2024-03-16T09:17:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66cb7fe640 1 -- 172.21.15.112:0/1910528733 learned_addr learned my addr 172.21.15.112:0/1910528733 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66caffd640 1 --2- 172.21.15.112:0/1910528733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d94240 0x5636a4e0aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66cbfff640 1 --2- 172.21.15.112:0/1910528733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d993b0 0x5636a4e11f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66cb7fe640 1 -- 172.21.15.112:0/1910528733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d94240 msgr2=0x5636a4e0aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66cb7fe640 1 --2- 172.21.15.112:0/1910528733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d94240 0x5636a4e0aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66cb7fe640 1 -- 172.21.15.112:0/1910528733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d993b0 msgr2=0x5636a4e11f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66cb7fe640 1 --2- 172.21.15.112:0/1910528733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d993b0 0x5636a4e11f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66cb7fe640 1 -- 172.21.15.112:0/1910528733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636a4e12510 con 0x5636a4d93850 2024-03-16T09:17:50.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66cb7fe640 1 --2- 172.21.15.112:0/1910528733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d93850 0x5636a4e0a9a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f66b4012740 tx=0x7f66b40207d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae833379956e0a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.084+0000 7f66aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.085+0000 7f66aaffd640 1 -- 172.21.15.112:0/1910528733 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b4012e40 con 0x5636a4d93850 2024-03-16T09:17:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.085+0000 7f66aaffd640 1 -- 172.21.15.112:0/1910528733 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f66b4005020 con 0x5636a4d93850 2024-03-16T09:17:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.085+0000 7f66d4dbb740 1 -- 172.21.15.112:0/1910528733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66bc09b7a0 con 0x5636a4d93850 2024-03-16T09:17:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.085+0000 7f66aaffd640 1 -- 172.21.15.112:0/1910528733 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f66b4005220 con 0x5636a4d93850 2024-03-16T09:17:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.085+0000 7f66d4dbb740 10 client.15957 fetch_fsmap learned FSMap version 29 2024-03-16T09:17:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.085+0000 7f66d4dbb740 1 -- 172.21.15.112:0/1910528733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636a4e138e0 con 0x5636a4d93850 2024-03-16T09:17:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.085+0000 7f66aaffd640 1 -- 172.21.15.112:0/1910528733 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f66bc09b7a0 con 0x5636a4d93850 2024-03-16T09:17:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.085+0000 7f66aaffd640 1 -- 172.21.15.112:0/1910528733 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f66b4022270 con 0x5636a4d93850 2024-03-16T09:17:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.085+0000 7f66d4dbb740 10 client.15957 fetch_fsmap finished waiting for FSMap version 29 2024-03-16T09:17:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.085+0000 7f66d4dbb740 10 client.15957 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-03-16T09:17:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.085+0000 7f66d4dbb740 20 client.15957 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.085+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/1910528733 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x5636a4e14320 0x5636a4e16740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.085+0000 7f66d4dbb740 4 client.15957 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T09:17:50.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.085+0000 7f66d4dbb740 1 -- 172.21.15.112:0/1910528733 --> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636a4cffd20 con 0x5636a4e14320 2024-03-16T09:17:50.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.085+0000 7f66caffd640 1 --2- 172.21.15.112:0/1910528733 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x5636a4e14320 0x5636a4e16740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.086+0000 7f66aaffd640 10 client.15957 ms_handle_connect on v2:172.21.15.159:6836/4023942203 2024-03-16T09:17:50.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.086+0000 7f66caffd640 1 --2- 172.21.15.112:0/1910528733 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x5636a4e14320 0x5636a4e16740 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=866a4bfc3f9e4043 server_cookie=e4d2053f1d687e0f in_seq=0 out_seq=0 2024-03-16T09:17:50.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.086+0000 7f66aaffd640 1 -- 172.21.15.112:0/1910528733 <== mds.2 v2:172.21.15.159:6836/4023942203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5888 (crc 0 0 0) 0x5636a4cffd20 con 0x5636a4e14320 2024-03-16T09:17:50.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.086+0000 7f66aaffd640 10 client.15957 handle_command_reply: tid=0 2024-03-16T09:17:50.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.086+0000 7f66d4dbb740 1 client.15957 shutdown 2024-03-16T09:17:50.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.086+0000 7f66d4dbb740 1 -- 172.21.15.112:0/1910528733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d93850 msgr2=0x5636a4e0a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.086+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/1910528733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d93850 0x5636a4e0a9a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f66b4012740 tx=0x7f66b40207d0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.086+0000 7f66d4dbb740 1 -- 172.21.15.112:0/1910528733 shutdown_connections 2024-03-16T09:17:50.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.086+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/1910528733 >> [v2:172.21.15.159:6836/4023942203,v1:172.21.15.159:6837/4023942203] conn(0x5636a4e14320 0x5636a4e16740 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.086+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/1910528733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636a4d993b0 0x5636a4e11f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.086+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/1910528733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5636a4d94240 0x5636a4e0aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.086+0000 7f66aaffd640 0 client.15957 ms_handle_reset on v2:172.21.15.159:6836/4023942203 2024-03-16T09:17:50.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.086+0000 7f66d4dbb740 1 --2- 172.21.15.112:0/1910528733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5636a4d93850 0x5636a4e0a9a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.086+0000 7f66d4dbb740 1 -- 172.21.15.112:0/1910528733 >> 172.21.15.112:0/1910528733 conn(0x5636a4d8edc0 msgr2=0x5636a4d9dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:50.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.086+0000 7f66d4dbb740 1 -- 172.21.15.112:0/1910528733 shutdown_connections 2024-03-16T09:17:50.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.086+0000 7f66d4dbb740 1 -- 172.21.15.112:0/1910528733 wait complete. 2024-03-16T09:17:50.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.086+0000 7f66d4dbb740 20 client.15957 trim_cache size 0 max 16384 2024-03-16T09:17:50.098 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-03-16T09:17:50.098 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.099 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.099 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 83395569, 2024-03-16T09:17:50.099 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631953, 2024-03-16T09:17:50.099 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/README.ymfsb" 2024-03-16T09:17:50.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.099 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.099 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.099 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 487487948, 2024-03-16T09:17:50.099 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631944, 2024-03-16T09:17:50.099 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Introduction" 2024-03-16T09:17:50.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.099 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.099 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.100 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 544673768, 2024-03-16T09:17:50.100 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631949, 2024-03-16T09:17:50.100 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/PSS" 2024-03-16T09:17:50.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.100 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.100 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.100 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 589471892, 2024-03-16T09:17:50.100 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631956, 2024-03-16T09:17:50.100 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Tropez+" 2024-03-16T09:17:50.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.100 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.100 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.100 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 631965930, 2024-03-16T09:17:50.100 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631941, 2024-03-16T09:17:50.101 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/CMI8330" 2024-03-16T09:17:50.101 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.101 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.101 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.101 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 891641363, 2024-03-16T09:17:50.101 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631947, 2024-03-16T09:17:50.101 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Opti" 2024-03-16T09:17:50.101 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.101 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.101 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.101 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 1198099658, 2024-03-16T09:17:50.101 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631951, 2024-03-16T09:17:50.101 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/README.OSS" 2024-03-16T09:17:50.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.102 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.102 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.102 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 1234591597, 2024-03-16T09:17:50.102 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631943, 2024-03-16T09:17:50.102 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ESS1868" 2024-03-16T09:17:50.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.102 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.102 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.102 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 1476919553, 2024-03-16T09:17:50.102 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631955, 2024-03-16T09:17:50.102 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Soundblaster" 2024-03-16T09:17:50.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.103 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.103 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 1597495206, 2024-03-16T09:17:50.103 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631958, 2024-03-16T09:17:50.103 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/WaveArtist" 2024-03-16T09:17:50.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.103 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.103 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 1798551431, 2024-03-16T09:17:50.103 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631942, 2024-03-16T09:17:50.103 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ESS" 2024-03-16T09:17:50.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.103 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.104 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.104 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 1925064597, 2024-03-16T09:17:50.104 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631948, 2024-03-16T09:17:50.104 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/PAS16" 2024-03-16T09:17:50.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.104 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.104 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.104 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 2278602691, 2024-03-16T09:17:50.104 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631945, 2024-03-16T09:17:50.104 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/MultiSound" 2024-03-16T09:17:50.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.104 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.104 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.104 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 2358129246, 2024-03-16T09:17:50.105 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631961, 2024-03-16T09:17:50.105 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/oss-parameters.txt" 2024-03-16T09:17:50.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.105 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.105 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.105 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 2470824463, 2024-03-16T09:17:50.105 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631939, 2024-03-16T09:17:50.105 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ALS" 2024-03-16T09:17:50.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.105 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.105 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.105 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 2539037022, 2024-03-16T09:17:50.105 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631940, 2024-03-16T09:17:50.105 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/AudioExcelDSP16" 2024-03-16T09:17:50.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.106 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.106 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.106 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 2753980021, 2024-03-16T09:17:50.106 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631959, 2024-03-16T09:17:50.106 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/btaudio" 2024-03-16T09:17:50.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.106 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.106 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.106 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 3087282462, 2024-03-16T09:17:50.106 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631954, 2024-03-16T09:17:50.106 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/SoundPro" 2024-03-16T09:17:50.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.106 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.107 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.107 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 3166684771, 2024-03-16T09:17:50.107 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631962, 2024-03-16T09:17:50.107 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ultrasound" 2024-03-16T09:17:50.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.107 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.107 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.107 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 3198924672, 2024-03-16T09:17:50.107 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631952, 2024-03-16T09:17:50.107 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/README.modules" 2024-03-16T09:17:50.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.107 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.107 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.107 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 3837691752, 2024-03-16T09:17:50.107 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631957, 2024-03-16T09:17:50.108 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/VIBRA16" 2024-03-16T09:17:50.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.108 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.108 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.108 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 3899585692, 2024-03-16T09:17:50.108 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631960, 2024-03-16T09:17:50.108 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/mwave" 2024-03-16T09:17:50.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.108 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.108 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.108 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 4001822679, 2024-03-16T09:17:50.108 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631950, 2024-03-16T09:17:50.108 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/PSS-updates" 2024-03-16T09:17:50.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:17:50.108 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:17:50.109 INFO:teuthology.orchestra.run.smithi112.stdout: "damage_type": "backtrace", 2024-03-16T09:17:50.109 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 4247261286, 2024-03-16T09:17:50.109 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 1099511631946, 2024-03-16T09:17:50.109 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/OPL3" 2024-03-16T09:17:50.109 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:17:50.109 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-03-16T09:17:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.087+0000 7f66d20f1640 1 -- 172.21.15.112:0/2870214168 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f66b80a2880 msgr2=0x7f66b80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.087+0000 7f66d20f1640 1 --2- 172.21.15.112:0/2870214168 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f66b80a2880 0x7f66b80a4d40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f66c404c000 tx=0x7f66c4045000 comp rx=0 tx=0).stop 2024-03-16T09:17:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.087+0000 7f66d20f1640 1 -- 172.21.15.112:0/2870214168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66cc1018d0 msgr2=0x7f66cc198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.087+0000 7f66d20f1640 1 --2- 172.21.15.112:0/2870214168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66cc1018d0 0x7f66cc198100 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00e9f0 tx=0x7f66bc00eeb0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.087+0000 7f66d20f1640 1 -- 172.21.15.112:0/2870214168 shutdown_connections 2024-03-16T09:17:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.087+0000 7f66d20f1640 1 --2- 172.21.15.112:0/2870214168 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f66b80a2880 0x7f66b80a4d40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.087+0000 7f66d20f1640 1 --2- 172.21.15.112:0/2870214168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f66cc102290 0x7f66cc19f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.087+0000 7f66d20f1640 1 --2- 172.21.15.112:0/2870214168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66cc1018d0 0x7f66cc198100 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.087+0000 7f66d20f1640 1 --2- 172.21.15.112:0/2870214168 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66cc1006e0 0x7f66cc197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.087+0000 7f66d20f1640 1 -- 172.21.15.112:0/2870214168 >> 172.21.15.112:0/2870214168 conn(0x7f66cc0fbc70 msgr2=0x7f66cc0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.087+0000 7f66d20f1640 1 -- 172.21.15.112:0/2870214168 shutdown_connections 2024-03-16T09:17:50.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.087+0000 7f66d20f1640 1 -- 172.21.15.112:0/2870214168 wait complete. 2024-03-16T09:17:50.111 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-16T09:17:50.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.214+0000 7f89c7b9d640 1 -- 172.21.15.112:0/3483854196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c01006e0 msgr2=0x7f89c0100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.214+0000 7f89c7b9d640 1 --2- 172.21.15.112:0/3483854196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c01006e0 0x7f89c0100b00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009900 tx=0x7f89bc02bc30 comp rx=0 tx=0).stop 2024-03-16T09:17:50.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.214+0000 7f89c7b9d640 1 -- 172.21.15.112:0/3483854196 shutdown_connections 2024-03-16T09:17:50.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.214+0000 7f89c7b9d640 1 --2- 172.21.15.112:0/3483854196 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89c0102290 0x7f89c0108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.214+0000 7f89c7b9d640 1 --2- 172.21.15.112:0/3483854196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89c01018d0 0x7f89c0101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.214+0000 7f89c7b9d640 1 --2- 172.21.15.112:0/3483854196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c01006e0 0x7f89c0100b00 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.214+0000 7f89c7b9d640 1 -- 172.21.15.112:0/3483854196 >> 172.21.15.112:0/3483854196 conn(0x7f89c00fbc70 msgr2=0x7f89c00fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:50.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.214+0000 7f89c7b9d640 1 -- 172.21.15.112:0/3483854196 shutdown_connections 2024-03-16T09:17:50.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.214+0000 7f89c7b9d640 1 -- 172.21.15.112:0/3483854196 wait complete. 2024-03-16T09:17:50.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.215+0000 7f89c7b9d640 1 Processor -- start 2024-03-16T09:17:50.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.215+0000 7f89c7b9d640 1 -- start start 2024-03-16T09:17:50.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.215+0000 7f89c7b9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c01006e0 0x7f89c00694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.215+0000 7f89c7b9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89c01018d0 0x7f89c019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.215+0000 7f89c7b9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89c0102290 0x7f89c019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.215+0000 7f89c7b9d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89c010c440 con 0x7f89c01006e0 2024-03-16T09:17:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.215+0000 7f89c7b9d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f89c010c2c0 con 0x7f89c0102290 2024-03-16T09:17:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.215+0000 7f89c7b9d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f89c010c5c0 con 0x7f89c01018d0 2024-03-16T09:17:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.215+0000 7f89c5939640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c01006e0 0x7f89c00694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.215+0000 7f89c5939640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c01006e0 0x7f89c00694b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53080/0 (socket says 172.21.15.112:53080) 2024-03-16T09:17:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.215+0000 7f89c5939640 1 -- 172.21.15.112:0/4256919208 learned_addr learned my addr 172.21.15.112:0/4256919208 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.215+0000 7f89c5138640 1 --2- 172.21.15.112:0/4256919208 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89c01018d0 0x7f89c019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.215+0000 7f89c613a640 1 --2- 172.21.15.112:0/4256919208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89c0102290 0x7f89c019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.216+0000 7f89c5138640 1 -- 172.21.15.112:0/4256919208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89c0102290 msgr2=0x7f89c019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.216+0000 7f89c5138640 1 --2- 172.21.15.112:0/4256919208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89c0102290 0x7f89c019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.216+0000 7f89c5138640 1 -- 172.21.15.112:0/4256919208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c01006e0 msgr2=0x7f89c00694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.216+0000 7f89c5138640 1 --2- 172.21.15.112:0/4256919208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c01006e0 0x7f89c00694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.216+0000 7f89c5138640 1 -- 172.21.15.112:0/4256919208 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c019f870 con 0x7f89c01018d0 2024-03-16T09:17:50.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.216+0000 7f89c5939640 1 --2- 172.21.15.112:0/4256919208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c01006e0 0x7f89c00694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:50.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.216+0000 7f89c613a640 1 --2- 172.21.15.112:0/4256919208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89c0102290 0x7f89c019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.216+0000 7f89c5138640 1 --2- 172.21.15.112:0/4256919208 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89c01018d0 0x7f89c019c940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f89b000e9f0 tx=0x7f89b000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.216+0000 7f89b6ffd640 1 -- 172.21.15.112:0/4256919208 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b000ccc0 con 0x7f89c01018d0 2024-03-16T09:17:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.216+0000 7f89b6ffd640 1 -- 172.21.15.112:0/4256919208 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f89b0010930 con 0x7f89c01018d0 2024-03-16T09:17:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.216+0000 7f89c7b9d640 1 -- 172.21.15.112:0/4256919208 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c019fb00 con 0x7f89c01018d0 2024-03-16T09:17:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.216+0000 7f89b6ffd640 1 -- 172.21.15.112:0/4256919208 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b00189f0 con 0x7f89c01018d0 2024-03-16T09:17:50.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.216+0000 7f89c7b9d640 1 -- 172.21.15.112:0/4256919208 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c01a0050 con 0x7f89c01018d0 2024-03-16T09:17:50.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.218+0000 7f89b6ffd640 1 -- 172.21.15.112:0/4256919208 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7f89b0010ae0 con 0x7f89c01018d0 2024-03-16T09:17:50.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.219+0000 7f89ca782740 1 Processor -- start 2024-03-16T09:17:50.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.219+0000 7f89ca782740 1 -- start start 2024-03-16T09:17:50.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.219+0000 7f89ca782740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7937850 0x5567b7937c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.219+0000 7f89ca782740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b7938240 0x5567b793cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.219+0000 7f89ca782740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b793d3b0 0x5567b793f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.219+0000 7f89ca782740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5567b789c490 con 0x5567b793d3b0 2024-03-16T09:17:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.219+0000 7f89ca782740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5567b7844c90 con 0x5567b7937850 2024-03-16T09:17:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.219+0000 7f89ca782740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5567b7898530 con 0x5567b7938240 2024-03-16T09:17:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.219+0000 7f89b6ffd640 1 --2- 172.21.15.112:0/4256919208 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f89ac0a2800 0x7f89ac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.219+0000 7f89c5939640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b7938240 0x5567b793cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.219+0000 7f89c5138640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b793d3b0 0x5567b793f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.219+0000 7f89c5939640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b7938240 0x5567b793cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:37540/0 (socket says 172.21.15.112:37540) 2024-03-16T09:17:50.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.219+0000 7f89c5939640 1 -- 172.21.15.112:0/3404835325 learned_addr learned my addr 172.21.15.112:0/3404835325 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:50.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.219+0000 7f89c613a640 1 --2- 172.21.15.112:0/3404835325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7937850 0x5567b7937c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.219+0000 7f89c613a640 1 --2- 172.21.15.112:0/4256919208 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f89ac0a2800 0x7f89ac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.219+0000 7f89b6ffd640 1 -- 172.21.15.112:0/4256919208 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f89b00ced90 con 0x7f89c01018d0 2024-03-16T09:17:50.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.220+0000 7f89c5939640 1 -- 172.21.15.112:0/3404835325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7937850 msgr2=0x5567b7937c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.220+0000 7f89c5939640 1 --2- 172.21.15.112:0/3404835325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7937850 0x5567b7937c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.220+0000 7f89c5939640 1 -- 172.21.15.112:0/3404835325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b793d3b0 msgr2=0x5567b793f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.220+0000 7f89c5939640 1 --2- 172.21.15.112:0/3404835325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b793d3b0 0x5567b793f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.220+0000 7f89c5939640 1 -- 172.21.15.112:0/3404835325 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567b78ff500 con 0x5567b7938240 2024-03-16T09:17:50.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.220+0000 7f89c613a640 1 --2- 172.21.15.112:0/4256919208 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f89ac0a2800 0x7f89ac0a4cc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x5567b7938aa0 tx=0x7f89b8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:50.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.220+0000 7f89c5939640 1 --2- 172.21.15.112:0/3404835325 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b7938240 0x5567b793cd40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f89bc000c00 tx=0x7f89bc02de90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9233fb1ddc89522f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.220+0000 7f89abfff640 1 -- 172.21.15.112:0/3404835325 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc045020 con 0x5567b7938240 2024-03-16T09:17:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.220+0000 7f89abfff640 1 -- 172.21.15.112:0/3404835325 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f89bc02c5d0 con 0x5567b7938240 2024-03-16T09:17:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.220+0000 7f89abfff640 1 -- 172.21.15.112:0/3404835325 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc004920 con 0x5567b7938240 2024-03-16T09:17:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.221+0000 7f89ca782740 1 -- 172.21.15.112:0/3404835325 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b7938240 msgr2=0x5567b793cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.221+0000 7f89ca782740 1 --2- 172.21.15.112:0/3404835325 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b7938240 0x5567b793cd40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f89bc000c00 tx=0x7f89bc02de90 comp rx=0 tx=0).stop 2024-03-16T09:17:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.221+0000 7f89ca782740 1 -- 172.21.15.112:0/3404835325 shutdown_connections 2024-03-16T09:17:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.221+0000 7f89ca782740 1 --2- 172.21.15.112:0/3404835325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b793d3b0 0x5567b793f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.221+0000 7f89ca782740 1 --2- 172.21.15.112:0/3404835325 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b7938240 0x5567b793cd40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.221+0000 7f89ca782740 1 --2- 172.21.15.112:0/3404835325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7937850 0x5567b7937c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.221+0000 7f89ca782740 1 -- 172.21.15.112:0/3404835325 >> 172.21.15.112:0/3404835325 conn(0x5567b7932dc0 msgr2=0x5567b79331c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:50.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.221+0000 7f89ca782740 1 -- 172.21.15.112:0/3404835325 shutdown_connections 2024-03-16T09:17:50.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.221+0000 7f89ca782740 1 -- 172.21.15.112:0/3404835325 wait complete. 2024-03-16T09:17:50.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.221+0000 7f89ca782740 1 -- start start 2024-03-16T09:17:50.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.221+0000 7f89ca782740 1 Processor -- start 2024-03-16T09:17:50.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89ca782740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7937850 0x5567b7941d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89ca782740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b7938240 0x5567b7942260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89ca782740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b793d3b0 0x5567b7949260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89ca782740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5567b7940300 con 0x5567b7938240 2024-03-16T09:17:50.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89ca782740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5567b7858230 con 0x5567b7937850 2024-03-16T09:17:50.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89ca782740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5567b7940480 con 0x5567b793d3b0 2024-03-16T09:17:50.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89ca782740 10 client.0 _ll_register_callbacks cb 0x5567b7894010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:50.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89c5138640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b7938240 0x5567b7942260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89c5138640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b7938240 0x5567b7942260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53112/0 (socket says 172.21.15.112:53112) 2024-03-16T09:17:50.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89c613a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b793d3b0 0x5567b7949260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89c5138640 1 -- 172.21.15.112:0/3174879386 learned_addr learned my addr 172.21.15.112:0/3174879386 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:50.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89c5939640 1 --2- 172.21.15.112:0/3174879386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7937850 0x5567b7941d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89c5138640 1 -- 172.21.15.112:0/3174879386 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b793d3b0 msgr2=0x5567b7949260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89c5138640 1 --2- 172.21.15.112:0/3174879386 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b793d3b0 0x5567b7949260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89c5138640 1 -- 172.21.15.112:0/3174879386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7937850 msgr2=0x5567b7941d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89c5138640 1 --2- 172.21.15.112:0/3174879386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7937850 0x5567b7941d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89c5138640 1 -- 172.21.15.112:0/3174879386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567b7867100 con 0x5567b7938240 2024-03-16T09:17:50.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89c5138640 1 --2- 172.21.15.112:0/3174879386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b7938240 0x5567b7942260 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f89b00cc840 tx=0x7f89b006b2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22362e4b1d5a6912 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:50.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:50.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89a97fa640 1 -- 172.21.15.112:0/3174879386 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0099d50 con 0x5567b7938240 2024-03-16T09:17:50.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89a97fa640 1 -- 172.21.15.112:0/3174879386 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f89b00860b0 con 0x5567b7938240 2024-03-16T09:17:50.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.222+0000 7f89ca782740 1 -- 172.21.15.112:0/3174879386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89bc02c5d0 con 0x5567b7938240 2024-03-16T09:17:50.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.223+0000 7f89a97fa640 1 -- 172.21.15.112:0/3174879386 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f89b009f830 con 0x5567b7938240 2024-03-16T09:17:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.223+0000 7f89ca782740 10 client.15969 fetch_fsmap learned FSMap version 29 2024-03-16T09:17:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.223+0000 7f89ca782740 1 -- 172.21.15.112:0/3174879386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5567b75b03f0 con 0x5567b7938240 2024-03-16T09:17:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.223+0000 7f89a97fa640 1 -- 172.21.15.112:0/3174879386 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f89bc02c5d0 con 0x5567b7938240 2024-03-16T09:17:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.223+0000 7f89a97fa640 1 -- 172.21.15.112:0/3174879386 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f89b00994f0 con 0x5567b7938240 2024-03-16T09:17:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.223+0000 7f89ca782740 10 client.15969 fetch_fsmap finished waiting for FSMap version 29 2024-03-16T09:17:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.223+0000 7f89ca782740 10 client.15969 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T09:17:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.223+0000 7f89ca782740 20 client.15969 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.223+0000 7f89ca782740 1 --2- 172.21.15.112:0/3174879386 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x5567b794db00 0x5567b794ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.223+0000 7f89ca782740 4 client.15969 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:17:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.223+0000 7f89ca782740 1 -- 172.21.15.112:0/3174879386 --> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567b78a99d0 con 0x5567b794db00 2024-03-16T09:17:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.223+0000 7f89c5939640 1 --2- 172.21.15.112:0/3174879386 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x5567b794db00 0x5567b794ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.223+0000 7f89a97fa640 10 client.15969 ms_handle_connect on v2:172.21.15.194:6838/2528943309 2024-03-16T09:17:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.223+0000 7f89c5939640 1 --2- 172.21.15.112:0/3174879386 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x5567b794db00 0x5567b794ff80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e942ec692c55d2c1 server_cookie=1096609abee79079 in_seq=0 out_seq=0 2024-03-16T09:17:50.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.224+0000 7f89a97fa640 1 -- 172.21.15.112:0/3174879386 <== mds.1 v2:172.21.15.194:6838/2528943309 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5567b78a99d0 con 0x5567b794db00 2024-03-16T09:17:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.224+0000 7f89a97fa640 10 client.15969 handle_command_reply: tid=0 2024-03-16T09:17:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.224+0000 7f89ca782740 1 client.15969 shutdown 2024-03-16T09:17:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.224+0000 7f89ca782740 1 -- 172.21.15.112:0/3174879386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b7938240 msgr2=0x5567b7942260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.224+0000 7f89ca782740 1 --2- 172.21.15.112:0/3174879386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b7938240 0x5567b7942260 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f89b00cc840 tx=0x7f89b006b2b0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.224+0000 7f89ca782740 1 -- 172.21.15.112:0/3174879386 shutdown_connections 2024-03-16T09:17:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.224+0000 7f89ca782740 1 --2- 172.21.15.112:0/3174879386 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x5567b794db00 0x5567b794ff80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.224+0000 7f89ca782740 1 --2- 172.21.15.112:0/3174879386 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b793d3b0 0x5567b7949260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.224+0000 7f89ca782740 1 --2- 172.21.15.112:0/3174879386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b7938240 0x5567b7942260 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.224+0000 7f89a97fa640 0 client.15969 ms_handle_reset on v2:172.21.15.194:6838/2528943309 2024-03-16T09:17:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.224+0000 7f89ca782740 1 --2- 172.21.15.112:0/3174879386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7937850 0x5567b7941d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.224+0000 7f89ca782740 1 -- 172.21.15.112:0/3174879386 >> 172.21.15.112:0/3174879386 conn(0x5567b7932dc0 msgr2=0x5567b7931fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.224+0000 7f89ca782740 1 -- 172.21.15.112:0/3174879386 shutdown_connections 2024-03-16T09:17:50.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.224+0000 7f89ca782740 1 -- 172.21.15.112:0/3174879386 wait complete. 2024-03-16T09:17:50.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.225+0000 7f89ca782740 20 client.15969 trim_cache size 0 max 16384 2024-03-16T09:17:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89ca782740 1 Processor -- start 2024-03-16T09:17:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89ca782740 1 -- start start 2024-03-16T09:17:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89ca782740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b7937850 0x5567b7954840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89ca782740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7938240 0x5567b7949220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89ca782740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b793d3b0 0x5567b7954d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89ca782740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01ab640 con 0x5567b7937850 2024-03-16T09:17:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89ca782740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01a8e80 con 0x5567b7938240 2024-03-16T09:17:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89ca782740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01a6cd0 con 0x5567b793d3b0 2024-03-16T09:17:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89c5939640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b7937850 0x5567b7954840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89c5939640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b7937850 0x5567b7954840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53118/0 (socket says 172.21.15.112:53118) 2024-03-16T09:17:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89c5939640 1 -- 172.21.15.112:0/1740373921 learned_addr learned my addr 172.21.15.112:0/1740373921 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89c613a640 1 --2- 172.21.15.112:0/1740373921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b793d3b0 0x5567b7954d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89c5138640 1 --2- 172.21.15.112:0/1740373921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7938240 0x5567b7949220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89c5939640 1 -- 172.21.15.112:0/1740373921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b793d3b0 msgr2=0x5567b7954d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89c5939640 1 --2- 172.21.15.112:0/1740373921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b793d3b0 0x5567b7954d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89c5939640 1 -- 172.21.15.112:0/1740373921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7938240 msgr2=0x5567b7949220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89c5939640 1 --2- 172.21.15.112:0/1740373921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7938240 0x5567b7949220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89c5939640 1 -- 172.21.15.112:0/1740373921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567b767a7d0 con 0x5567b7937850 2024-03-16T09:17:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89c613a640 1 --2- 172.21.15.112:0/1740373921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b793d3b0 0x5567b7954d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:17:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89c5939640 1 --2- 172.21.15.112:0/1740373921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b7937850 0x5567b7954840 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f89bc02ca30 tx=0x7f89bc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d48f0ddaa5954fe server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.238+0000 7f89ab7fe640 1 -- 172.21.15.112:0/1740373921 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc045040 con 0x5567b7937850 2024-03-16T09:17:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.239+0000 7f89ab7fe640 1 -- 172.21.15.112:0/1740373921 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f89bc009520 con 0x5567b7937850 2024-03-16T09:17:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.239+0000 7f89ab7fe640 1 -- 172.21.15.112:0/1740373921 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc034740 con 0x5567b7937850 2024-03-16T09:17:50.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.239+0000 7f89ca782740 1 -- 172.21.15.112:0/1740373921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b7937850 msgr2=0x5567b7954840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.239+0000 7f89ca782740 1 --2- 172.21.15.112:0/1740373921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b7937850 0x5567b7954840 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f89bc02ca30 tx=0x7f89bc02bc30 comp rx=0 tx=0).stop 2024-03-16T09:17:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.239+0000 7f89ca782740 1 -- 172.21.15.112:0/1740373921 shutdown_connections 2024-03-16T09:17:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.239+0000 7f89ca782740 1 --2- 172.21.15.112:0/1740373921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b793d3b0 0x5567b7954d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.239+0000 7f89ca782740 1 --2- 172.21.15.112:0/1740373921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7938240 0x5567b7949220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.239+0000 7f89ca782740 1 --2- 172.21.15.112:0/1740373921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b7937850 0x5567b7954840 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.239+0000 7f89ca782740 1 -- 172.21.15.112:0/1740373921 >> 172.21.15.112:0/1740373921 conn(0x5567b7932dc0 msgr2=0x5567b79368b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.239+0000 7f89ca782740 1 -- 172.21.15.112:0/1740373921 shutdown_connections 2024-03-16T09:17:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.239+0000 7f89ca782740 1 -- 172.21.15.112:0/1740373921 wait complete. 2024-03-16T09:17:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.239+0000 7f89ca782740 1 -- start start 2024-03-16T09:17:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89ca782740 1 Processor -- start 2024-03-16T09:17:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89ca782740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7937850 0x5567b7946d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89ca782740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b7938240 0x5567b79472b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89ca782740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b793d3b0 0x5567b7948270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89ca782740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01a7340 con 0x5567b793d3b0 2024-03-16T09:17:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89ca782740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01a87b0 con 0x5567b7937850 2024-03-16T09:17:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89ca782740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01a7a10 con 0x5567b7938240 2024-03-16T09:17:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89ca782740 10 client.0 _ll_register_callbacks cb 0x7f89b000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89c613a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b793d3b0 0x5567b7948270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89c5138640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b7938240 0x5567b79472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89c613a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b793d3b0 0x5567b7948270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53128/0 (socket says 172.21.15.112:53128) 2024-03-16T09:17:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89c5939640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7937850 0x5567b7946d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89c613a640 1 -- 172.21.15.112:0/4258370898 learned_addr learned my addr 172.21.15.112:0/4258370898 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89c613a640 1 -- 172.21.15.112:0/4258370898 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b7938240 msgr2=0x5567b79472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89c613a640 1 --2- 172.21.15.112:0/4258370898 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b7938240 0x5567b79472b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89c613a640 1 -- 172.21.15.112:0/4258370898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7937850 msgr2=0x5567b7946d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89c613a640 1 --2- 172.21.15.112:0/4258370898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7937850 0x5567b7946d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89c613a640 1 -- 172.21.15.112:0/4258370898 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567b7948860 con 0x5567b793d3b0 2024-03-16T09:17:50.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89c613a640 1 --2- 172.21.15.112:0/4258370898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b793d3b0 0x5567b7948270 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f89b8003f40 tx=0x7f89b8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cd23bf449ad324 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.240+0000 7f89a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.241+0000 7f89a97fa640 1 -- 172.21.15.112:0/4258370898 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8004d20 con 0x5567b793d3b0 2024-03-16T09:17:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.241+0000 7f89a97fa640 1 -- 172.21.15.112:0/4258370898 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f89b801f660 con 0x5567b793d3b0 2024-03-16T09:17:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.241+0000 7f89ca782740 1 -- 172.21.15.112:0/4258370898 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89bc009520 con 0x5567b793d3b0 2024-03-16T09:17:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.241+0000 7f89a97fa640 1 -- 172.21.15.112:0/4258370898 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f89b8011920 con 0x5567b793d3b0 2024-03-16T09:17:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.241+0000 7f89ca782740 10 client.15975 fetch_fsmap learned FSMap version 29 2024-03-16T09:17:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.241+0000 7f89ca782740 1 -- 172.21.15.112:0/4258370898 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5567b79b4300 con 0x5567b793d3b0 2024-03-16T09:17:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.241+0000 7f89a97fa640 1 -- 172.21.15.112:0/4258370898 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f89bc009520 con 0x5567b793d3b0 2024-03-16T09:17:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.241+0000 7f89a97fa640 1 -- 172.21.15.112:0/4258370898 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f89b801d040 con 0x5567b793d3b0 2024-03-16T09:17:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.241+0000 7f89ca782740 10 client.15975 fetch_fsmap finished waiting for FSMap version 29 2024-03-16T09:17:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.241+0000 7f89ca782740 10 client.15975 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-16T09:17:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.241+0000 7f89ca782740 20 client.15975 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.241+0000 7f89ca782740 1 --2- 172.21.15.112:0/4258370898 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x5567b79b4d90 0x5567b79b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.241+0000 7f89ca782740 4 client.15975 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T09:17:50.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.241+0000 7f89ca782740 1 -- 172.21.15.112:0/4258370898 --> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567b78a3d20 con 0x5567b79b4d90 2024-03-16T09:17:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.241+0000 7f89c5939640 1 --2- 172.21.15.112:0/4258370898 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x5567b79b4d90 0x5567b79b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.242+0000 7f89a97fa640 10 client.15975 ms_handle_connect on v2:172.21.15.194:6838/2528943309 2024-03-16T09:17:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.242+0000 7f89c5939640 1 --2- 172.21.15.112:0/4258370898 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x5567b79b4d90 0x5567b79b71b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5e514f5edf1ff5ce server_cookie=c41173a21b3aac6f in_seq=0 out_seq=0 2024-03-16T09:17:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.242+0000 7f89a97fa640 1 -- 172.21.15.112:0/4258370898 <== mds.1 v2:172.21.15.194:6838/2528943309 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567b78a3d20 con 0x5567b79b4d90 2024-03-16T09:17:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.242+0000 7f89a97fa640 10 client.15975 handle_command_reply: tid=0 2024-03-16T09:17:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.242+0000 7f89ca782740 1 client.15975 shutdown 2024-03-16T09:17:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.242+0000 7f89ca782740 1 -- 172.21.15.112:0/4258370898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b793d3b0 msgr2=0x5567b7948270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.242+0000 7f89ca782740 1 --2- 172.21.15.112:0/4258370898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b793d3b0 0x5567b7948270 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f89b8003f40 tx=0x7f89b8004ab0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.242+0000 7f89ca782740 1 -- 172.21.15.112:0/4258370898 shutdown_connections 2024-03-16T09:17:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.242+0000 7f89ca782740 1 --2- 172.21.15.112:0/4258370898 >> [v2:172.21.15.194:6838/2528943309,v1:172.21.15.194:6839/2528943309] conn(0x5567b79b4d90 0x5567b79b71b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.242+0000 7f89ca782740 1 --2- 172.21.15.112:0/4258370898 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5567b793d3b0 0x5567b7948270 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.242+0000 7f89a97fa640 0 client.15975 ms_handle_reset on v2:172.21.15.194:6838/2528943309 2024-03-16T09:17:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.242+0000 7f89ca782740 1 --2- 172.21.15.112:0/4258370898 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5567b7938240 0x5567b79472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.242+0000 7f89ca782740 1 --2- 172.21.15.112:0/4258370898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b7937850 0x5567b7946d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.242+0000 7f89ca782740 1 -- 172.21.15.112:0/4258370898 >> 172.21.15.112:0/4258370898 conn(0x5567b7932dc0 msgr2=0x5567b7931750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:50.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.242+0000 7f89ca782740 1 -- 172.21.15.112:0/4258370898 shutdown_connections 2024-03-16T09:17:50.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.242+0000 7f89ca782740 1 -- 172.21.15.112:0/4258370898 wait complete. 2024-03-16T09:17:50.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.242+0000 7f89ca782740 20 client.15975 trim_cache size 0 max 16384 2024-03-16T09:17:50.254 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-03-16T09:17:50.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.243+0000 7f89c7b9d640 1 -- 172.21.15.112:0/4256919208 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f89ac0a2800 msgr2=0x7f89ac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.243+0000 7f89c7b9d640 1 --2- 172.21.15.112:0/4256919208 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f89ac0a2800 0x7f89ac0a4cc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x5567b7938aa0 tx=0x7f89b8009500 comp rx=0 tx=0).stop 2024-03-16T09:17:50.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.243+0000 7f89c7b9d640 1 -- 172.21.15.112:0/4256919208 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89c01018d0 msgr2=0x7f89c019c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.243+0000 7f89c7b9d640 1 --2- 172.21.15.112:0/4256919208 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89c01018d0 0x7f89c019c940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f89b000e9f0 tx=0x7f89b000eeb0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.243+0000 7f89c7b9d640 1 -- 172.21.15.112:0/4256919208 shutdown_connections 2024-03-16T09:17:50.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.243+0000 7f89c7b9d640 1 --2- 172.21.15.112:0/4256919208 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f89ac0a2800 0x7f89ac0a4cc0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.243+0000 7f89c7b9d640 1 --2- 172.21.15.112:0/4256919208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89c0102290 0x7f89c019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.243+0000 7f89c7b9d640 1 --2- 172.21.15.112:0/4256919208 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f89c01018d0 0x7f89c019c940 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.243+0000 7f89c7b9d640 1 --2- 172.21.15.112:0/4256919208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c01006e0 0x7f89c00694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.243+0000 7f89c7b9d640 1 -- 172.21.15.112:0/4256919208 >> 172.21.15.112:0/4256919208 conn(0x7f89c00fbc70 msgr2=0x7f89c0104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:50.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.243+0000 7f89c7b9d640 1 -- 172.21.15.112:0/4256919208 shutdown_connections 2024-03-16T09:17:50.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.243+0000 7f89c7b9d640 1 -- 172.21.15.112:0/4256919208 wait complete. 2024-03-16T09:17:50.256 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:50 smithi112 ceph-mon[32307]: pgmap v417: 97 pgs: 97 active+clean; 151 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 964 op/s 2024-03-16T09:17:50.256 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:50 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2756152397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:17:50.257 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-16T09:17:50.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.361+0000 7f9ffb4b2640 1 -- 172.21.15.112:0/567128473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff4101810 msgr2=0x7f9ff4101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.361+0000 7f9ffb4b2640 1 --2- 172.21.15.112:0/567128473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff4101810 0x7f9ff4101c30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009900 tx=0x7f9ff002bc30 comp rx=0 tx=0).stop 2024-03-16T09:17:50.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.361+0000 7f9ffb4b2640 1 -- 172.21.15.112:0/567128473 shutdown_connections 2024-03-16T09:17:50.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.361+0000 7f9ffb4b2640 1 --2- 172.21.15.112:0/567128473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ff41033c0 0x7f9ff4109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.361+0000 7f9ffb4b2640 1 --2- 172.21.15.112:0/567128473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ff4102a00 0x7f9ff4102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.361+0000 7f9ffb4b2640 1 --2- 172.21.15.112:0/567128473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff4101810 0x7f9ff4101c30 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.361+0000 7f9ffb4b2640 1 -- 172.21.15.112:0/567128473 >> 172.21.15.112:0/567128473 conn(0x7f9ff40fcd80 msgr2=0x7f9ff40ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.361+0000 7f9ffb4b2640 1 -- 172.21.15.112:0/567128473 shutdown_connections 2024-03-16T09:17:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.361+0000 7f9ffb4b2640 1 -- 172.21.15.112:0/567128473 wait complete. 2024-03-16T09:17:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.362+0000 7f9ffb4b2640 1 Processor -- start 2024-03-16T09:17:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.362+0000 7f9ffb4b2640 1 -- start start 2024-03-16T09:17:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.362+0000 7f9ffb4b2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ff4101810 0x7f9ff4197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.362+0000 7f9ffb4b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff4102a00 0x7f9ff4198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.362+0000 7f9ffb4b2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ff41033c0 0x7f9ff419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.362+0000 7f9ffb4b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff410d560 con 0x7f9ff4102a00 2024-03-16T09:17:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.362+0000 7f9ffb4b2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff410d3e0 con 0x7f9ff4101810 2024-03-16T09:17:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.362+0000 7f9ffb4b2640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff410d6e0 con 0x7f9ff41033c0 2024-03-16T09:17:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.362+0000 7f9ff8a4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff4102a00 0x7f9ff4198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.362+0000 7f9ff8a4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff4102a00 0x7f9ff4198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53146/0 (socket says 172.21.15.112:53146) 2024-03-16T09:17:50.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.362+0000 7f9ff8a4d640 1 -- 172.21.15.112:0/3200437044 learned_addr learned my addr 172.21.15.112:0/3200437044 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.362+0000 7f9ff924e640 1 --2- 172.21.15.112:0/3200437044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ff4101810 0x7f9ff4197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.362+0000 7f9ff9a4f640 1 --2- 172.21.15.112:0/3200437044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ff41033c0 0x7f9ff419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.362+0000 7f9ff8a4d640 1 -- 172.21.15.112:0/3200437044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ff41033c0 msgr2=0x7f9ff419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.362+0000 7f9ff8a4d640 1 --2- 172.21.15.112:0/3200437044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ff41033c0 0x7f9ff419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.362+0000 7f9ff8a4d640 1 -- 172.21.15.112:0/3200437044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ff4101810 msgr2=0x7f9ff4197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.362+0000 7f9ff8a4d640 1 --2- 172.21.15.112:0/3200437044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ff4101810 0x7f9ff4197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.362+0000 7f9ff8a4d640 1 -- 172.21.15.112:0/3200437044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff419f830 con 0x7f9ff4102a00 2024-03-16T09:17:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.362+0000 7f9ff8a4d640 1 --2- 172.21.15.112:0/3200437044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff4102a00 0x7f9ff4198160 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400e9f0 tx=0x7f9fe400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.363+0000 7f9fea7fc640 1 -- 172.21.15.112:0/3200437044 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe400cd80 con 0x7f9ff4102a00 2024-03-16T09:17:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.363+0000 7f9fea7fc640 1 -- 172.21.15.112:0/3200437044 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9fe4010930 con 0x7f9ff4102a00 2024-03-16T09:17:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.363+0000 7f9fea7fc640 1 -- 172.21.15.112:0/3200437044 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4018b00 con 0x7f9ff4102a00 2024-03-16T09:17:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.363+0000 7f9ffb4b2640 1 -- 172.21.15.112:0/3200437044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff419fb20 con 0x7f9ff4102a00 2024-03-16T09:17:50.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.363+0000 7f9ffb4b2640 1 -- 172.21.15.112:0/3200437044 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff41a0040 con 0x7f9ff4102a00 2024-03-16T09:17:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.365+0000 7f9fea7fc640 1 -- 172.21.15.112:0/3200437044 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7f9fe4018ca0 con 0x7f9ff4102a00 2024-03-16T09:17:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.365+0000 7f9ffe17b740 1 Processor -- start 2024-03-16T09:17:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.365+0000 7f9ffe17b740 1 -- start start 2024-03-16T09:17:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ffe17b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 0x55f4b3a3aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ffe17b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3b060 0x55f4b3a3f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ffe17b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3fc20 0x55f4b3a420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9fea7fc640 1 --2- 172.21.15.112:0/3200437044 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9fe00a2800 0x7f9fe00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ffe17b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f4b3996490 con 0x55f4b3a3a670 2024-03-16T09:17:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ffe17b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f4b393ec90 con 0x55f4b3a3b060 2024-03-16T09:17:50.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ffe17b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f4b3992530 con 0x55f4b3a3fc20 2024-03-16T09:17:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ff9a4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 0x55f4b3a3aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ff924e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3b060 0x55f4b3a3f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ff9a4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 0x55f4b3a3aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53158/0 (socket says 172.21.15.112:53158) 2024-03-16T09:17:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ff9a4f640 1 -- 172.21.15.112:0/34166271 learned_addr learned my addr 172.21.15.112:0/34166271 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ff8a4d640 1 --2- 172.21.15.112:0/34166271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3fc20 0x55f4b3a420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ff9a4f640 1 --2- 172.21.15.112:0/3200437044 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9fe00a2800 0x7f9fe00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9fea7fc640 1 -- 172.21.15.112:0/3200437044 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f9fe40cee30 con 0x7f9ff4102a00 2024-03-16T09:17:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ff924e640 1 -- 172.21.15.112:0/34166271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3fc20 msgr2=0x55f4b3a420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ff924e640 1 --2- 172.21.15.112:0/34166271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3fc20 0x55f4b3a420b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ff924e640 1 -- 172.21.15.112:0/34166271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 msgr2=0x55f4b3a3aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ff924e640 1 --2- 172.21.15.112:0/34166271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 0x55f4b3a3aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ff924e640 1 -- 172.21.15.112:0/34166271 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4b39f9500 con 0x55f4b3a3b060 2024-03-16T09:17:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ff9a4f640 1 --2- 172.21.15.112:0/34166271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 0x55f4b3a3aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ff8a4d640 1 --2- 172.21.15.112:0/34166271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3fc20 0x55f4b3a420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:50.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ff9a4f640 1 --2- 172.21.15.112:0/3200437044 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9fe00a2800 0x7f9fe00a4cc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55f4b3a30a00 tx=0x7f9fec009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.366+0000 7f9ff924e640 1 --2- 172.21.15.112:0/34166271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3b060 0x55f4b3a3f5b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9ff002c170 tx=0x7f9ff0009670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f8f1a3119c5ab4e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.367+0000 7f9fdf7fe640 1 -- 172.21.15.112:0/34166271 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff002c5d0 con 0x55f4b3a3b060 2024-03-16T09:17:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.367+0000 7f9fdf7fe640 1 -- 172.21.15.112:0/34166271 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9ff0004040 con 0x55f4b3a3b060 2024-03-16T09:17:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.367+0000 7f9fdf7fe640 1 -- 172.21.15.112:0/34166271 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff00046c0 con 0x55f4b3a3b060 2024-03-16T09:17:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.367+0000 7f9ffe17b740 1 -- 172.21.15.112:0/34166271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3b060 msgr2=0x55f4b3a3f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.367+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/34166271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3b060 0x55f4b3a3f5b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9ff002c170 tx=0x7f9ff0009670 comp rx=0 tx=0).stop 2024-03-16T09:17:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.367+0000 7f9ffe17b740 1 -- 172.21.15.112:0/34166271 shutdown_connections 2024-03-16T09:17:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.367+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/34166271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3fc20 0x55f4b3a420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.367+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/34166271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3b060 0x55f4b3a3f5b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.367+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/34166271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 0x55f4b3a3aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.367+0000 7f9ffe17b740 1 -- 172.21.15.112:0/34166271 >> 172.21.15.112:0/34166271 conn(0x55f4b3a2cd40 msgr2=0x55f4b3a2d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.367+0000 7f9ffe17b740 1 -- 172.21.15.112:0/34166271 shutdown_connections 2024-03-16T09:17:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.367+0000 7f9ffe17b740 1 -- 172.21.15.112:0/34166271 wait complete. 2024-03-16T09:17:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.368+0000 7f9ffe17b740 1 -- start start 2024-03-16T09:17:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.368+0000 7f9ffe17b740 1 Processor -- start 2024-03-16T09:17:50.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.368+0000 7f9ffe17b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 0x55f4b3a444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.368+0000 7f9ffe17b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3b060 0x55f4b3a44a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.368+0000 7f9ffe17b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3fc20 0x55f4b3a4ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.368+0000 7f9ffe17b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f4b3a42b70 con 0x55f4b3a3a670 2024-03-16T09:17:50.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.368+0000 7f9ffe17b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f4b3952230 con 0x55f4b3a3fc20 2024-03-16T09:17:50.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.368+0000 7f9ffe17b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f4b3a42cf0 con 0x55f4b3a3b060 2024-03-16T09:17:50.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.368+0000 7f9ffe17b740 10 client.0 _ll_register_callbacks cb 0x55f4b398e010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:50.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.368+0000 7f9ff924e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 0x55f4b3a444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.368+0000 7f9ff8a4d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3b060 0x55f4b3a44a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.368+0000 7f9ff924e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 0x55f4b3a444d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53174/0 (socket says 172.21.15.112:53174) 2024-03-16T09:17:50.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.368+0000 7f9ff924e640 1 -- 172.21.15.112:0/3069035908 learned_addr learned my addr 172.21.15.112:0/3069035908 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:50.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.368+0000 7f9ff9a4f640 1 --2- 172.21.15.112:0/3069035908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3fc20 0x55f4b3a4ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.368+0000 7f9ff924e640 1 -- 172.21.15.112:0/3069035908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3b060 msgr2=0x55f4b3a44a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.368+0000 7f9ff924e640 1 --2- 172.21.15.112:0/3069035908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3b060 0x55f4b3a44a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.368+0000 7f9ff924e640 1 -- 172.21.15.112:0/3069035908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3fc20 msgr2=0x55f4b3a4ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.368+0000 7f9ff924e640 1 --2- 172.21.15.112:0/3069035908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3fc20 0x55f4b3a4ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.368+0000 7f9ff924e640 1 -- 172.21.15.112:0/3069035908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4b3961100 con 0x55f4b3a3a670 2024-03-16T09:17:50.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.369+0000 7f9ff8a4d640 1 --2- 172.21.15.112:0/3069035908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3b060 0x55f4b3a44a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:50.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.369+0000 7f9ff924e640 1 --2- 172.21.15.112:0/3069035908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 0x55f4b3a444d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0000c00 tx=0x7f9ff00041f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5352d8217ab74a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:50.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.369+0000 7f9fdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:50.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.369+0000 7f9fdcff9640 1 -- 172.21.15.112:0/3069035908 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff002cad0 con 0x55f4b3a3a670 2024-03-16T09:17:50.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.369+0000 7f9fdcff9640 1 -- 172.21.15.112:0/3069035908 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f9ff002cc70 con 0x55f4b3a3a670 2024-03-16T09:17:50.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.369+0000 7f9ffe17b740 1 -- 172.21.15.112:0/3069035908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ff0004040 con 0x55f4b3a3a670 2024-03-16T09:17:50.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.369+0000 7f9fdcff9640 1 -- 172.21.15.112:0/3069035908 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9ff00035f0 con 0x55f4b3a3a670 2024-03-16T09:17:50.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.369+0000 7f9ffe17b740 10 client.15987 fetch_fsmap learned FSMap version 29 2024-03-16T09:17:50.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.369+0000 7f9ffe17b740 1 -- 172.21.15.112:0/3069035908 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f4b36aa3f0 con 0x55f4b3a3a670 2024-03-16T09:17:50.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.369+0000 7f9fdcff9640 1 -- 172.21.15.112:0/3069035908 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9ff0004040 con 0x55f4b3a3a670 2024-03-16T09:17:50.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.369+0000 7f9fdcff9640 1 -- 172.21.15.112:0/3069035908 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f9ff0034950 con 0x55f4b3a3a670 2024-03-16T09:17:50.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.369+0000 7f9ffe17b740 10 client.15987 fetch_fsmap finished waiting for FSMap version 29 2024-03-16T09:17:50.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.369+0000 7f9ffe17b740 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:50.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.369+0000 7f9ffe17b740 20 client.15987 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:50.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.369+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/3069035908 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55f4b3a4ff40 0x55f4b3a52330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.369+0000 7f9ffe17b740 4 client.15987 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:17:50.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.369+0000 7f9ffe17b740 1 -- 172.21.15.112:0/3069035908 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4b39a39d0 con 0x55f4b3a4ff40 2024-03-16T09:17:50.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.370+0000 7f9ff8a4d640 1 --2- 172.21.15.112:0/3069035908 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55f4b3a4ff40 0x55f4b3a52330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.370+0000 7f9ff8a4d640 1 --2- 172.21.15.112:0/3069035908 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55f4b3a4ff40 0x55f4b3a52330 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60d95de0b9e0affb server_cookie=171f22bf9dc86339 in_seq=0 out_seq=0 2024-03-16T09:17:50.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.370+0000 7f9fdcff9640 10 client.15987 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.372+0000 7f9fdcff9640 1 -- 172.21.15.112:0/3069035908 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f4b39a39d0 con 0x55f4b3a4ff40 2024-03-16T09:17:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.372+0000 7f9fdcff9640 10 client.15987 handle_command_reply: tid=0 2024-03-16T09:17:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.372+0000 7f9ffe17b740 1 client.15987 shutdown 2024-03-16T09:17:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.372+0000 7f9ffe17b740 1 -- 172.21.15.112:0/3069035908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 msgr2=0x55f4b3a444d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.372+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/3069035908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 0x55f4b3a444d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0000c00 tx=0x7f9ff00041f0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.372+0000 7f9ffe17b740 1 -- 172.21.15.112:0/3069035908 shutdown_connections 2024-03-16T09:17:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.372+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/3069035908 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55f4b3a4ff40 0x55f4b3a52330 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.372+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/3069035908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3fc20 0x55f4b3a4ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.372+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/3069035908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3b060 0x55f4b3a44a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.372+0000 7f9fdcff9640 0 client.15987 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.372+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/3069035908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 0x55f4b3a444d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.372+0000 7f9ffe17b740 1 -- 172.21.15.112:0/3069035908 >> 172.21.15.112:0/3069035908 conn(0x55f4b3a2cd40 msgr2=0x55f4b3a39d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:50.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.372+0000 7f9ffe17b740 1 -- 172.21.15.112:0/3069035908 shutdown_connections 2024-03-16T09:17:50.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.372+0000 7f9ffe17b740 1 -- 172.21.15.112:0/3069035908 wait complete. 2024-03-16T09:17:50.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.373+0000 7f9ffe17b740 20 client.15987 trim_cache size 0 max 16384 2024-03-16T09:17:50.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9ffe17b740 1 Processor -- start 2024-03-16T09:17:50.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9ffe17b740 1 -- start start 2024-03-16T09:17:50.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9ffe17b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3a670 0x55f4b3a2bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9ffe17b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3b060 0x55f4b3a4ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9ffe17b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3fc20 0x55f4b3a43af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9ffe17b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff41a9390 con 0x55f4b3a3b060 2024-03-16T09:17:50.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9ffe17b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff41a6bd0 con 0x55f4b3a3fc20 2024-03-16T09:17:50.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9ffe17b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff41a4a20 con 0x55f4b3a3a670 2024-03-16T09:17:50.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9ff8a4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3b060 0x55f4b3a4ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9ff8a4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3b060 0x55f4b3a4ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53186/0 (socket says 172.21.15.112:53186) 2024-03-16T09:17:50.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9ff8a4d640 1 -- 172.21.15.112:0/1724921360 learned_addr learned my addr 172.21.15.112:0/1724921360 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:50.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9ff9a4f640 1 --2- 172.21.15.112:0/1724921360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3fc20 0x55f4b3a43af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9ff924e640 1 --2- 172.21.15.112:0/1724921360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3a670 0x55f4b3a2bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9ff8a4d640 1 -- 172.21.15.112:0/1724921360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3a670 msgr2=0x55f4b3a2bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9ff8a4d640 1 --2- 172.21.15.112:0/1724921360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3a670 0x55f4b3a2bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9ff8a4d640 1 -- 172.21.15.112:0/1724921360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3fc20 msgr2=0x55f4b3a43af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9ff8a4d640 1 --2- 172.21.15.112:0/1724921360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3fc20 0x55f4b3a43af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9ff8a4d640 1 -- 172.21.15.112:0/1724921360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4b37747d0 con 0x55f4b3a3b060 2024-03-16T09:17:50.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9ff8a4d640 1 --2- 172.21.15.112:0/1724921360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3b060 0x55f4b3a4ba10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400cfd0 tx=0x7f9fe40ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204b639ae9c37e95 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:50.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9fdeffd640 1 -- 172.21.15.112:0/1724921360 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe409b890 con 0x55f4b3a3b060 2024-03-16T09:17:50.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9fdeffd640 1 -- 172.21.15.112:0/1724921360 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9fe409f930 con 0x55f4b3a3b060 2024-03-16T09:17:50.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.386+0000 7f9fdeffd640 1 -- 172.21.15.112:0/1724921360 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe40b02f0 con 0x55f4b3a3b060 2024-03-16T09:17:50.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.387+0000 7f9ffe17b740 1 -- 172.21.15.112:0/1724921360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3b060 msgr2=0x55f4b3a4ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.387+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/1724921360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3b060 0x55f4b3a4ba10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400cfd0 tx=0x7f9fe40ce040 comp rx=0 tx=0).stop 2024-03-16T09:17:50.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.387+0000 7f9ffe17b740 1 -- 172.21.15.112:0/1724921360 shutdown_connections 2024-03-16T09:17:50.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.387+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/1724921360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3fc20 0x55f4b3a43af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.387+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/1724921360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3b060 0x55f4b3a4ba10 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.387+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/1724921360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3a670 0x55f4b3a2bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.387+0000 7f9ffe17b740 1 -- 172.21.15.112:0/1724921360 >> 172.21.15.112:0/1724921360 conn(0x55f4b3a2cd40 msgr2=0x55f4b3a2d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:50.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.387+0000 7f9ffe17b740 1 -- 172.21.15.112:0/1724921360 shutdown_connections 2024-03-16T09:17:50.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.387+0000 7f9ffe17b740 1 -- 172.21.15.112:0/1724921360 wait complete. 2024-03-16T09:17:50.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.387+0000 7f9ffe17b740 1 -- start start 2024-03-16T09:17:50.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.387+0000 7f9ffe17b740 1 Processor -- start 2024-03-16T09:17:50.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9ffe17b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 0x55f4b3a48b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9ffe17b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3b060 0x55f4b3a49040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9ffe17b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3fc20 0x55f4b3a4a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9ffe17b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff41a5090 con 0x55f4b3a3a670 2024-03-16T09:17:50.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9ffe17b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff41a6500 con 0x55f4b3a3b060 2024-03-16T09:17:50.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9ffe17b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff41a5760 con 0x55f4b3a3fc20 2024-03-16T09:17:50.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9ffe17b740 10 client.0 _ll_register_callbacks cb 0x7f9fec011460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:50.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9ff924e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 0x55f4b3a48b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9ff924e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 0x55f4b3a48b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53188/0 (socket says 172.21.15.112:53188) 2024-03-16T09:17:50.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9ff924e640 1 -- 172.21.15.112:0/1491765946 learned_addr learned my addr 172.21.15.112:0/1491765946 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:50.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9ff8a4d640 1 --2- 172.21.15.112:0/1491765946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3b060 0x55f4b3a49040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9ff9a4f640 1 --2- 172.21.15.112:0/1491765946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3fc20 0x55f4b3a4a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9ff924e640 1 -- 172.21.15.112:0/1491765946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3fc20 msgr2=0x55f4b3a4a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9ff924e640 1 --2- 172.21.15.112:0/1491765946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3fc20 0x55f4b3a4a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9ff924e640 1 -- 172.21.15.112:0/1491765946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3b060 msgr2=0x55f4b3a49040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:50.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9ff924e640 1 --2- 172.21.15.112:0/1491765946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3b060 0x55f4b3a49040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9ff924e640 1 -- 172.21.15.112:0/1491765946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4b3a4a5f0 con 0x55f4b3a3a670 2024-03-16T09:17:50.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9ff924e640 1 --2- 172.21.15.112:0/1491765946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 0x55f4b3a48b00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9ff002c900 tx=0x7f9ff0005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ec6bd5b38830ef server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:50.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9fdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:50.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9fdcff9640 1 -- 172.21.15.112:0/1491765946 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0060070 con 0x55f4b3a3a670 2024-03-16T09:17:50.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9fdcff9640 1 -- 172.21.15.112:0/1491765946 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f9ff002ccd0 con 0x55f4b3a3a670 2024-03-16T09:17:50.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.388+0000 7f9ffe17b740 1 -- 172.21.15.112:0/1491765946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fe409f930 con 0x55f4b3a3a670 2024-03-16T09:17:50.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.389+0000 7f9fdcff9640 1 -- 172.21.15.112:0/1491765946 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9ff00344e0 con 0x55f4b3a3a670 2024-03-16T09:17:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.389+0000 7f9ffe17b740 10 client.15993 fetch_fsmap learned FSMap version 29 2024-03-16T09:17:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.389+0000 7f9ffe17b740 1 -- 172.21.15.112:0/1491765946 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f4b3a4b8f0 con 0x55f4b3a3a670 2024-03-16T09:17:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.389+0000 7f9fdcff9640 1 -- 172.21.15.112:0/1491765946 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9fe409f930 con 0x55f4b3a3a670 2024-03-16T09:17:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.389+0000 7f9fdcff9640 1 -- 172.21.15.112:0/1491765946 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f9ff0063020 con 0x55f4b3a3a670 2024-03-16T09:17:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.389+0000 7f9ffe17b740 10 client.15993 fetch_fsmap finished waiting for FSMap version 29 2024-03-16T09:17:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.389+0000 7f9ffe17b740 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.389+0000 7f9ffe17b740 20 client.15993 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.389+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/1491765946 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55f4b3ab6640 0x55f4b3ab8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.389+0000 7f9ffe17b740 4 client.15993 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-16T09:17:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.389+0000 7f9ffe17b740 1 -- 172.21.15.112:0/1491765946 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4b399dd20 con 0x55f4b3ab6640 2024-03-16T09:17:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.389+0000 7f9ff8a4d640 1 --2- 172.21.15.112:0/1491765946 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55f4b3ab6640 0x55f4b3ab8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.390+0000 7f9fdcff9640 10 client.15993 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.390+0000 7f9ff8a4d640 1 --2- 172.21.15.112:0/1491765946 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55f4b3ab6640 0x55f4b3ab8a60 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de533c145bd14ff7 server_cookie=ad75fc87845714c9 in_seq=0 out_seq=0 2024-03-16T09:17:50.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:50 smithi194 ceph-mon[36729]: pgmap v417: 97 pgs: 97 active+clean; 151 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 964 op/s 2024-03-16T09:17:50.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:50 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2756152397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:17:50.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.413+0000 7f9fdcff9640 1 -- 172.21.15.112:0/1491765946 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4b399dd20 con 0x55f4b3ab6640 2024-03-16T09:17:50.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.413+0000 7f9fdcff9640 10 client.15993 handle_command_reply: tid=0 2024-03-16T09:17:50.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.413+0000 7f9ffe17b740 1 client.15993 shutdown 2024-03-16T09:17:50.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.413+0000 7f9ffe17b740 1 -- 172.21.15.112:0/1491765946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 msgr2=0x55f4b3a48b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.413+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/1491765946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 0x55f4b3a48b00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9ff002c900 tx=0x7f9ff0005be0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.413+0000 7f9ffe17b740 1 -- 172.21.15.112:0/1491765946 shutdown_connections 2024-03-16T09:17:50.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.413+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/1491765946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f4b3a3fc20 0x55f4b3a4a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.413+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/1491765946 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55f4b3ab6640 0x55f4b3ab8a60 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.413+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/1491765946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4b3a3b060 0x55f4b3a49040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.413+0000 7f9ffe17b740 1 --2- 172.21.15.112:0/1491765946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f4b3a3a670 0x55f4b3a48b00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.413+0000 7f9fdcff9640 0 client.15993 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:50.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.413+0000 7f9ffe17b740 1 -- 172.21.15.112:0/1491765946 >> 172.21.15.112:0/1491765946 conn(0x55f4b3a2cd40 msgr2=0x55f4b3a39d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:50.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.413+0000 7f9ffe17b740 1 -- 172.21.15.112:0/1491765946 shutdown_connections 2024-03-16T09:17:50.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.413+0000 7f9ffe17b740 1 -- 172.21.15.112:0/1491765946 wait complete. 2024-03-16T09:17:50.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.413+0000 7f9ffe17b740 20 client.15993 trim_cache size 0 max 16384 2024-03-16T09:17:50.418 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-03-16T09:17:50.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.414+0000 7f9ffb4b2640 1 -- 172.21.15.112:0/3200437044 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9fe00a2800 msgr2=0x7f9fe00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.414+0000 7f9ffb4b2640 1 --2- 172.21.15.112:0/3200437044 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9fe00a2800 0x7f9fe00a4cc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55f4b3a30a00 tx=0x7f9fec009500 comp rx=0 tx=0).stop 2024-03-16T09:17:50.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.414+0000 7f9ffb4b2640 1 -- 172.21.15.112:0/3200437044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff4102a00 msgr2=0x7f9ff4198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:50.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.414+0000 7f9ffb4b2640 1 --2- 172.21.15.112:0/3200437044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff4102a00 0x7f9ff4198160 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400e9f0 tx=0x7f9fe400eeb0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.414+0000 7f9ffb4b2640 1 -- 172.21.15.112:0/3200437044 shutdown_connections 2024-03-16T09:17:50.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.414+0000 7f9ffb4b2640 1 --2- 172.21.15.112:0/3200437044 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f9fe00a2800 0x7f9fe00a4cc0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.414+0000 7f9ffb4b2640 1 --2- 172.21.15.112:0/3200437044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ff41033c0 0x7f9ff419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.414+0000 7f9ffb4b2640 1 --2- 172.21.15.112:0/3200437044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ff4102a00 0x7f9ff4198160 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.414+0000 7f9ffb4b2640 1 --2- 172.21.15.112:0/3200437044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ff4101810 0x7f9ff4197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:50.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.414+0000 7f9ffb4b2640 1 -- 172.21.15.112:0/3200437044 >> 172.21.15.112:0/3200437044 conn(0x7f9ff40fcd80 msgr2=0x7f9ff4105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:50.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.414+0000 7f9ffb4b2640 1 -- 172.21.15.112:0/3200437044 shutdown_connections 2024-03-16T09:17:50.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:17:50.414+0000 7f9ffb4b2640 1 -- 172.21.15.112:0/3200437044 wait complete. 2024-03-16T09:17:50.427 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 83395569, 'ino': 1099511631953, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/README.ymfsb'}, {'damage_type': 'backtrace', 'id': 487487948, 'ino': 1099511631944, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Introduction'}, {'damage_type': 'backtrace', 'id': 544673768, 'ino': 1099511631949, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/PSS'}, {'damage_type': 'backtrace', 'id': 589471892, 'ino': 1099511631956, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Tropez+'}, {'damage_type': 'backtrace', 'id': 631965930, 'ino': 1099511631941, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/CMI8330'}, {'damage_type': 'backtrace', 'id': 891641363, 'ino': 1099511631947, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Opti'}, {'damage_type': 'backtrace', 'id': 1198099658, 'ino': 1099511631951, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/README.OSS'}, {'damage_type': 'backtrace', 'id': 1234591597, 'ino': 1099511631943, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ESS1868'}, {'damage_type': 'backtrace', 'id': 1476919553, 'ino': 1099511631955, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/Soundblaster'}, {'damage_type': 'backtrace', 'id': 1597495206, 'ino': 1099511631958, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/WaveArtist'}, {'damage_type': 'backtrace', 'id': 1798551431, 'ino': 1099511631942, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ESS'}, {'damage_type': 'backtrace', 'id': 1925064597, 'ino': 1099511631948, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/PAS16'}, {'damage_type': 'backtrace', 'id': 2278602691, 'ino': 1099511631945, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/MultiSound'}, {'damage_type': 'backtrace', 'id': 2358129246, 'ino': 1099511631961, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/oss-parameters.txt'}, {'damage_type': 'backtrace', 'id': 2470824463, 'ino': 1099511631939, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ALS'}, {'damage_type': 'backtrace', 'id': 2539037022, 'ino': 1099511631940, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/AudioExcelDSP16'}, {'damage_type': 'backtrace', 'id': 2753980021, 'ino': 1099511631959, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/btaudio'}, {'damage_type': 'backtrace', 'id': 3087282462, 'ino': 1099511631954, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/SoundPro'}, {'damage_type': 'backtrace', 'id': 3166684771, 'ino': 1099511631962, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/ultrasound'}, {'damage_type': 'backtrace', 'id': 3198924672, 'ino': 1099511631952, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/README.modules'}, {'damage_type': 'backtrace', 'id': 3837691752, 'ino': 1099511631957, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/VIBRA16'}, {'damage_type': 'backtrace', 'id': 3899585692, 'ino': 1099511631960, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/mwave'}, {'damage_type': 'backtrace', 'id': 4001822679, 'ino': 1099511631950, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/PSS-updates'}, {'damage_type': 'backtrace', 'id': 4247261286, 'ino': 1099511631946, 'path': '/volumes/qa/sv_1/c3301382-2c5f-45a7-a72f-5326fd07baff/client.0/tmp/untar_tar/linux-4.0.5/Documentation/sound/oss/OPL3'}] 2024-03-16T09:17:50.428 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-03-16T09:17:50.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:50 smithi159 ceph-mon[36739]: pgmap v417: 97 pgs: 97 active+clean; 151 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 964 op/s 2024-03-16T09:17:50.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:50 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2756152397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:17:51.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:51 smithi112 ceph-mon[32307]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:17:51.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:51 smithi194 ceph-mon[36729]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:17:51.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:51 smithi159 ceph-mon[36739]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-16T09:17:52.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:52 smithi112 ceph-mon[32307]: pgmap v418: 97 pgs: 97 active+clean; 152 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 773 op/s 2024-03-16T09:17:52.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:52 smithi194 ceph-mon[36729]: pgmap v418: 97 pgs: 97 active+clean; 152 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 773 op/s 2024-03-16T09:17:52.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:52 smithi159 ceph-mon[36739]: pgmap v418: 97 pgs: 97 active+clean; 152 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 773 op/s 2024-03-16T09:17:54.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:54 smithi112 ceph-mon[32307]: pgmap v419: 97 pgs: 97 active+clean; 154 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 748 op/s 2024-03-16T09:17:54.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:54 smithi194 ceph-mon[36729]: pgmap v419: 97 pgs: 97 active+clean; 154 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 748 op/s 2024-03-16T09:17:54.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:54 smithi159 ceph-mon[36739]: pgmap v419: 97 pgs: 97 active+clean; 154 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 748 op/s 2024-03-16T09:17:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.204+0000 7fef03f77640 1 -- 172.21.15.112:0/1330853597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc101910 msgr2=0x7feefc101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.204+0000 7fef03f77640 1 --2- 172.21.15.112:0/1330853597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc101910 0x7feefc101d90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7feeec009900 tx=0x7feeec02bc30 comp rx=0 tx=0).stop 2024-03-16T09:17:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.204+0000 7fef03f77640 1 -- 172.21.15.112:0/1330853597 shutdown_connections 2024-03-16T09:17:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.204+0000 7fef03f77640 1 --2- 172.21.15.112:0/1330853597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feefc1022d0 0x7feefc108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.204+0000 7fef03f77640 1 --2- 172.21.15.112:0/1330853597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc101910 0x7feefc101d90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.204+0000 7fef03f77640 1 --2- 172.21.15.112:0/1330853597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feefc100720 0x7feefc100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.204+0000 7fef03f77640 1 -- 172.21.15.112:0/1330853597 >> 172.21.15.112:0/1330853597 conn(0x7feefc0fbc90 msgr2=0x7feefc0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.204+0000 7fef03f77640 1 -- 172.21.15.112:0/1330853597 shutdown_connections 2024-03-16T09:17:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.204+0000 7fef03f77640 1 -- 172.21.15.112:0/1330853597 wait complete. 2024-03-16T09:17:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef03f77640 1 Processor -- start 2024-03-16T09:17:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef03f77640 1 -- start start 2024-03-16T09:17:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef03f77640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feefc100720 0x7feefc197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef03f77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc101910 0x7feefc198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef03f77640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feefc1022d0 0x7feefc19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef03f77640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feefc10c480 con 0x7feefc101910 2024-03-16T09:17:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef03f77640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feefc10c300 con 0x7feefc1022d0 2024-03-16T09:17:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef03f77640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feefc10c600 con 0x7feefc100720 2024-03-16T09:17:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef01512640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc101910 0x7feefc198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef01d13640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feefc100720 0x7feefc197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef01512640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc101910 0x7feefc198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59028/0 (socket says 172.21.15.112:59028) 2024-03-16T09:17:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef01512640 1 -- 172.21.15.112:0/429493690 learned_addr learned my addr 172.21.15.112:0/429493690 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef02514640 1 --2- 172.21.15.112:0/429493690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feefc1022d0 0x7feefc19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef01512640 1 -- 172.21.15.112:0/429493690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feefc100720 msgr2=0x7feefc197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef01512640 1 --2- 172.21.15.112:0/429493690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feefc100720 0x7feefc197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef01512640 1 -- 172.21.15.112:0/429493690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feefc1022d0 msgr2=0x7feefc19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef01512640 1 --2- 172.21.15.112:0/429493690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feefc1022d0 0x7feefc19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef01512640 1 -- 172.21.15.112:0/429493690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc19f750 con 0x7feefc101910 2024-03-16T09:17:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef01d13640 1 --2- 172.21.15.112:0/429493690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feefc100720 0x7feefc197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.205+0000 7fef01512640 1 --2- 172.21.15.112:0/429493690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc101910 0x7feefc198070 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7feeec009230 tx=0x7feeec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.206+0000 7feef2ffd640 1 -- 172.21.15.112:0/429493690 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec043020 con 0x7feefc101910 2024-03-16T09:17:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.206+0000 7fef03f77640 1 -- 172.21.15.112:0/429493690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feefc19f9e0 con 0x7feefc101910 2024-03-16T09:17:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.206+0000 7feef2ffd640 1 -- 172.21.15.112:0/429493690 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feeec004250 con 0x7feefc101910 2024-03-16T09:17:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.206+0000 7feef2ffd640 1 -- 172.21.15.112:0/429493690 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec003b50 con 0x7feefc101910 2024-03-16T09:17:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.206+0000 7fef03f77640 1 -- 172.21.15.112:0/429493690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feefc19ff80 con 0x7feefc101910 2024-03-16T09:17:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.208+0000 7feef2ffd640 1 -- 172.21.15.112:0/429493690 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7feeec046020 con 0x7feefc101910 2024-03-16T09:17:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.208+0000 7fef06c3a740 1 Processor -- start 2024-03-16T09:17:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.208+0000 7fef06c3a740 1 -- start start 2024-03-16T09:17:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.208+0000 7fef06c3a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408340 0x564120408760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.208+0000 7fef06c3a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408d30 0x56412040d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.208+0000 7fef06c3a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56412040dea0 0x564120410330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.208+0000 7fef06c3a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56412036da40 con 0x564120408d30 2024-03-16T09:17:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.208+0000 7fef06c3a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564120316920 con 0x564120408340 2024-03-16T09:17:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.208+0000 7fef06c3a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564120369ae0 con 0x56412040dea0 2024-03-16T09:17:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.208+0000 7fef01d13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408d30 0x56412040d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.209+0000 7feef2ffd640 1 --2- 172.21.15.112:0/429493690 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feee80a28a0 0x7feee80a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.209+0000 7fef02514640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408340 0x564120408760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.209+0000 7fef01d13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408d30 0x56412040d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59038/0 (socket says 172.21.15.112:59038) 2024-03-16T09:17:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.209+0000 7fef01d13640 1 -- 172.21.15.112:0/987259998 learned_addr learned my addr 172.21.15.112:0/987259998 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.209+0000 7fef02514640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408340 0x564120408760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:34280/0 (socket says 172.21.15.112:34280) 2024-03-16T09:17:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.209+0000 7fef01512640 1 --2- 172.21.15.112:0/987259998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56412040dea0 0x564120410330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.209+0000 7fef02514640 1 --2- 172.21.15.112:0/429493690 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feee80a28a0 0x7feee80a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.209+0000 7fef01d13640 1 -- 172.21.15.112:0/987259998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56412040dea0 msgr2=0x564120410330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.209+0000 7fef01d13640 1 --2- 172.21.15.112:0/987259998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56412040dea0 0x564120410330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.209+0000 7fef01d13640 1 -- 172.21.15.112:0/987259998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408340 msgr2=0x564120408760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.209+0000 7fef01d13640 1 --2- 172.21.15.112:0/987259998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408340 0x564120408760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.209+0000 7fef01d13640 1 -- 172.21.15.112:0/987259998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56412032e460 con 0x564120408d30 2024-03-16T09:17:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.209+0000 7feef2ffd640 1 -- 172.21.15.112:0/429493690 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7feeec0f0b60 con 0x7feefc101910 2024-03-16T09:17:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.209+0000 7fef02514640 1 --2- 172.21.15.112:0/987259998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408340 0x564120408760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.209+0000 7fef01d13640 1 --2- 172.21.15.112:0/987259998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408d30 0x56412040d830 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7feef80114c0 tx=0x7feef8011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e7e112bb35ded2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.209+0000 7fef02514640 1 --2- 172.21.15.112:0/429493690 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feee80a28a0 0x7feee80a4d60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x564120409590 tx=0x7feef4004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.209+0000 7feee7fff640 1 -- 172.21.15.112:0/987259998 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef8004020 con 0x564120408d30 2024-03-16T09:17:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.209+0000 7feee7fff640 1 -- 172.21.15.112:0/987259998 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feef80041c0 con 0x564120408d30 2024-03-16T09:17:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.209+0000 7feee7fff640 1 -- 172.21.15.112:0/987259998 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef800fa60 con 0x564120408d30 2024-03-16T09:17:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.210+0000 7fef06c3a740 1 -- 172.21.15.112:0/987259998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408d30 msgr2=0x56412040d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.210+0000 7fef06c3a740 1 --2- 172.21.15.112:0/987259998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408d30 0x56412040d830 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7feef80114c0 tx=0x7feef8011980 comp rx=0 tx=0).stop 2024-03-16T09:17:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.210+0000 7fef06c3a740 1 -- 172.21.15.112:0/987259998 shutdown_connections 2024-03-16T09:17:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.210+0000 7fef06c3a740 1 --2- 172.21.15.112:0/987259998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56412040dea0 0x564120410330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.210+0000 7fef06c3a740 1 --2- 172.21.15.112:0/987259998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408d30 0x56412040d830 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.210+0000 7fef06c3a740 1 --2- 172.21.15.112:0/987259998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408340 0x564120408760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.210+0000 7fef06c3a740 1 -- 172.21.15.112:0/987259998 >> 172.21.15.112:0/987259998 conn(0x5641204038b0 msgr2=0x564120403cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.210+0000 7fef06c3a740 1 -- 172.21.15.112:0/987259998 shutdown_connections 2024-03-16T09:17:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.210+0000 7fef06c3a740 1 -- 172.21.15.112:0/987259998 wait complete. 2024-03-16T09:17:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.210+0000 7fef06c3a740 1 -- start start 2024-03-16T09:17:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.210+0000 7fef06c3a740 1 Processor -- start 2024-03-16T09:17:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.211+0000 7fef06c3a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408340 0x564120412640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.211+0000 7fef06c3a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408d30 0x564120412b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.211+0000 7fef06c3a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56412040dea0 0x564120419bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.211+0000 7fef06c3a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564120410e50 con 0x564120408340 2024-03-16T09:17:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.211+0000 7fef06c3a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564120329d70 con 0x564120408d30 2024-03-16T09:17:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.211+0000 7fef06c3a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564120410fd0 con 0x56412040dea0 2024-03-16T09:17:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.211+0000 7fef06c3a740 10 client.0 _ll_register_callbacks cb 0x564120081a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.211+0000 7fef01d13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408340 0x564120412640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.211+0000 7fef01d13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408340 0x564120412640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59050/0 (socket says 172.21.15.112:59050) 2024-03-16T09:17:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.211+0000 7fef01d13640 1 -- 172.21.15.112:0/2792981484 learned_addr learned my addr 172.21.15.112:0/2792981484 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.211+0000 7fef02514640 1 --2- 172.21.15.112:0/2792981484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56412040dea0 0x564120419bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.211+0000 7fef01512640 1 --2- 172.21.15.112:0/2792981484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408d30 0x564120412b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.211+0000 7fef01d13640 1 -- 172.21.15.112:0/2792981484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56412040dea0 msgr2=0x564120419bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.211+0000 7fef01d13640 1 --2- 172.21.15.112:0/2792981484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56412040dea0 0x564120419bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.211+0000 7fef01d13640 1 -- 172.21.15.112:0/2792981484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408d30 msgr2=0x564120412b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.211+0000 7fef01d13640 1 --2- 172.21.15.112:0/2792981484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408d30 0x564120412b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.211+0000 7fef01d13640 1 -- 172.21.15.112:0/2792981484 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564120082210 con 0x564120408340 2024-03-16T09:17:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.211+0000 7fef01d13640 1 --2- 172.21.15.112:0/2792981484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408340 0x564120412640 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7feef8011e80 tx=0x7feef8003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5699de9545623725 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.211+0000 7feee57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.212+0000 7feee57fa640 1 -- 172.21.15.112:0/2792981484 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef801d050 con 0x564120408340 2024-03-16T09:17:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.212+0000 7feee57fa640 1 -- 172.21.15.112:0/2792981484 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7feef8005040 con 0x564120408340 2024-03-16T09:17:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.212+0000 7fef06c3a740 1 -- 172.21.15.112:0/2792981484 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feef80041c0 con 0x564120408340 2024-03-16T09:17:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.212+0000 7feee57fa640 1 -- 172.21.15.112:0/2792981484 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feef8004000 con 0x564120408340 2024-03-16T09:17:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.212+0000 7fef06c3a740 10 client.16005 fetch_fsmap learned FSMap version 29 2024-03-16T09:17:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.212+0000 7fef06c3a740 1 -- 172.21.15.112:0/2792981484 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56412014e620 con 0x564120408340 2024-03-16T09:17:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.212+0000 7feee57fa640 1 -- 172.21.15.112:0/2792981484 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feef80041c0 con 0x564120408340 2024-03-16T09:17:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.212+0000 7feee57fa640 1 -- 172.21.15.112:0/2792981484 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7feef80046c0 con 0x564120408340 2024-03-16T09:17:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.212+0000 7fef06c3a740 10 client.16005 fetch_fsmap finished waiting for FSMap version 29 2024-03-16T09:17:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.212+0000 7fef06c3a740 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.212+0000 7fef06c3a740 20 client.16005 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.212+0000 7fef06c3a740 1 --2- 172.21.15.112:0/2792981484 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x56412041e2f0 0x564120420700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.212+0000 7fef06c3a740 4 client.16005 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:17:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.212+0000 7fef06c3a740 1 -- 172.21.15.112:0/2792981484 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56412037af80 con 0x56412041e2f0 2024-03-16T09:17:55.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.212+0000 7fef01512640 1 --2- 172.21.15.112:0/2792981484 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x56412041e2f0 0x564120420700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.213+0000 7feee57fa640 10 client.16005 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.213+0000 7fef01512640 1 --2- 172.21.15.112:0/2792981484 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x56412041e2f0 0x564120420700 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7b3d1879e36b7e2 server_cookie=e97ab28dc6d9c707 in_seq=0 out_seq=0 2024-03-16T09:17:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.214+0000 7feee57fa640 1 -- 172.21.15.112:0/2792981484 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56412037af80 con 0x56412041e2f0 2024-03-16T09:17:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.214+0000 7feee57fa640 10 client.16005 handle_command_reply: tid=0 2024-03-16T09:17:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.214+0000 7fef06c3a740 1 client.16005 shutdown 2024-03-16T09:17:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.215+0000 7fef06c3a740 1 -- 172.21.15.112:0/2792981484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408340 msgr2=0x564120412640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.215+0000 7fef06c3a740 1 --2- 172.21.15.112:0/2792981484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408340 0x564120412640 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7feef8011e80 tx=0x7feef8003cf0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.215+0000 7fef06c3a740 1 -- 172.21.15.112:0/2792981484 shutdown_connections 2024-03-16T09:17:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.215+0000 7fef06c3a740 1 --2- 172.21.15.112:0/2792981484 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56412040dea0 0x564120419bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.215+0000 7fef06c3a740 1 --2- 172.21.15.112:0/2792981484 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x56412041e2f0 0x564120420700 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.215+0000 7fef06c3a740 1 --2- 172.21.15.112:0/2792981484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408d30 0x564120412b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.215+0000 7fef06c3a740 1 --2- 172.21.15.112:0/2792981484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408340 0x564120412640 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.215+0000 7feee57fa640 0 client.16005 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:55.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.215+0000 7fef06c3a740 1 -- 172.21.15.112:0/2792981484 >> 172.21.15.112:0/2792981484 conn(0x5641204038b0 msgr2=0x564120402bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.215+0000 7fef06c3a740 1 -- 172.21.15.112:0/2792981484 shutdown_connections 2024-03-16T09:17:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.215+0000 7fef06c3a740 1 -- 172.21.15.112:0/2792981484 wait complete. 2024-03-16T09:17:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.215+0000 7fef06c3a740 20 client.16005 trim_cache size 0 max 16384 2024-03-16T09:17:55.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.230+0000 7fef06c3a740 1 Processor -- start 2024-03-16T09:17:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.230+0000 7fef06c3a740 1 -- start start 2024-03-16T09:17:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7fef06c3a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408340 0x564120407220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7fef06c3a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408d30 0x564120419b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7fef06c3a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56412040dea0 0x564120411c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7fef06c3a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feefc1ab690 con 0x564120408d30 2024-03-16T09:17:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7fef06c3a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feefc1a8ed0 con 0x564120408340 2024-03-16T09:17:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7fef06c3a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feefc1a6d20 con 0x56412040dea0 2024-03-16T09:17:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7fef01512640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408d30 0x564120419b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7fef01512640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408d30 0x564120419b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59058/0 (socket says 172.21.15.112:59058) 2024-03-16T09:17:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7fef01512640 1 -- 172.21.15.112:0/3845978929 learned_addr learned my addr 172.21.15.112:0/3845978929 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7fef02514640 1 --2- 172.21.15.112:0/3845978929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56412040dea0 0x564120411c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7fef01d13640 1 --2- 172.21.15.112:0/3845978929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408340 0x564120407220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7fef01512640 1 -- 172.21.15.112:0/3845978929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56412040dea0 msgr2=0x564120411c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7fef01512640 1 --2- 172.21.15.112:0/3845978929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56412040dea0 0x564120411c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7fef01512640 1 -- 172.21.15.112:0/3845978929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408340 msgr2=0x564120407220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7fef01512640 1 --2- 172.21.15.112:0/3845978929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408340 0x564120407220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7fef01512640 1 -- 172.21.15.112:0/3845978929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641203f5140 con 0x564120408d30 2024-03-16T09:17:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7fef01512640 1 --2- 172.21.15.112:0/3845978929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408d30 0x564120419b80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7feeec0b9970 tx=0x7feeec02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07861c71cd8e152 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7feee77fe640 1 -- 172.21.15.112:0/3845978929 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec0bd510 con 0x564120408d30 2024-03-16T09:17:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7feee77fe640 1 -- 172.21.15.112:0/3845978929 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feeec0c1660 con 0x564120408d30 2024-03-16T09:17:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.231+0000 7feee77fe640 1 -- 172.21.15.112:0/3845978929 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec0b9b80 con 0x564120408d30 2024-03-16T09:17:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.232+0000 7fef06c3a740 1 -- 172.21.15.112:0/3845978929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408d30 msgr2=0x564120419b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.232+0000 7fef06c3a740 1 --2- 172.21.15.112:0/3845978929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408d30 0x564120419b80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7feeec0b9970 tx=0x7feeec02bc30 comp rx=0 tx=0).stop 2024-03-16T09:17:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.232+0000 7fef06c3a740 1 -- 172.21.15.112:0/3845978929 shutdown_connections 2024-03-16T09:17:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.232+0000 7fef06c3a740 1 --2- 172.21.15.112:0/3845978929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56412040dea0 0x564120411c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.232+0000 7fef06c3a740 1 --2- 172.21.15.112:0/3845978929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564120408d30 0x564120419b80 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.232+0000 7fef06c3a740 1 --2- 172.21.15.112:0/3845978929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408340 0x564120407220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.232+0000 7fef06c3a740 1 -- 172.21.15.112:0/3845978929 >> 172.21.15.112:0/3845978929 conn(0x5641204038b0 msgr2=0x564120406b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.232+0000 7fef06c3a740 1 -- 172.21.15.112:0/3845978929 shutdown_connections 2024-03-16T09:17:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.232+0000 7fef06c3a740 1 -- 172.21.15.112:0/3845978929 wait complete. 2024-03-16T09:17:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.232+0000 7fef06c3a740 1 -- start start 2024-03-16T09:17:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.232+0000 7fef06c3a740 1 Processor -- start 2024-03-16T09:17:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef06c3a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408340 0x564120417230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef06c3a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564120408d30 0x564120417770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef06c3a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56412040dea0 0x564120418710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef06c3a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feefc1a7390 con 0x56412040dea0 2024-03-16T09:17:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef06c3a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feefc1a8800 con 0x564120408340 2024-03-16T09:17:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef06c3a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feefc1a7a60 con 0x564120408d30 2024-03-16T09:17:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef06c3a740 10 client.0 _ll_register_callbacks cb 0x7feef4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:17:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef02514640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56412040dea0 0x564120418710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef01d13640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408340 0x564120417230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef02514640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56412040dea0 0x564120418710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59064/0 (socket says 172.21.15.112:59064) 2024-03-16T09:17:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef01512640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564120408d30 0x564120417770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef02514640 1 -- 172.21.15.112:0/1626743009 learned_addr learned my addr 172.21.15.112:0/1626743009 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:17:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef01d13640 1 --2- 172.21.15.112:0/1626743009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408340 0x564120417230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:34306/0 (socket says 172.21.15.112:34306) 2024-03-16T09:17:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef02514640 1 -- 172.21.15.112:0/1626743009 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564120408d30 msgr2=0x564120417770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef02514640 1 --2- 172.21.15.112:0/1626743009 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564120408d30 0x564120417770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef02514640 1 -- 172.21.15.112:0/1626743009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408340 msgr2=0x564120417230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:17:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef02514640 1 --2- 172.21.15.112:0/1626743009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408340 0x564120417230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef02514640 1 -- 172.21.15.112:0/1626743009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564120418d00 con 0x56412040dea0 2024-03-16T09:17:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef01512640 1 --2- 172.21.15.112:0/1626743009 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564120408d30 0x564120417770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef01d13640 1 --2- 172.21.15.112:0/1626743009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408340 0x564120417230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:17:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7fef02514640 1 --2- 172.21.15.112:0/1626743009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56412040dea0 0x564120418710 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7feef4013fc0 tx=0x7feef4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164429e9d1273422 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:17:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.233+0000 7feee57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:17:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.234+0000 7feee57fa640 1 -- 172.21.15.112:0/1626743009 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4004760 con 0x56412040dea0 2024-03-16T09:17:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.234+0000 7feee57fa640 1 -- 172.21.15.112:0/1626743009 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7feef402c470 con 0x56412040dea0 2024-03-16T09:17:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.234+0000 7fef06c3a740 1 -- 172.21.15.112:0/1626743009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeec0c1660 con 0x56412040dea0 2024-03-16T09:17:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.234+0000 7feee57fa640 1 -- 172.21.15.112:0/1626743009 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feef402c670 con 0x56412040dea0 2024-03-16T09:17:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.234+0000 7fef06c3a740 10 client.16011 fetch_fsmap learned FSMap version 29 2024-03-16T09:17:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.234+0000 7fef06c3a740 1 -- 172.21.15.112:0/1626743009 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564120484640 con 0x56412040dea0 2024-03-16T09:17:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.234+0000 7feee57fa640 1 -- 172.21.15.112:0/1626743009 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feeec0c1660 con 0x56412040dea0 2024-03-16T09:17:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.234+0000 7feee57fa640 1 -- 172.21.15.112:0/1626743009 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7feef401f040 con 0x56412040dea0 2024-03-16T09:17:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.234+0000 7fef06c3a740 10 client.16011 fetch_fsmap finished waiting for FSMap version 29 2024-03-16T09:17:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.234+0000 7fef06c3a740 10 client.16011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:17:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.234+0000 7fef06c3a740 20 client.16011 populate_metadata read hostname 'smithi112' 2024-03-16T09:17:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.234+0000 7fef06c3a740 1 --2- 172.21.15.112:0/1626743009 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x564120485070 0x564120487490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:17:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.234+0000 7fef06c3a740 4 client.16011 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T09:17:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.234+0000 7fef06c3a740 1 -- 172.21.15.112:0/1626743009 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5641203752d0 con 0x564120485070 2024-03-16T09:17:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.234+0000 7fef01d13640 1 --2- 172.21.15.112:0/1626743009 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x564120485070 0x564120487490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:17:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.235+0000 7feee57fa640 10 client.16011 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.235+0000 7fef01d13640 1 --2- 172.21.15.112:0/1626743009 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x564120485070 0x564120487490 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9414ed9983a6284 server_cookie=d07c3137d1194888 in_seq=0 out_seq=0 2024-03-16T09:17:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.429+0000 7feee57fa640 1 -- 172.21.15.112:0/1626743009 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5641203752d0 con 0x564120485070 2024-03-16T09:17:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.429+0000 7feee57fa640 10 client.16011 handle_command_reply: tid=0 2024-03-16T09:17:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.429+0000 7fef06c3a740 1 client.16011 shutdown 2024-03-16T09:17:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.429+0000 7fef06c3a740 1 -- 172.21.15.112:0/1626743009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56412040dea0 msgr2=0x564120418710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.429+0000 7fef06c3a740 1 --2- 172.21.15.112:0/1626743009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56412040dea0 0x564120418710 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7feef4013fc0 tx=0x7feef4004a40 comp rx=0 tx=0).stop 2024-03-16T09:17:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7fef06c3a740 1 -- 172.21.15.112:0/1626743009 shutdown_connections 2024-03-16T09:17:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7fef06c3a740 1 --2- 172.21.15.112:0/1626743009 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56412040dea0 0x564120418710 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7fef06c3a740 1 --2- 172.21.15.112:0/1626743009 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564120408d30 0x564120417770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7fef06c3a740 1 --2- 172.21.15.112:0/1626743009 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x564120485070 0x564120487490 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7fef06c3a740 1 --2- 172.21.15.112:0/1626743009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564120408340 0x564120417230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7feee57fa640 0 client.16011 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:17:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7fef06c3a740 1 -- 172.21.15.112:0/1626743009 >> 172.21.15.112:0/1626743009 conn(0x5641204038b0 msgr2=0x5641204073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7fef06c3a740 1 -- 172.21.15.112:0/1626743009 shutdown_connections 2024-03-16T09:17:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7fef06c3a740 1 -- 172.21.15.112:0/1626743009 wait complete. 2024-03-16T09:17:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7fef06c3a740 20 client.16011 trim_cache size 0 max 16384 2024-03-16T09:17:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-03-16T09:17:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-03-16T09:17:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-03-16T09:17:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-03-16T09:17:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7fef03f77640 1 -- 172.21.15.112:0/429493690 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feee80a28a0 msgr2=0x7feee80a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7fef03f77640 1 --2- 172.21.15.112:0/429493690 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feee80a28a0 0x7feee80a4d60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x564120409590 tx=0x7feef4004ab0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7fef03f77640 1 -- 172.21.15.112:0/429493690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc101910 msgr2=0x7feefc198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:17:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7fef03f77640 1 --2- 172.21.15.112:0/429493690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc101910 0x7feefc198070 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7feeec009230 tx=0x7feeec004ab0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7fef03f77640 1 -- 172.21.15.112:0/429493690 shutdown_connections 2024-03-16T09:17:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7fef03f77640 1 --2- 172.21.15.112:0/429493690 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7feee80a28a0 0x7feee80a4d60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7fef03f77640 1 --2- 172.21.15.112:0/429493690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feefc1022d0 0x7feefc19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7fef03f77640 1 --2- 172.21.15.112:0/429493690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feefc101910 0x7feefc198070 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7fef03f77640 1 --2- 172.21.15.112:0/429493690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feefc100720 0x7feefc197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:17:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.430+0000 7fef03f77640 1 -- 172.21.15.112:0/429493690 >> 172.21.15.112:0/429493690 conn(0x7feefc0fbc90 msgr2=0x7feefc0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:17:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.431+0000 7fef03f77640 1 -- 172.21.15.112:0/429493690 shutdown_connections 2024-03-16T09:17:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:17:55.431+0000 7fef03f77640 1 -- 172.21.15.112:0/429493690 wait complete. 2024-03-16T09:17:55.989 INFO:tasks.workunit.client.0.smithi112.stderr:+ tar xf doc.tar 2024-03-16T09:17:56.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:56 smithi112 ceph-mon[32307]: pgmap v420: 97 pgs: 97 active+clean; 150 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 643 op/s 2024-03-16T09:17:56.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:56 smithi194 ceph-mon[36729]: pgmap v420: 97 pgs: 97 active+clean; 150 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 643 op/s 2024-03-16T09:17:56.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:56 smithi159 ceph-mon[36739]: pgmap v420: 97 pgs: 97 active+clean; 150 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 643 op/s 2024-03-16T09:17:58.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:58 smithi112 ceph-mon[32307]: pgmap v421: 97 pgs: 97 active+clean; 149 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 356 op/s 2024-03-16T09:17:58.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:58 smithi194 ceph-mon[36729]: pgmap v421: 97 pgs: 97 active+clean; 149 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 356 op/s 2024-03-16T09:17:58.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:58 smithi159 ceph-mon[36739]: pgmap v421: 97 pgs: 97 active+clean; 149 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 356 op/s 2024-03-16T09:17:59.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:17:59 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:17:59.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:17:59 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:17:59.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:17:59 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:18:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:00 smithi112 ceph-mon[32307]: pgmap v422: 97 pgs: 97 active+clean; 140 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 418 op/s 2024-03-16T09:18:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:00 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:18:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:00 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:18:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:00 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:18:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:00 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:18:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:00 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:18:00.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:00 smithi194 ceph-mon[36729]: pgmap v422: 97 pgs: 97 active+clean; 140 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 418 op/s 2024-03-16T09:18:00.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:00 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:18:00.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:00 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:18:00.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:00 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:18:00.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:00 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:18:00.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:00 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:18:00.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:00 smithi159 ceph-mon[36739]: pgmap v422: 97 pgs: 97 active+clean; 140 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 418 op/s 2024-03-16T09:18:00.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:00 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:18:00.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:00 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:18:00.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:00 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:18:00.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:00 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:18:00.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:00 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:18:02.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:02 smithi112 ceph-mon[32307]: pgmap v423: 97 pgs: 97 active+clean; 140 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 378 op/s 2024-03-16T09:18:02.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:02 smithi194 ceph-mon[36729]: pgmap v423: 97 pgs: 97 active+clean; 140 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 378 op/s 2024-03-16T09:18:02.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:02 smithi159 ceph-mon[36739]: pgmap v423: 97 pgs: 97 active+clean; 140 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 378 op/s 2024-03-16T09:18:04.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:04 smithi112 ceph-mon[32307]: pgmap v424: 97 pgs: 97 active+clean; 143 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 353 op/s 2024-03-16T09:18:04.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:04 smithi194 ceph-mon[36729]: pgmap v424: 97 pgs: 97 active+clean; 143 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 353 op/s 2024-03-16T09:18:04.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:04 smithi159 ceph-mon[36739]: pgmap v424: 97 pgs: 97 active+clean; 143 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 353 op/s 2024-03-16T09:18:05.827 INFO:tasks.workunit.client.0.smithi112.stderr:+ sync 2024-03-16T09:18:06.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:06 smithi112 ceph-mon[32307]: pgmap v425: 97 pgs: 97 active+clean; 143 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 292 op/s 2024-03-16T09:18:06.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:06 smithi194 ceph-mon[36729]: pgmap v425: 97 pgs: 97 active+clean; 143 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 292 op/s 2024-03-16T09:18:06.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:06 smithi159 ceph-mon[36739]: pgmap v425: 97 pgs: 97 active+clean; 143 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 292 op/s 2024-03-16T09:18:08.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:08 smithi112 ceph-mon[32307]: pgmap v426: 97 pgs: 97 active+clean; 144 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1000 KiB/s wr, 236 op/s 2024-03-16T09:18:08.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:08 smithi194 ceph-mon[36729]: pgmap v426: 97 pgs: 97 active+clean; 144 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1000 KiB/s wr, 236 op/s 2024-03-16T09:18:08.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:08 smithi159 ceph-mon[36739]: pgmap v426: 97 pgs: 97 active+clean; 144 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1000 KiB/s wr, 236 op/s 2024-03-16T09:18:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.526+0000 7f5bacc6c640 1 -- 172.21.15.112:0/1076640200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ba8104c30 msgr2=0x7f5ba8107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.526+0000 7f5bacc6c640 1 --2- 172.21.15.112:0/1076640200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ba8104c30 0x7f5ba8107020 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009860 tx=0x7f5b9c02bb70 comp rx=0 tx=0).stop 2024-03-16T09:18:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.526+0000 7f5bacc6c640 1 -- 172.21.15.112:0/1076640200 shutdown_connections 2024-03-16T09:18:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.526+0000 7f5bacc6c640 1 --2- 172.21.15.112:0/1076640200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ba8107560 0x7f5ba8109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.526+0000 7f5bacc6c640 1 --2- 172.21.15.112:0/1076640200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ba8104c30 0x7f5ba8107020 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.526+0000 7f5bacc6c640 1 --2- 172.21.15.112:0/1076640200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ba8102300 0x7f5ba81046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.526+0000 7f5bacc6c640 1 -- 172.21.15.112:0/1076640200 >> 172.21.15.112:0/1076640200 conn(0x7f5ba80fbc90 msgr2=0x7f5ba80fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.526+0000 7f5bacc6c640 1 -- 172.21.15.112:0/1076640200 shutdown_connections 2024-03-16T09:18:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.526+0000 7f5bacc6c640 1 -- 172.21.15.112:0/1076640200 wait complete. 2024-03-16T09:18:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.527+0000 7f5bacc6c640 1 Processor -- start 2024-03-16T09:18:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.527+0000 7f5bacc6c640 1 -- start start 2024-03-16T09:18:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.527+0000 7f5bacc6c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ba8102300 0x7f5ba8197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.527+0000 7f5bacc6c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ba8104c30 0x7f5ba81980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.527+0000 7f5bacc6c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ba8107560 0x7f5ba819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.527+0000 7f5bacc6c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba810d5d0 con 0x7f5ba8107560 2024-03-16T09:18:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.527+0000 7f5bacc6c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba810d450 con 0x7f5ba8104c30 2024-03-16T09:18:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.527+0000 7f5bacc6c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba810d750 con 0x7f5ba8102300 2024-03-16T09:18:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.527+0000 7f5ba6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ba8107560 0x7f5ba819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.527+0000 7f5ba659c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ba8102300 0x7f5ba8197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.527+0000 7f5ba6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ba8107560 0x7f5ba819f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55770/0 (socket says 172.21.15.112:55770) 2024-03-16T09:18:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.527+0000 7f5ba6d9d640 1 -- 172.21.15.112:0/1402460998 learned_addr learned my addr 172.21.15.112:0/1402460998 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:18:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.527+0000 7f5ba5d9b640 1 --2- 172.21.15.112:0/1402460998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ba8104c30 0x7f5ba81980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.528+0000 7f5ba6d9d640 1 -- 172.21.15.112:0/1402460998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ba8102300 msgr2=0x7f5ba8197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.528+0000 7f5ba6d9d640 1 --2- 172.21.15.112:0/1402460998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ba8102300 0x7f5ba8197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.528+0000 7f5ba6d9d640 1 -- 172.21.15.112:0/1402460998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ba8104c30 msgr2=0x7f5ba81980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.528+0000 7f5ba6d9d640 1 --2- 172.21.15.112:0/1402460998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ba8104c30 0x7f5ba81980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.528+0000 7f5ba6d9d640 1 -- 172.21.15.112:0/1402460998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba819f780 con 0x7f5ba8107560 2024-03-16T09:18:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.528+0000 7f5ba659c640 1 --2- 172.21.15.112:0/1402460998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ba8102300 0x7f5ba8197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:18:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.528+0000 7f5ba6d9d640 1 --2- 172.21.15.112:0/1402460998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ba8107560 0x7f5ba819f110 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000efc0 tx=0x7f5ba000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.528+0000 7f5b977fe640 1 -- 172.21.15.112:0/1402460998 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0003970 con 0x7f5ba8107560 2024-03-16T09:18:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.528+0000 7f5bacc6c640 1 -- 172.21.15.112:0/1402460998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba819fa70 con 0x7f5ba8107560 2024-03-16T09:18:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.528+0000 7f5b977fe640 1 -- 172.21.15.112:0/1402460998 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5ba0004520 con 0x7f5ba8107560 2024-03-16T09:18:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.528+0000 7f5b977fe640 1 -- 172.21.15.112:0/1402460998 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba000bb40 con 0x7f5ba8107560 2024-03-16T09:18:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.528+0000 7f5bacc6c640 1 -- 172.21.15.112:0/1402460998 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba819ff90 con 0x7f5ba8107560 2024-03-16T09:18:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.530+0000 7f5b977fe640 1 -- 172.21.15.112:0/1402460998 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7f5ba001d020 con 0x7f5ba8107560 2024-03-16T09:18:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.530+0000 7f5baf84c740 1 Processor -- start 2024-03-16T09:18:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5baf84c740 1 -- start start 2024-03-16T09:18:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5baf84c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3340 0x55cb6d7f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5baf84c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3d30 0x55cb6d7f8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5baf84c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f8ea0 0x55cb6d7fb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5baf84c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cb6d758a40 con 0x55cb6d7f3340 2024-03-16T09:18:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5b977fe640 1 --2- 172.21.15.112:0/1402460998 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5b900a28a0 0x7f5b900a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5baf84c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cb6d701920 con 0x55cb6d7f8ea0 2024-03-16T09:18:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5baf84c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cb6d754ae0 con 0x55cb6d7f3d30 2024-03-16T09:18:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5ba5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3340 0x55cb6d7f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5ba6d9d640 1 --2- 172.21.15.112:0/1402460998 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5b900a28a0 0x7f5b900a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5ba5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3340 0x55cb6d7f3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55782/0 (socket says 172.21.15.112:55782) 2024-03-16T09:18:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5ba5d9b640 1 -- 172.21.15.112:0/776571272 learned_addr learned my addr 172.21.15.112:0/776571272 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:18:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5ba659c640 1 --2- 172.21.15.112:0/776571272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3d30 0x55cb6d7f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5ba5d9b640 1 --2- 172.21.15.112:0/776571272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f8ea0 0x55cb6d7fb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5b977fe640 1 -- 172.21.15.112:0/1402460998 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5ba00ce920 con 0x7f5ba8107560 2024-03-16T09:18:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5ba5d9b640 1 -- 172.21.15.112:0/776571272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3d30 msgr2=0x55cb6d7f8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:18:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5ba5d9b640 1 --2- 172.21.15.112:0/776571272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3d30 0x55cb6d7f8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5ba5d9b640 1 -- 172.21.15.112:0/776571272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f8ea0 msgr2=0x55cb6d7fb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:18:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5ba5d9b640 1 --2- 172.21.15.112:0/776571272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f8ea0 0x55cb6d7fb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5ba5d9b640 1 -- 172.21.15.112:0/776571272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb6d719460 con 0x55cb6d7f3340 2024-03-16T09:18:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.531+0000 7f5ba659c640 1 --2- 172.21.15.112:0/776571272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3d30 0x55cb6d7f8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:18:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.532+0000 7f5ba5d9b640 1 --2- 172.21.15.112:0/776571272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3340 0x55cb6d7f3760 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0057c0 tx=0x7f5b9c034b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2bbdd05cca5322 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.532+0000 7f5ba6d9d640 1 --2- 172.21.15.112:0/1402460998 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5b900a28a0 0x7f5b900a4d60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5ba009afd0 tx=0x7f5ba006e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.532+0000 7f5b94ff9640 1 -- 172.21.15.112:0/776571272 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c048070 con 0x55cb6d7f3340 2024-03-16T09:18:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.532+0000 7f5b94ff9640 1 -- 172.21.15.112:0/776571272 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5b9c034700 con 0x55cb6d7f3340 2024-03-16T09:18:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.532+0000 7f5b94ff9640 1 -- 172.21.15.112:0/776571272 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c039040 con 0x55cb6d7f3340 2024-03-16T09:18:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.532+0000 7f5baf84c740 1 -- 172.21.15.112:0/776571272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3340 msgr2=0x55cb6d7f3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.532+0000 7f5baf84c740 1 --2- 172.21.15.112:0/776571272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3340 0x55cb6d7f3760 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0057c0 tx=0x7f5b9c034b70 comp rx=0 tx=0).stop 2024-03-16T09:18:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5baf84c740 1 -- 172.21.15.112:0/776571272 shutdown_connections 2024-03-16T09:18:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5baf84c740 1 --2- 172.21.15.112:0/776571272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f8ea0 0x55cb6d7fb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5baf84c740 1 --2- 172.21.15.112:0/776571272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3d30 0x55cb6d7f8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5baf84c740 1 --2- 172.21.15.112:0/776571272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3340 0x55cb6d7f3760 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5baf84c740 1 -- 172.21.15.112:0/776571272 >> 172.21.15.112:0/776571272 conn(0x55cb6d7ee8b0 msgr2=0x55cb6d7eecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5baf84c740 1 -- 172.21.15.112:0/776571272 shutdown_connections 2024-03-16T09:18:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5baf84c740 1 -- 172.21.15.112:0/776571272 wait complete. 2024-03-16T09:18:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5baf84c740 1 -- start start 2024-03-16T09:18:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5baf84c740 1 Processor -- start 2024-03-16T09:18:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5baf84c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3340 0x55cb6d7fd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5baf84c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3d30 0x55cb6d7fdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5baf84c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f8ea0 0x55cb6d804c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5baf84c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cb6d7fbdf0 con 0x55cb6d7f3d30 2024-03-16T09:18:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5baf84c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cb6d714d70 con 0x55cb6d7f8ea0 2024-03-16T09:18:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5baf84c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cb6d7fbf70 con 0x55cb6d7f3340 2024-03-16T09:18:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5baf84c740 10 client.0 _ll_register_callbacks cb 0x55cb6d46ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:18:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5ba659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3d30 0x55cb6d7fdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5ba5d9b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3340 0x55cb6d7fd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5ba659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3d30 0x55cb6d7fdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55788/0 (socket says 172.21.15.112:55788) 2024-03-16T09:18:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5ba5d9b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3340 0x55cb6d7fd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:46068/0 (socket says 172.21.15.112:46068) 2024-03-16T09:18:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.533+0000 7f5ba659c640 1 -- 172.21.15.112:0/878192350 learned_addr learned my addr 172.21.15.112:0/878192350 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:18:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.534+0000 7f5ba5d9b640 1 --2- 172.21.15.112:0/878192350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f8ea0 0x55cb6d804c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.534+0000 7f5ba5d9b640 1 -- 172.21.15.112:0/878192350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f8ea0 msgr2=0x55cb6d804c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:18:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.534+0000 7f5ba5d9b640 1 --2- 172.21.15.112:0/878192350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f8ea0 0x55cb6d804c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.534+0000 7f5ba5d9b640 1 -- 172.21.15.112:0/878192350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3d30 msgr2=0x55cb6d7fdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:18:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.534+0000 7f5ba5d9b640 1 --2- 172.21.15.112:0/878192350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3d30 0x55cb6d7fdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.534+0000 7f5ba5d9b640 1 -- 172.21.15.112:0/878192350 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb6d46d210 con 0x55cb6d7f3340 2024-03-16T09:18:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.534+0000 7f5ba659c640 1 --2- 172.21.15.112:0/878192350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3d30 0x55cb6d7fdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:18:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.534+0000 7f5ba5d9b640 1 --2- 172.21.15.112:0/878192350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3340 0x55cb6d7fd6a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c02c0b0 tx=0x7f5b9c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c56e1613e721b723 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.534+0000 7f5b8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-03-16T09:18:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.534+0000 7f5b8dffb640 1 -- 172.21.15.112:0/878192350 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c048070 con 0x55cb6d7f3340 2024-03-16T09:18:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.534+0000 7f5baf84c740 1 -- 172.21.15.112:0/878192350 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b9c034700 con 0x55cb6d7f3340 2024-03-16T09:18:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.534+0000 7f5b8dffb640 1 -- 172.21.15.112:0/878192350 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5b9c004310 con 0x55cb6d7f3340 2024-03-16T09:18:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.534+0000 7f5b8dffb640 1 -- 172.21.15.112:0/878192350 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5b9c058050 con 0x55cb6d7f3340 2024-03-16T09:18:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.534+0000 7f5baf84c740 10 client.25882 fetch_fsmap learned FSMap version 29 2024-03-16T09:18:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.534+0000 7f5baf84c740 1 -- 172.21.15.112:0/878192350 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cb6d539620 con 0x55cb6d7f3340 2024-03-16T09:18:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.535+0000 7f5b8dffb640 1 -- 172.21.15.112:0/878192350 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5b9c034700 con 0x55cb6d7f3340 2024-03-16T09:18:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.535+0000 7f5b8dffb640 1 -- 172.21.15.112:0/878192350 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f5b9c03c070 con 0x55cb6d7f3340 2024-03-16T09:18:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.535+0000 7f5baf84c740 10 client.25882 fetch_fsmap finished waiting for FSMap version 29 2024-03-16T09:18:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.535+0000 7f5baf84c740 10 client.25882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:18:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.535+0000 7f5baf84c740 20 client.25882 populate_metadata read hostname 'smithi112' 2024-03-16T09:18:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.535+0000 7f5baf84c740 1 --2- 172.21.15.112:0/878192350 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55cb6d809110 0x55cb6d80b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.535+0000 7f5baf84c740 4 client.25882 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:18:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.535+0000 7f5baf84c740 1 -- 172.21.15.112:0/878192350 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb6d765f80 con 0x55cb6d809110 2024-03-16T09:18:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.535+0000 7f5ba659c640 1 --2- 172.21.15.112:0/878192350 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55cb6d809110 0x55cb6d80b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.535+0000 7f5b8dffb640 10 client.25882 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:18:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.535+0000 7f5ba659c640 1 --2- 172.21.15.112:0/878192350 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55cb6d809110 0x55cb6d80b500 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=998da2a75d2f23f8 server_cookie=36039091f0033ae8 in_seq=0 out_seq=0 2024-03-16T09:18:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.536+0000 7f5b8dffb640 1 -- 172.21.15.112:0/878192350 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cb6d765f80 con 0x55cb6d809110 2024-03-16T09:18:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.536+0000 7f5b8dffb640 10 client.25882 handle_command_reply: tid=0 2024-03-16T09:18:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.536+0000 7f5baf84c740 1 client.25882 shutdown 2024-03-16T09:18:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.537+0000 7f5baf84c740 1 -- 172.21.15.112:0/878192350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3340 msgr2=0x55cb6d7fd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.537+0000 7f5baf84c740 1 --2- 172.21.15.112:0/878192350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3340 0x55cb6d7fd6a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c02c0b0 tx=0x7f5b9c005850 comp rx=0 tx=0).stop 2024-03-16T09:18:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.537+0000 7f5baf84c740 1 -- 172.21.15.112:0/878192350 shutdown_connections 2024-03-16T09:18:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.537+0000 7f5baf84c740 1 --2- 172.21.15.112:0/878192350 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55cb6d809110 0x55cb6d80b500 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.537+0000 7f5baf84c740 1 --2- 172.21.15.112:0/878192350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f8ea0 0x55cb6d804c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.537+0000 7f5baf84c740 1 --2- 172.21.15.112:0/878192350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3d30 0x55cb6d7fdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.537+0000 7f5baf84c740 1 --2- 172.21.15.112:0/878192350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3340 0x55cb6d7fd6a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.537+0000 7f5b8dffb640 0 client.25882 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:18:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.537+0000 7f5baf84c740 1 -- 172.21.15.112:0/878192350 >> 172.21.15.112:0/878192350 conn(0x55cb6d7ee8b0 msgr2=0x55cb6d7edbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.537+0000 7f5baf84c740 1 -- 172.21.15.112:0/878192350 shutdown_connections 2024-03-16T09:18:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.537+0000 7f5baf84c740 1 -- 172.21.15.112:0/878192350 wait complete. 2024-03-16T09:18:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.537+0000 7f5baf84c740 20 client.25882 trim_cache size 0 max 16384 2024-03-16T09:18:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.551+0000 7f5baf84c740 1 Processor -- start 2024-03-16T09:18:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.551+0000 7f5baf84c740 1 -- start start 2024-03-16T09:18:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5baf84c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f3340 0x55cb6d7fd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5baf84c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3d30 0x55cb6d804be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5baf84c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f8ea0 0x55cb6d7ed320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5baf84c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba81ab5c0 con 0x55cb6d7f8ea0 2024-03-16T09:18:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5baf84c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba81a8e00 con 0x55cb6d7f3340 2024-03-16T09:18:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5baf84c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba81a6c50 con 0x55cb6d7f3d30 2024-03-16T09:18:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5ba5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f8ea0 0x55cb6d7ed320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5ba5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f8ea0 0x55cb6d7ed320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55794/0 (socket says 172.21.15.112:55794) 2024-03-16T09:18:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5ba5d9b640 1 -- 172.21.15.112:0/1606520065 learned_addr learned my addr 172.21.15.112:0/1606520065 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:18:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5ba5d9b640 1 --2- 172.21.15.112:0/1606520065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f3340 0x55cb6d7fd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5ba659c640 1 --2- 172.21.15.112:0/1606520065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3d30 0x55cb6d804be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5ba5d9b640 1 -- 172.21.15.112:0/1606520065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3d30 msgr2=0x55cb6d804be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:18:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5ba5d9b640 1 --2- 172.21.15.112:0/1606520065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3d30 0x55cb6d804be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5ba5d9b640 1 -- 172.21.15.112:0/1606520065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f3340 msgr2=0x55cb6d7fd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:18:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5ba5d9b640 1 --2- 172.21.15.112:0/1606520065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f3340 0x55cb6d7fd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5ba5d9b640 1 -- 172.21.15.112:0/1606520065 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb6d7ef300 con 0x55cb6d7f8ea0 2024-03-16T09:18:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5ba5d9b640 1 --2- 172.21.15.112:0/1606520065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f8ea0 0x55cb6d7ed320 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c001eb0 tx=0x7f5b9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3cd6a5cb6d5c9a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5b8ffff640 1 -- 172.21.15.112:0/1606520065 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c048070 con 0x55cb6d7f8ea0 2024-03-16T09:18:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5b8ffff640 1 -- 172.21.15.112:0/1606520065 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5b9c049040 con 0x55cb6d7f8ea0 2024-03-16T09:18:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.552+0000 7f5b8ffff640 1 -- 172.21.15.112:0/1606520065 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c039040 con 0x55cb6d7f8ea0 2024-03-16T09:18:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.553+0000 7f5baf84c740 1 -- 172.21.15.112:0/1606520065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f8ea0 msgr2=0x55cb6d7ed320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.553+0000 7f5baf84c740 1 --2- 172.21.15.112:0/1606520065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f8ea0 0x55cb6d7ed320 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c001eb0 tx=0x7f5b9c0046c0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.553+0000 7f5baf84c740 1 -- 172.21.15.112:0/1606520065 shutdown_connections 2024-03-16T09:18:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.553+0000 7f5baf84c740 1 --2- 172.21.15.112:0/1606520065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f8ea0 0x55cb6d7ed320 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.553+0000 7f5baf84c740 1 --2- 172.21.15.112:0/1606520065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3d30 0x55cb6d804be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.553+0000 7f5baf84c740 1 --2- 172.21.15.112:0/1606520065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f3340 0x55cb6d7fd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.553+0000 7f5baf84c740 1 -- 172.21.15.112:0/1606520065 >> 172.21.15.112:0/1606520065 conn(0x55cb6d7ee8b0 msgr2=0x55cb6d7fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.553+0000 7f5baf84c740 1 -- 172.21.15.112:0/1606520065 shutdown_connections 2024-03-16T09:18:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.553+0000 7f5baf84c740 1 -- 172.21.15.112:0/1606520065 wait complete. 2024-03-16T09:18:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.553+0000 7f5baf84c740 1 -- start start 2024-03-16T09:18:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.553+0000 7f5baf84c740 1 Processor -- start 2024-03-16T09:18:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.553+0000 7f5baf84c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3340 0x55cb6d802050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.553+0000 7f5baf84c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3d30 0x55cb6d802590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.553+0000 7f5baf84c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f8ea0 0x55cb6d803500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.553+0000 7f5baf84c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba81a72c0 con 0x55cb6d7f3d30 2024-03-16T09:18:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.553+0000 7f5baf84c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba81a8730 con 0x55cb6d7f8ea0 2024-03-16T09:18:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.553+0000 7f5baf84c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba81a7990 con 0x55cb6d7f3340 2024-03-16T09:18:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5baf84c740 10 client.0 _ll_register_callbacks cb 0x7f5b9c046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:18:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5ba659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3d30 0x55cb6d802590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5ba5d9b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3340 0x55cb6d802050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5ba659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3d30 0x55cb6d802590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55806/0 (socket says 172.21.15.112:55806) 2024-03-16T09:18:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5ba659c640 1 -- 172.21.15.112:0/2605826119 learned_addr learned my addr 172.21.15.112:0/2605826119 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:18:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5ba5d9b640 1 --2- 172.21.15.112:0/2605826119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f8ea0 0x55cb6d803500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5ba659c640 1 -- 172.21.15.112:0/2605826119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3340 msgr2=0x55cb6d802050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:18:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5ba659c640 1 --2- 172.21.15.112:0/2605826119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3340 0x55cb6d802050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5ba659c640 1 -- 172.21.15.112:0/2605826119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f8ea0 msgr2=0x55cb6d803500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:18:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5ba659c640 1 --2- 172.21.15.112:0/2605826119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f8ea0 0x55cb6d803500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5ba659c640 1 -- 172.21.15.112:0/2605826119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb6d803af0 con 0x55cb6d7f3d30 2024-03-16T09:18:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5ba659c640 1 --2- 172.21.15.112:0/2605826119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3d30 0x55cb6d802590 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5b98004700 tx=0x7f5b9801f7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38edc9a8232a020 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5b8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:18:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5b8dffb640 1 -- 172.21.15.112:0/2605826119 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b98004030 con 0x55cb6d7f3d30 2024-03-16T09:18:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5b8dffb640 1 -- 172.21.15.112:0/2605826119 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5b980041d0 con 0x55cb6d7f3d30 2024-03-16T09:18:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5baf84c740 1 -- 172.21.15.112:0/2605826119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b9c049040 con 0x55cb6d7f3d30 2024-03-16T09:18:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5b8dffb640 1 -- 172.21.15.112:0/2605826119 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5b980043d0 con 0x55cb6d7f3d30 2024-03-16T09:18:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5baf84c740 10 client.16029 fetch_fsmap learned FSMap version 29 2024-03-16T09:18:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5baf84c740 1 -- 172.21.15.112:0/2605826119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cb6d804f00 con 0x55cb6d7f3d30 2024-03-16T09:18:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.554+0000 7f5b8dffb640 1 -- 172.21.15.112:0/2605826119 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5b9c049040 con 0x55cb6d7f3d30 2024-03-16T09:18:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.555+0000 7f5b8dffb640 1 -- 172.21.15.112:0/2605826119 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f5b98004bb0 con 0x55cb6d7f3d30 2024-03-16T09:18:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.555+0000 7f5baf84c740 10 client.16029 fetch_fsmap finished waiting for FSMap version 29 2024-03-16T09:18:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.555+0000 7f5baf84c740 10 client.16029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:18:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.555+0000 7f5baf84c740 20 client.16029 populate_metadata read hostname 'smithi112' 2024-03-16T09:18:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.555+0000 7f5baf84c740 1 --2- 172.21.15.112:0/2605826119 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55cb6d86faf0 0x55cb6d871f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.555+0000 7f5baf84c740 4 client.16029 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T09:18:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.555+0000 7f5baf84c740 1 -- 172.21.15.112:0/2605826119 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cb6d7602d0 con 0x55cb6d86faf0 2024-03-16T09:18:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.555+0000 7f5ba5d9b640 1 --2- 172.21.15.112:0/2605826119 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55cb6d86faf0 0x55cb6d871f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.555+0000 7f5b8dffb640 10 client.16029 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:18:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:08.555+0000 7f5ba5d9b640 1 --2- 172.21.15.112:0/2605826119 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55cb6d86faf0 0x55cb6d871f10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b508217d7d9a9ac server_cookie=8f125f3d420e1cee in_seq=0 out_seq=0 2024-03-16T09:18:10.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:10 smithi112 ceph-mon[32307]: pgmap v427: 97 pgs: 97 active+clean; 159 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 395 op/s 2024-03-16T09:18:10.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:10 smithi194 ceph-mon[36729]: pgmap v427: 97 pgs: 97 active+clean; 159 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 395 op/s 2024-03-16T09:18:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.429+0000 7f5b8dffb640 1 -- 172.21.15.112:0/2605826119 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cb6d7602d0 con 0x55cb6d86faf0 2024-03-16T09:18:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.429+0000 7f5b8dffb640 10 client.16029 handle_command_reply: tid=0 2024-03-16T09:18:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.429+0000 7f5baf84c740 1 client.16029 shutdown 2024-03-16T09:18:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.429+0000 7f5baf84c740 1 -- 172.21.15.112:0/2605826119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3d30 msgr2=0x55cb6d802590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.429+0000 7f5baf84c740 1 --2- 172.21.15.112:0/2605826119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3d30 0x55cb6d802590 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5b98004700 tx=0x7f5b9801f7c0 comp rx=0 tx=0).stop 2024-03-16T09:18:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.429+0000 7f5baf84c740 1 -- 172.21.15.112:0/2605826119 shutdown_connections 2024-03-16T09:18:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.429+0000 7f5baf84c740 1 --2- 172.21.15.112:0/2605826119 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55cb6d86faf0 0x55cb6d871f10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.429+0000 7f5baf84c740 1 --2- 172.21.15.112:0/2605826119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb6d7f8ea0 0x55cb6d803500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.429+0000 7f5baf84c740 1 --2- 172.21.15.112:0/2605826119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cb6d7f3d30 0x55cb6d802590 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.429+0000 7f5b8dffb640 0 client.16029 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:18:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.429+0000 7f5baf84c740 1 --2- 172.21.15.112:0/2605826119 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cb6d7f3340 0x55cb6d802050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:10.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.429+0000 7f5baf84c740 1 -- 172.21.15.112:0/2605826119 >> 172.21.15.112:0/2605826119 conn(0x55cb6d7ee8b0 msgr2=0x55cb6d7f1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.429+0000 7f5baf84c740 1 -- 172.21.15.112:0/2605826119 shutdown_connections 2024-03-16T09:18:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.429+0000 7f5baf84c740 1 -- 172.21.15.112:0/2605826119 wait complete. 2024-03-16T09:18:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.429+0000 7f5baf84c740 20 client.16029 trim_cache size 0 max 16384 2024-03-16T09:18:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-03-16T09:18:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-03-16T09:18:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-03-16T09:18:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-03-16T09:18:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.430+0000 7f5bacc6c640 1 -- 172.21.15.112:0/1402460998 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5b900a28a0 msgr2=0x7f5b900a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.430+0000 7f5bacc6c640 1 --2- 172.21.15.112:0/1402460998 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5b900a28a0 0x7f5b900a4d60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5ba009afd0 tx=0x7f5ba006e000 comp rx=0 tx=0).stop 2024-03-16T09:18:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.430+0000 7f5bacc6c640 1 -- 172.21.15.112:0/1402460998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ba8107560 msgr2=0x7f5ba819f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.430+0000 7f5bacc6c640 1 --2- 172.21.15.112:0/1402460998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ba8107560 0x7f5ba819f110 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000efc0 tx=0x7f5ba000c800 comp rx=0 tx=0).stop 2024-03-16T09:18:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.430+0000 7f5bacc6c640 1 -- 172.21.15.112:0/1402460998 shutdown_connections 2024-03-16T09:18:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.430+0000 7f5bacc6c640 1 --2- 172.21.15.112:0/1402460998 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5b900a28a0 0x7f5b900a4d60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.430+0000 7f5bacc6c640 1 --2- 172.21.15.112:0/1402460998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ba8107560 0x7f5ba819f110 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.430+0000 7f5bacc6c640 1 --2- 172.21.15.112:0/1402460998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ba8104c30 0x7f5ba81980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.430+0000 7f5bacc6c640 1 --2- 172.21.15.112:0/1402460998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ba8102300 0x7f5ba8197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.430+0000 7f5bacc6c640 1 -- 172.21.15.112:0/1402460998 >> 172.21.15.112:0/1402460998 conn(0x7f5ba80fbc90 msgr2=0x7f5ba8102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.430+0000 7f5bacc6c640 1 -- 172.21.15.112:0/1402460998 shutdown_connections 2024-03-16T09:18:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:10.430+0000 7f5bacc6c640 1 -- 172.21.15.112:0/1402460998 wait complete. 2024-03-16T09:18:10.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:10 smithi159 ceph-mon[36739]: pgmap v427: 97 pgs: 97 active+clean; 159 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 395 op/s 2024-03-16T09:18:11.476 INFO:tasks.workunit.client.0.smithi112.stderr:+ tar c linux-4.0.5 2024-03-16T09:18:11.851 INFO:tasks.workunit.client.0.smithi112.stdout:Ok 2024-03-16T09:18:11.852 INFO:tasks.workunit.client.0.smithi112.stderr:+ echo Ok 2024-03-16T09:18:11.853 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-16T09:18:11.853 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-16T09:18:12.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:12 smithi112 ceph-mon[32307]: pgmap v428: 97 pgs: 97 active+clean; 165 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.8 MiB/s wr, 375 op/s 2024-03-16T09:18:12.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:12 smithi194 ceph-mon[36729]: pgmap v428: 97 pgs: 97 active+clean; 165 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.8 MiB/s wr, 375 op/s 2024-03-16T09:18:12.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:12 smithi159 ceph-mon[36739]: pgmap v428: 97 pgs: 97 active+clean; 165 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.8 MiB/s wr, 375 op/s 2024-03-16T09:18:14.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:14 smithi112 ceph-mon[32307]: pgmap v429: 97 pgs: 97 active+clean; 152 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 2.7 MiB/s wr, 651 op/s 2024-03-16T09:18:14.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:14 smithi194 ceph-mon[36729]: pgmap v429: 97 pgs: 97 active+clean; 152 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 2.7 MiB/s wr, 651 op/s 2024-03-16T09:18:14.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:14 smithi159 ceph-mon[36739]: pgmap v429: 97 pgs: 97 active+clean; 152 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 2.7 MiB/s wr, 651 op/s 2024-03-16T09:18:16.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:16 smithi112 ceph-mon[32307]: pgmap v430: 97 pgs: 97 active+clean; 81 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 2.9 MiB/s wr, 1.06k op/s 2024-03-16T09:18:16.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:16 smithi194 ceph-mon[36729]: pgmap v430: 97 pgs: 97 active+clean; 81 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 2.9 MiB/s wr, 1.06k op/s 2024-03-16T09:18:16.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:16 smithi159 ceph-mon[36739]: pgmap v430: 97 pgs: 97 active+clean; 81 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 2.9 MiB/s wr, 1.06k op/s 2024-03-16T09:18:18.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:18 smithi112 ceph-mon[32307]: pgmap v431: 97 pgs: 97 active+clean; 74 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 2.9 MiB/s wr, 1.07k op/s 2024-03-16T09:18:18.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:18 smithi194 ceph-mon[36729]: pgmap v431: 97 pgs: 97 active+clean; 74 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 2.9 MiB/s wr, 1.07k op/s 2024-03-16T09:18:18.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:18 smithi159 ceph-mon[36739]: pgmap v431: 97 pgs: 97 active+clean; 74 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 2.9 MiB/s wr, 1.07k op/s 2024-03-16T09:18:20.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:20 smithi112 ceph-mon[32307]: pgmap v432: 97 pgs: 97 active+clean; 58 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 1.13k op/s 2024-03-16T09:18:20.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:20 smithi194 ceph-mon[36729]: pgmap v432: 97 pgs: 97 active+clean; 58 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 1.13k op/s 2024-03-16T09:18:20.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:20 smithi159 ceph-mon[36739]: pgmap v432: 97 pgs: 97 active+clean; 58 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 1.13k op/s 2024-03-16T09:18:22.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:22 smithi112 ceph-mon[32307]: pgmap v433: 97 pgs: 97 active+clean; 61 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 990 op/s 2024-03-16T09:18:22.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:22 smithi194 ceph-mon[36729]: pgmap v433: 97 pgs: 97 active+clean; 61 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 990 op/s 2024-03-16T09:18:22.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:22 smithi159 ceph-mon[36739]: pgmap v433: 97 pgs: 97 active+clean; 61 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 990 op/s 2024-03-16T09:18:23.522 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.517+0000 7f5b97863640 1 -- 172.21.15.112:0/1798738082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b90101830 msgr2=0x7f5b90101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.517+0000 7f5b97863640 1 --2- 172.21.15.112:0/1798738082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b90101830 0x7f5b90101c50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009900 tx=0x7f5b8c02bc30 comp rx=0 tx=0).stop 2024-03-16T09:18:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.517+0000 7f5b97863640 1 -- 172.21.15.112:0/1798738082 shutdown_connections 2024-03-16T09:18:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.517+0000 7f5b97863640 1 --2- 172.21.15.112:0/1798738082 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b901033e0 0x7f5b90109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.517+0000 7f5b97863640 1 --2- 172.21.15.112:0/1798738082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b90102a20 0x7f5b90102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.517+0000 7f5b97863640 1 --2- 172.21.15.112:0/1798738082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b90101830 0x7f5b90101c50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.517+0000 7f5b97863640 1 -- 172.21.15.112:0/1798738082 >> 172.21.15.112:0/1798738082 conn(0x7f5b900fcda0 msgr2=0x7f5b900ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.517+0000 7f5b97863640 1 -- 172.21.15.112:0/1798738082 shutdown_connections 2024-03-16T09:18:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.517+0000 7f5b97863640 1 -- 172.21.15.112:0/1798738082 wait complete. 2024-03-16T09:18:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b97863640 1 Processor -- start 2024-03-16T09:18:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b97863640 1 -- start start 2024-03-16T09:18:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b97863640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b90101830 0x7f5b90197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b97863640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b90102a20 0x7f5b90198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b97863640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b901033e0 0x7f5b9019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b97863640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9010d590 con 0x7f5b90101830 2024-03-16T09:18:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b97863640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9010d410 con 0x7f5b901033e0 2024-03-16T09:18:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b97863640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9010d710 con 0x7f5b90102a20 2024-03-16T09:18:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b955ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b90101830 0x7f5b90197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b955ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b90101830 0x7f5b90197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52174/0 (socket says 172.21.15.112:52174) 2024-03-16T09:18:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b955ff640 1 -- 172.21.15.112:0/3709284375 learned_addr learned my addr 172.21.15.112:0/3709284375 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:18:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b94dfe640 1 --2- 172.21.15.112:0/3709284375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b90102a20 0x7f5b90198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b95e00640 1 --2- 172.21.15.112:0/3709284375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b901033e0 0x7f5b9019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b955ff640 1 -- 172.21.15.112:0/3709284375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b90102a20 msgr2=0x7f5b90198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b955ff640 1 --2- 172.21.15.112:0/3709284375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b90102a20 0x7f5b90198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b955ff640 1 -- 172.21.15.112:0/3709284375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b901033e0 msgr2=0x7f5b9019f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b955ff640 1 --2- 172.21.15.112:0/3709284375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b901033e0 0x7f5b9019f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b955ff640 1 -- 172.21.15.112:0/3709284375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9019f740 con 0x7f5b90101830 2024-03-16T09:18:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b955ff640 1 --2- 172.21.15.112:0/3709284375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b90101830 0x7f5b90197b20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c000c00 tx=0x7f5b8c004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.518+0000 7f5b867fc640 1 -- 172.21.15.112:0/3709284375 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c043020 con 0x7f5b90101830 2024-03-16T09:18:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.519+0000 7f5b867fc640 1 -- 172.21.15.112:0/3709284375 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5b8c004250 con 0x7f5b90101830 2024-03-16T09:18:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.519+0000 7f5b97863640 1 -- 172.21.15.112:0/3709284375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9019f9d0 con 0x7f5b90101830 2024-03-16T09:18:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.519+0000 7f5b867fc640 1 -- 172.21.15.112:0/3709284375 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c003b90 con 0x7f5b90101830 2024-03-16T09:18:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.519+0000 7f5b97863640 1 -- 172.21.15.112:0/3709284375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9019ff20 con 0x7f5b90101830 2024-03-16T09:18:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.521+0000 7f5b867fc640 1 -- 172.21.15.112:0/3709284375 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7f5b8c046020 con 0x7f5b90101830 2024-03-16T09:18:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.521+0000 7f5b9a448740 1 Processor -- start 2024-03-16T09:18:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.521+0000 7f5b9a448740 1 -- start start 2024-03-16T09:18:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.521+0000 7f5b867fc640 1 --2- 172.21.15.112:0/3709284375 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5b7c0a28a0 0x7f5b7c0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.521+0000 7f5b9a448740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 0x55a0dde99760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.521+0000 7f5b9a448740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde99d30 0x55a0dde9e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.521+0000 7f5b9a448740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde9eea0 0x55a0ddea1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.521+0000 7f5b9a448740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a0dddfea40 con 0x55a0dde9eea0 2024-03-16T09:18:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.521+0000 7f5b9a448740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a0ddda7920 con 0x55a0dde99340 2024-03-16T09:18:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.521+0000 7f5b9a448740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a0dddfaae0 con 0x55a0dde99d30 2024-03-16T09:18:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.521+0000 7f5b95e00640 1 --2- 172.21.15.112:0/3709284375 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5b7c0a28a0 0x7f5b7c0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.521+0000 7f5b955ff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 0x55a0dde99760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.521+0000 7f5b867fc640 1 -- 172.21.15.112:0/3709284375 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5b8c0f0b20 con 0x7f5b90101830 2024-03-16T09:18:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.521+0000 7f5b95e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde9eea0 0x55a0ddea1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.522+0000 7f5b955ff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 0x55a0dde99760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:37498/0 (socket says 172.21.15.112:37498) 2024-03-16T09:18:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.522+0000 7f5b955ff640 1 -- 172.21.15.112:0/3091540504 learned_addr learned my addr 172.21.15.112:0/3091540504 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:18:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.522+0000 7f5b94dfe640 1 --2- 172.21.15.112:0/3091540504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde99d30 0x55a0dde9e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.522+0000 7f5b955ff640 1 -- 172.21.15.112:0/3091540504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde99d30 msgr2=0x55a0dde9e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:18:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.522+0000 7f5b955ff640 1 --2- 172.21.15.112:0/3091540504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde99d30 0x55a0dde9e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.522+0000 7f5b955ff640 1 -- 172.21.15.112:0/3091540504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde9eea0 msgr2=0x55a0ddea1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:18:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.522+0000 7f5b955ff640 1 --2- 172.21.15.112:0/3091540504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde9eea0 0x55a0ddea1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.522+0000 7f5b955ff640 1 -- 172.21.15.112:0/3091540504 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0dddbf460 con 0x55a0dde99340 2024-03-16T09:18:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.522+0000 7f5b94dfe640 1 --2- 172.21.15.112:0/3091540504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde99d30 0x55a0dde9e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:18:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.522+0000 7f5b95e00640 1 --2- 172.21.15.112:0/3709284375 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5b7c0a28a0 0x7f5b7c0a4d60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5b88005fd0 tx=0x7f5b88009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.522+0000 7f5b955ff640 1 --2- 172.21.15.112:0/3091540504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 0x55a0dde99760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0bd1d0 tx=0x7f5b8c02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1759f5dec8d28911 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.522+0000 7f5b7b7fe640 1 -- 172.21.15.112:0/3091540504 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c0bce30 con 0x55a0dde99340 2024-03-16T09:18:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.522+0000 7f5b7b7fe640 1 -- 172.21.15.112:0/3091540504 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5b8c0bdbd0 con 0x55a0dde99340 2024-03-16T09:18:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.522+0000 7f5b7b7fe640 1 -- 172.21.15.112:0/3091540504 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c09c200 con 0x55a0dde99340 2024-03-16T09:18:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.523+0000 7f5b9a448740 1 -- 172.21.15.112:0/3091540504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 msgr2=0x55a0dde99760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.523+0000 7f5b9a448740 1 --2- 172.21.15.112:0/3091540504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 0x55a0dde99760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0bd1d0 tx=0x7f5b8c02bc30 comp rx=0 tx=0).stop 2024-03-16T09:18:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.523+0000 7f5b9a448740 1 -- 172.21.15.112:0/3091540504 shutdown_connections 2024-03-16T09:18:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.523+0000 7f5b9a448740 1 --2- 172.21.15.112:0/3091540504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde9eea0 0x55a0ddea1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.523+0000 7f5b9a448740 1 --2- 172.21.15.112:0/3091540504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde99d30 0x55a0dde9e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.523+0000 7f5b9a448740 1 --2- 172.21.15.112:0/3091540504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 0x55a0dde99760 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.523+0000 7f5b9a448740 1 -- 172.21.15.112:0/3091540504 >> 172.21.15.112:0/3091540504 conn(0x55a0dde948b0 msgr2=0x55a0dde94cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.523+0000 7f5b9a448740 1 -- 172.21.15.112:0/3091540504 shutdown_connections 2024-03-16T09:18:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.523+0000 7f5b9a448740 1 -- 172.21.15.112:0/3091540504 wait complete. 2024-03-16T09:18:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.523+0000 7f5b9a448740 1 -- start start 2024-03-16T09:18:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.523+0000 7f5b9a448740 1 Processor -- start 2024-03-16T09:18:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b9a448740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 0x55a0ddea33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b9a448740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde99d30 0x55a0ddea3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b9a448740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde9eea0 0x55a0ddeaa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b9a448740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a0ddea1c10 con 0x55a0dde99d30 2024-03-16T09:18:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b9a448740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a0dddbad70 con 0x55a0dde99340 2024-03-16T09:18:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b9a448740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a0ddea1d90 con 0x55a0dde9eea0 2024-03-16T09:18:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b9a448740 10 client.0 _ll_register_callbacks cb 0x55a0ddb12a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:18:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b94dfe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde99d30 0x55a0ddea3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b955ff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 0x55a0ddea33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b94dfe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde99d30 0x55a0ddea3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52200/0 (socket says 172.21.15.112:52200) 2024-03-16T09:18:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b95e00640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde9eea0 0x55a0ddeaa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b94dfe640 1 -- 172.21.15.112:0/2728804132 learned_addr learned my addr 172.21.15.112:0/2728804132 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:18:23.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b94dfe640 1 -- 172.21.15.112:0/2728804132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde9eea0 msgr2=0x55a0ddeaa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:18:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b94dfe640 1 --2- 172.21.15.112:0/2728804132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde9eea0 0x55a0ddeaa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b94dfe640 1 -- 172.21.15.112:0/2728804132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 msgr2=0x55a0ddea33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:18:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b94dfe640 1 --2- 172.21.15.112:0/2728804132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 0x55a0ddea33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b94dfe640 1 -- 172.21.15.112:0/2728804132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0ddb13210 con 0x55a0dde99d30 2024-03-16T09:18:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b955ff640 1 --2- 172.21.15.112:0/2728804132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 0x55a0ddea33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:18:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b94dfe640 1 --2- 172.21.15.112:0/2728804132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde99d30 0x55a0ddea3910 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5b80012700 tx=0x7f5b80012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480d1e6e0a5e664d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:18:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b78ff9640 1 -- 172.21.15.112:0/2728804132 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b80004030 con 0x55a0dde99d30 2024-03-16T09:18:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b78ff9640 1 -- 172.21.15.112:0/2728804132 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5b800041d0 con 0x55a0dde99d30 2024-03-16T09:18:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.524+0000 7f5b9a448740 1 -- 172.21.15.112:0/2728804132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b8c0bdbd0 con 0x55a0dde99d30 2024-03-16T09:18:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.525+0000 7f5b78ff9640 1 -- 172.21.15.112:0/2728804132 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5b800043d0 con 0x55a0dde99d30 2024-03-16T09:18:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.525+0000 7f5b9a448740 10 client.16041 fetch_fsmap learned FSMap version 29 2024-03-16T09:18:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.525+0000 7f5b9a448740 1 -- 172.21.15.112:0/2728804132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a0ddbdf620 con 0x55a0dde99d30 2024-03-16T09:18:23.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.525+0000 7f5b78ff9640 1 -- 172.21.15.112:0/2728804132 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5b8c0bdbd0 con 0x55a0dde99d30 2024-03-16T09:18:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.525+0000 7f5b78ff9640 1 -- 172.21.15.112:0/2728804132 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f5b80003c70 con 0x55a0dde99d30 2024-03-16T09:18:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.525+0000 7f5b9a448740 10 client.16041 fetch_fsmap finished waiting for FSMap version 29 2024-03-16T09:18:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.525+0000 7f5b9a448740 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:18:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.525+0000 7f5b9a448740 20 client.16041 populate_metadata read hostname 'smithi112' 2024-03-16T09:18:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.525+0000 7f5b9a448740 1 --2- 172.21.15.112:0/2728804132 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55a0ddeaf300 0x55a0ddeb1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.525+0000 7f5b9a448740 4 client.16041 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T09:18:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.525+0000 7f5b9a448740 1 -- 172.21.15.112:0/2728804132 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0dde0bf80 con 0x55a0ddeaf300 2024-03-16T09:18:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.525+0000 7f5b955ff640 1 --2- 172.21.15.112:0/2728804132 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55a0ddeaf300 0x55a0ddeb1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.525+0000 7f5b78ff9640 10 client.16041 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:18:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.525+0000 7f5b955ff640 1 --2- 172.21.15.112:0/2728804132 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55a0ddeaf300 0x55a0ddeb1740 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=940257817b76f34c server_cookie=51c8a89252450c56 in_seq=0 out_seq=0 2024-03-16T09:18:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.527+0000 7f5b78ff9640 1 -- 172.21.15.112:0/2728804132 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a0dde0bf80 con 0x55a0ddeaf300 2024-03-16T09:18:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.527+0000 7f5b78ff9640 10 client.16041 handle_command_reply: tid=0 2024-03-16T09:18:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.527+0000 7f5b9a448740 1 client.16041 shutdown 2024-03-16T09:18:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.528+0000 7f5b9a448740 1 -- 172.21.15.112:0/2728804132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde99d30 msgr2=0x55a0ddea3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.528+0000 7f5b9a448740 1 --2- 172.21.15.112:0/2728804132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde99d30 0x55a0ddea3910 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5b80012700 tx=0x7f5b80012bc0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.528+0000 7f5b9a448740 1 -- 172.21.15.112:0/2728804132 shutdown_connections 2024-03-16T09:18:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.528+0000 7f5b9a448740 1 --2- 172.21.15.112:0/2728804132 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde9eea0 0x55a0ddeaa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.528+0000 7f5b9a448740 1 --2- 172.21.15.112:0/2728804132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde99d30 0x55a0ddea3910 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.528+0000 7f5b9a448740 1 --2- 172.21.15.112:0/2728804132 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55a0ddeaf300 0x55a0ddeb1740 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.528+0000 7f5b9a448740 1 --2- 172.21.15.112:0/2728804132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 0x55a0ddea33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.528+0000 7f5b78ff9640 0 client.16041 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:18:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.528+0000 7f5b9a448740 1 -- 172.21.15.112:0/2728804132 >> 172.21.15.112:0/2728804132 conn(0x55a0dde948b0 msgr2=0x55a0dde93bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.528+0000 7f5b9a448740 1 -- 172.21.15.112:0/2728804132 shutdown_connections 2024-03-16T09:18:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.528+0000 7f5b9a448740 1 -- 172.21.15.112:0/2728804132 wait complete. 2024-03-16T09:18:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.528+0000 7f5b9a448740 20 client.16041 trim_cache size 0 max 16384 2024-03-16T09:18:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.542+0000 7f5b9a448740 1 Processor -- start 2024-03-16T09:18:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.542+0000 7f5b9a448740 1 -- start start 2024-03-16T09:18:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b9a448740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 0x55a0dde97b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b9a448740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde99d30 0x55a0ddeaa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b9a448740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde9eea0 0x55a0dde98050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b9a448740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901a9460 con 0x55a0dde99d30 2024-03-16T09:18:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b9a448740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901a6ca0 con 0x55a0dde99340 2024-03-16T09:18:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b9a448740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901a4af0 con 0x55a0dde9eea0 2024-03-16T09:18:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b94dfe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde99d30 0x55a0ddeaa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b94dfe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde99d30 0x55a0ddeaa910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52216/0 (socket says 172.21.15.112:52216) 2024-03-16T09:18:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b94dfe640 1 -- 172.21.15.112:0/1693341258 learned_addr learned my addr 172.21.15.112:0/1693341258 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:18:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b955ff640 1 --2- 172.21.15.112:0/1693341258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 0x55a0dde97b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b95e00640 1 --2- 172.21.15.112:0/1693341258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde9eea0 0x55a0dde98050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b94dfe640 1 -- 172.21.15.112:0/1693341258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde9eea0 msgr2=0x55a0dde98050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:18:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b94dfe640 1 --2- 172.21.15.112:0/1693341258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde9eea0 0x55a0dde98050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b94dfe640 1 -- 172.21.15.112:0/1693341258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 msgr2=0x55a0dde97b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:18:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b94dfe640 1 --2- 172.21.15.112:0/1693341258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 0x55a0dde97b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b94dfe640 1 -- 172.21.15.112:0/1693341258 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0dde9a110 con 0x55a0dde99d30 2024-03-16T09:18:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b94dfe640 1 --2- 172.21.15.112:0/1693341258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde99d30 0x55a0ddeaa910 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5b80000c00 tx=0x7f5b80009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc91e1e528c7920d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b7affd640 1 -- 172.21.15.112:0/1693341258 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b80010770 con 0x55a0dde99d30 2024-03-16T09:18:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b7affd640 1 -- 172.21.15.112:0/1693341258 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5b80010e10 con 0x55a0dde99d30 2024-03-16T09:18:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.543+0000 7f5b7affd640 1 -- 172.21.15.112:0/1693341258 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b80027780 con 0x55a0dde99d30 2024-03-16T09:18:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.544+0000 7f5b9a448740 1 -- 172.21.15.112:0/1693341258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde99d30 msgr2=0x55a0ddeaa910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.544+0000 7f5b9a448740 1 --2- 172.21.15.112:0/1693341258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde99d30 0x55a0ddeaa910 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5b80000c00 tx=0x7f5b80009ea0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.544+0000 7f5b9a448740 1 -- 172.21.15.112:0/1693341258 shutdown_connections 2024-03-16T09:18:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.544+0000 7f5b9a448740 1 --2- 172.21.15.112:0/1693341258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde9eea0 0x55a0dde98050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.544+0000 7f5b9a448740 1 --2- 172.21.15.112:0/1693341258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde99d30 0x55a0ddeaa910 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.544+0000 7f5b9a448740 1 --2- 172.21.15.112:0/1693341258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 0x55a0dde97b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.544+0000 7f5b9a448740 1 -- 172.21.15.112:0/1693341258 >> 172.21.15.112:0/1693341258 conn(0x55a0dde948b0 msgr2=0x55a0dde94c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.544+0000 7f5b9a448740 1 -- 172.21.15.112:0/1693341258 shutdown_connections 2024-03-16T09:18:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.544+0000 7f5b9a448740 1 -- 172.21.15.112:0/1693341258 wait complete. 2024-03-16T09:18:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.544+0000 7f5b9a448740 1 -- start start 2024-03-16T09:18:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.544+0000 7f5b9a448740 1 Processor -- start 2024-03-16T09:18:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b9a448740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 0x55a0ddeaa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b9a448740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde99d30 0x55a0ddeaa850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b9a448740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde9eea0 0x55a0ddea8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b9a448740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901a5160 con 0x55a0dde9eea0 2024-03-16T09:18:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b9a448740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901a65d0 con 0x55a0dde99340 2024-03-16T09:18:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b9a448740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901a5830 con 0x55a0dde99d30 2024-03-16T09:18:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b9a448740 10 client.0 _ll_register_callbacks cb 0x7f5b80009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T09:18:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b95e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde9eea0 0x55a0ddea8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b95e00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde9eea0 0x55a0ddea8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52224/0 (socket says 172.21.15.112:52224) 2024-03-16T09:18:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b95e00640 1 -- 172.21.15.112:0/700127443 learned_addr learned my addr 172.21.15.112:0/700127443 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:18:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b955ff640 1 --2- 172.21.15.112:0/700127443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 0x55a0ddeaa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b94dfe640 1 --2- 172.21.15.112:0/700127443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde99d30 0x55a0ddeaa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b95e00640 1 -- 172.21.15.112:0/700127443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde99d30 msgr2=0x55a0ddeaa850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:18:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b95e00640 1 --2- 172.21.15.112:0/700127443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde99d30 0x55a0ddeaa850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b95e00640 1 -- 172.21.15.112:0/700127443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 msgr2=0x55a0ddeaa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T09:18:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b95e00640 1 --2- 172.21.15.112:0/700127443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 0x55a0ddeaa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b95e00640 1 -- 172.21.15.112:0/700127443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0ddea91a0 con 0x55a0dde9eea0 2024-03-16T09:18:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b95e00640 1 --2- 172.21.15.112:0/700127443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde9eea0 0x55a0ddea8970 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5b88005c00 tx=0x7f5b88009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42d2cd39302ebd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-03-16T09:18:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b78ff9640 1 -- 172.21.15.112:0/700127443 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88026040 con 0x55a0dde9eea0 2024-03-16T09:18:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b78ff9640 1 -- 172.21.15.112:0/700127443 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5b8801a070 con 0x55a0dde9eea0 2024-03-16T09:18:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.545+0000 7f5b9a448740 1 -- 172.21.15.112:0/700127443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b80010e10 con 0x55a0dde9eea0 2024-03-16T09:18:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.546+0000 7f5b78ff9640 1 -- 172.21.15.112:0/700127443 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5b88004b20 con 0x55a0dde9eea0 2024-03-16T09:18:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.546+0000 7f5b9a448740 10 client.16047 fetch_fsmap learned FSMap version 29 2024-03-16T09:18:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.546+0000 7f5b9a448740 1 -- 172.21.15.112:0/700127443 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a0ddf161d0 con 0x55a0dde9eea0 2024-03-16T09:18:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.546+0000 7f5b78ff9640 1 -- 172.21.15.112:0/700127443 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5b80010e10 con 0x55a0dde9eea0 2024-03-16T09:18:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.546+0000 7f5b78ff9640 1 -- 172.21.15.112:0/700127443 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f5b88029030 con 0x55a0dde9eea0 2024-03-16T09:18:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.546+0000 7f5b9a448740 10 client.16047 fetch_fsmap finished waiting for FSMap version 29 2024-03-16T09:18:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.546+0000 7f5b9a448740 10 client.16047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-16T09:18:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.546+0000 7f5b9a448740 20 client.16047 populate_metadata read hostname 'smithi112' 2024-03-16T09:18:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.546+0000 7f5b9a448740 1 --2- 172.21.15.112:0/700127443 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55a0ddf16c90 0x55a0ddf190b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.546+0000 7f5b9a448740 4 client.16047 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-16T09:18:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.546+0000 7f5b9a448740 1 -- 172.21.15.112:0/700127443 --> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0dde062d0 con 0x55a0ddf16c90 2024-03-16T09:18:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.546+0000 7f5b955ff640 1 --2- 172.21.15.112:0/700127443 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55a0ddf16c90 0x55a0ddf190b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.546+0000 7f5b78ff9640 10 client.16047 ms_handle_connect on v2:172.21.15.159:6838/272895965 2024-03-16T09:18:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.546+0000 7f5b955ff640 1 --2- 172.21.15.112:0/700127443 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55a0ddf16c90 0x55a0ddf190b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3bdc655915ce15e server_cookie=d44e059acb6f5a11 in_seq=0 out_seq=0 2024-03-16T09:18:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.964+0000 7f5b78ff9640 1 -- 172.21.15.112:0/700127443 <== mds.0 v2:172.21.15.159:6838/272895965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0dde062d0 con 0x55a0ddf16c90 2024-03-16T09:18:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.964+0000 7f5b78ff9640 10 client.16047 handle_command_reply: tid=0 2024-03-16T09:18:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.964+0000 7f5b9a448740 1 client.16047 shutdown 2024-03-16T09:18:23.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.964+0000 7f5b9a448740 1 -- 172.21.15.112:0/700127443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde9eea0 msgr2=0x55a0ddea8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.964+0000 7f5b9a448740 1 --2- 172.21.15.112:0/700127443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde9eea0 0x55a0ddea8970 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5b88005c00 tx=0x7f5b88009500 comp rx=0 tx=0).stop 2024-03-16T09:18:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.964+0000 7f5b9a448740 1 -- 172.21.15.112:0/700127443 shutdown_connections 2024-03-16T09:18:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.964+0000 7f5b9a448740 1 --2- 172.21.15.112:0/700127443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a0dde9eea0 0x55a0ddea8970 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.964+0000 7f5b9a448740 1 --2- 172.21.15.112:0/700127443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0dde99d30 0x55a0ddeaa850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.964+0000 7f5b9a448740 1 --2- 172.21.15.112:0/700127443 >> [v2:172.21.15.159:6838/272895965,v1:172.21.15.159:6839/272895965] conn(0x55a0ddf16c90 0x55a0ddf190b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.964+0000 7f5b78ff9640 0 client.16047 ms_handle_reset on v2:172.21.15.159:6838/272895965 2024-03-16T09:18:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.964+0000 7f5b9a448740 1 --2- 172.21.15.112:0/700127443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0dde99340 0x55a0ddeaa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.964+0000 7f5b9a448740 1 -- 172.21.15.112:0/700127443 >> 172.21.15.112:0/700127443 conn(0x55a0dde948b0 msgr2=0x55a0dde97b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.964+0000 7f5b9a448740 1 -- 172.21.15.112:0/700127443 shutdown_connections 2024-03-16T09:18:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.964+0000 7f5b9a448740 1 -- 172.21.15.112:0/700127443 wait complete. 2024-03-16T09:18:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.964+0000 7f5b9a448740 20 client.16047 trim_cache size 0 max 16384 2024-03-16T09:18:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-03-16T09:18:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-03-16T09:18:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-03-16T09:18:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-03-16T09:18:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.965+0000 7f5b97863640 1 -- 172.21.15.112:0/3709284375 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5b7c0a28a0 msgr2=0x7f5b7c0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.965+0000 7f5b97863640 1 --2- 172.21.15.112:0/3709284375 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5b7c0a28a0 0x7f5b7c0a4d60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5b88005fd0 tx=0x7f5b88009f10 comp rx=0 tx=0).stop 2024-03-16T09:18:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.965+0000 7f5b97863640 1 -- 172.21.15.112:0/3709284375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b90101830 msgr2=0x7f5b90197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.965+0000 7f5b97863640 1 --2- 172.21.15.112:0/3709284375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b90101830 0x7f5b90197b20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c000c00 tx=0x7f5b8c004a30 comp rx=0 tx=0).stop 2024-03-16T09:18:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.965+0000 7f5b97863640 1 -- 172.21.15.112:0/3709284375 shutdown_connections 2024-03-16T09:18:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.965+0000 7f5b97863640 1 --2- 172.21.15.112:0/3709284375 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f5b7c0a28a0 0x7f5b7c0a4d60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.965+0000 7f5b97863640 1 --2- 172.21.15.112:0/3709284375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b901033e0 0x7f5b9019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.974 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.965+0000 7f5b97863640 1 --2- 172.21.15.112:0/3709284375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5b90102a20 0x7f5b90198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.965+0000 7f5b97863640 1 --2- 172.21.15.112:0/3709284375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b90101830 0x7f5b90197b20 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.965+0000 7f5b97863640 1 -- 172.21.15.112:0/3709284375 >> 172.21.15.112:0/3709284375 conn(0x7f5b900fcda0 msgr2=0x7f5b90105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.966+0000 7f5b97863640 1 -- 172.21.15.112:0/3709284375 shutdown_connections 2024-03-16T09:18:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-03-16T09:18:23.966+0000 7f5b97863640 1 -- 172.21.15.112:0/3709284375 wait complete. 2024-03-16T09:18:24.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:24 smithi194 ceph-mon[36729]: pgmap v434: 97 pgs: 97 active+clean; 63 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 960 op/s 2024-03-16T09:18:24.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:24 smithi159 ceph-mon[36739]: pgmap v434: 97 pgs: 97 active+clean; 63 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 960 op/s 2024-03-16T09:18:24.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:24 smithi112 ceph-mon[32307]: pgmap v434: 97 pgs: 97 active+clean; 63 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 960 op/s 2024-03-16T09:18:25.417 INFO:tasks.workunit:Stopping ['fs/norstats'] on client.0... 2024-03-16T09:18:25.418 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-16T09:18:25.685 DEBUG:teuthology.parallel:result is None 2024-03-16T09:18:25.685 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-16T09:18:25.722 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-16T09:18:25.722 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-16T09:18:25.733 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi112.front.sepia.ceph.com 2024-03-16T09:18:25.770 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-16T09:18:25.781 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-16T09:18:25.781 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-03-16T09:18:25.782 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-16T09:18:25.814 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:18:25.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.921+0000 7fd486c8a640 1 -- 172.21.15.112:0/4027342532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4801022b0 msgr2=0x7fd480108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:25.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.921+0000 7fd486c8a640 1 --2- 172.21.15.112:0/4027342532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4801022b0 0x7fd480108b40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd478009ca0 tx=0x7fd47802c0f0 comp rx=0 tx=0).stop 2024-03-16T09:18:25.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.921+0000 7fd486c8a640 1 -- 172.21.15.112:0/4027342532 shutdown_connections 2024-03-16T09:18:25.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.921+0000 7fd486c8a640 1 --2- 172.21.15.112:0/4027342532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd4801022b0 0x7fd480108b40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:25.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.921+0000 7fd486c8a640 1 --2- 172.21.15.112:0/4027342532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4801018f0 0x7fd480101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:25.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.921+0000 7fd486c8a640 1 --2- 172.21.15.112:0/4027342532 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd480100700 0x7fd480100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:25.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.922+0000 7fd486c8a640 1 -- 172.21.15.112:0/4027342532 >> 172.21.15.112:0/4027342532 conn(0x7fd4800fbc70 msgr2=0x7fd4800fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:25.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.922+0000 7fd486c8a640 1 -- 172.21.15.112:0/4027342532 shutdown_connections 2024-03-16T09:18:25.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.922+0000 7fd486c8a640 1 -- 172.21.15.112:0/4027342532 wait complete. 2024-03-16T09:18:25.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.922+0000 7fd486c8a640 1 Processor -- start 2024-03-16T09:18:25.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.922+0000 7fd486c8a640 1 -- start start 2024-03-16T09:18:25.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.922+0000 7fd486c8a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd480100700 0x7fd4800694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:25.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.922+0000 7fd486c8a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4801018f0 0x7fd48019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:25.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.922+0000 7fd486c8a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4801022b0 0x7fd48019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:25.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.922+0000 7fd486c8a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd48010c4b0 con 0x7fd480100700 2024-03-16T09:18:25.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.922+0000 7fd486c8a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd48010c330 con 0x7fd4801022b0 2024-03-16T09:18:25.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.922+0000 7fd486c8a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd48010c630 con 0x7fd4801018f0 2024-03-16T09:18:25.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.922+0000 7fd484a26640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd480100700 0x7fd4800694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:25.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.922+0000 7fd484a26640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd480100700 0x7fd4800694d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52230/0 (socket says 172.21.15.112:52230) 2024-03-16T09:18:25.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.922+0000 7fd484a26640 1 -- 172.21.15.112:0/55257129 learned_addr learned my addr 172.21.15.112:0/55257129 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:18:25.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.922+0000 7fd485227640 1 --2- 172.21.15.112:0/55257129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4801022b0 0x7fd48019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:25.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.922+0000 7fd477fff640 1 --2- 172.21.15.112:0/55257129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4801018f0 0x7fd48019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:25.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.923+0000 7fd484a26640 1 -- 172.21.15.112:0/55257129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4801018f0 msgr2=0x7fd48019c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:25.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.923+0000 7fd484a26640 1 --2- 172.21.15.112:0/55257129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4801018f0 0x7fd48019c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:25.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.923+0000 7fd484a26640 1 -- 172.21.15.112:0/55257129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4801022b0 msgr2=0x7fd48019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:25.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.923+0000 7fd484a26640 1 --2- 172.21.15.112:0/55257129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4801022b0 0x7fd48019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:25.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.923+0000 7fd484a26640 1 -- 172.21.15.112:0/55257129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48019f790 con 0x7fd480100700 2024-03-16T09:18:25.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.923+0000 7fd485227640 1 --2- 172.21.15.112:0/55257129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4801022b0 0x7fd48019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:18:25.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.923+0000 7fd484a26640 1 --2- 172.21.15.112:0/55257129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd480100700 0x7fd4800694d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00e9c0 tx=0x7fd47c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:25.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.923+0000 7fd475ffb640 1 -- 172.21.15.112:0/55257129 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c00cd80 con 0x7fd480100700 2024-03-16T09:18:25.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.923+0000 7fd475ffb640 1 -- 172.21.15.112:0/55257129 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd47c0035f0 con 0x7fd480100700 2024-03-16T09:18:25.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.923+0000 7fd475ffb640 1 -- 172.21.15.112:0/55257129 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c010670 con 0x7fd480100700 2024-03-16T09:18:25.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.923+0000 7fd486c8a640 1 -- 172.21.15.112:0/55257129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48019fa80 con 0x7fd480100700 2024-03-16T09:18:25.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.923+0000 7fd486c8a640 1 -- 172.21.15.112:0/55257129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48019ff80 con 0x7fd480100700 2024-03-16T09:18:25.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.924+0000 7fd486c8a640 1 -- 172.21.15.112:0/55257129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd480107950 con 0x7fd480100700 2024-03-16T09:18:25.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.925+0000 7fd475ffb640 1 -- 172.21.15.112:0/55257129 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7fd47c0037a0 con 0x7fd480100700 2024-03-16T09:18:25.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.926+0000 7fd475ffb640 1 --2- 172.21.15.112:0/55257129 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd4680a2800 0x7fd4680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:25.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.926+0000 7fd477fff640 1 --2- 172.21.15.112:0/55257129 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd4680a2800 0x7fd4680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:25.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.926+0000 7fd475ffb640 1 -- 172.21.15.112:0/55257129 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fd47c0cee00 con 0x7fd480100700 2024-03-16T09:18:25.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.926+0000 7fd477fff640 1 --2- 172.21.15.112:0/55257129 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd4680a2800 0x7fd4680a4cc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd480069240 tx=0x7fd470005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:25.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:25.930+0000 7fd475ffb640 1 -- 172.21.15.112:0/55257129 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd47c09be50 con 0x7fd480100700 2024-03-16T09:18:26.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.107+0000 7fd486c8a640 1 -- 172.21.15.112:0/55257129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4800623e0 con 0x7fd480100700 2024-03-16T09:18:26.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.109+0000 7fd475ffb640 1 -- 172.21.15.112:0/55257129 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22725 (secure 0 0 0) 0x7fd47c09b750 con 0x7fd480100700 2024-03-16T09:18:26.112 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:18:26.112 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":88,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:17:28.259672+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T09:13:35.040628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T09:13:35.689419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:18:26.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.111+0000 7fd486c8a640 1 -- 172.21.15.112:0/55257129 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd4680a2800 msgr2=0x7fd4680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:26.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.111+0000 7fd486c8a640 1 --2- 172.21.15.112:0/55257129 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd4680a2800 0x7fd4680a4cc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd480069240 tx=0x7fd470005f90 comp rx=0 tx=0).stop 2024-03-16T09:18:26.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.111+0000 7fd486c8a640 1 -- 172.21.15.112:0/55257129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd480100700 msgr2=0x7fd4800694d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:26.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.111+0000 7fd486c8a640 1 --2- 172.21.15.112:0/55257129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd480100700 0x7fd4800694d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00e9c0 tx=0x7fd47c00ee80 comp rx=0 tx=0).stop 2024-03-16T09:18:26.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.111+0000 7fd486c8a640 1 -- 172.21.15.112:0/55257129 shutdown_connections 2024-03-16T09:18:26.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.111+0000 7fd486c8a640 1 --2- 172.21.15.112:0/55257129 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd4680a2800 0x7fd4680a4cc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.111+0000 7fd486c8a640 1 --2- 172.21.15.112:0/55257129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4801022b0 0x7fd48019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.111+0000 7fd486c8a640 1 --2- 172.21.15.112:0/55257129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4801018f0 0x7fd48019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.111+0000 7fd486c8a640 1 --2- 172.21.15.112:0/55257129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd480100700 0x7fd4800694d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.111+0000 7fd486c8a640 1 -- 172.21.15.112:0/55257129 >> 172.21.15.112:0/55257129 conn(0x7fd4800fbc70 msgr2=0x7fd480104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:26.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.111+0000 7fd486c8a640 1 -- 172.21.15.112:0/55257129 shutdown_connections 2024-03-16T09:18:26.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.111+0000 7fd486c8a640 1 -- 172.21.15.112:0/55257129 wait complete. 2024-03-16T09:18:26.126 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T09:08:57.135312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T09:13:35.040628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T09:13:35.689419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T09:18:26.126 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T09:18:26.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.227+0000 7f158d0ac640 1 -- 172.21.15.112:0/3748101643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15881018f0 msgr2=0x7f1588101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:26.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.227+0000 7f158d0ac640 1 --2- 172.21.15.112:0/3748101643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15881018f0 0x7f1588101d70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1570009900 tx=0x7f157002bc30 comp rx=0 tx=0).stop 2024-03-16T09:18:26.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.227+0000 7f158d0ac640 1 -- 172.21.15.112:0/3748101643 shutdown_connections 2024-03-16T09:18:26.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.227+0000 7f158d0ac640 1 --2- 172.21.15.112:0/3748101643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15881022b0 0x7f1588108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.227+0000 7f158d0ac640 1 --2- 172.21.15.112:0/3748101643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15881018f0 0x7f1588101d70 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.227+0000 7f158d0ac640 1 --2- 172.21.15.112:0/3748101643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1588100700 0x7f1588100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.227+0000 7f158d0ac640 1 -- 172.21.15.112:0/3748101643 >> 172.21.15.112:0/3748101643 conn(0x7f15880fbc90 msgr2=0x7f15880fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:26.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.227+0000 7f158d0ac640 1 -- 172.21.15.112:0/3748101643 shutdown_connections 2024-03-16T09:18:26.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.227+0000 7f158d0ac640 1 -- 172.21.15.112:0/3748101643 wait complete. 2024-03-16T09:18:26.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f158d0ac640 1 Processor -- start 2024-03-16T09:18:26.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f158d0ac640 1 -- start start 2024-03-16T09:18:26.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f158d0ac640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1588100700 0x7f158806a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:26.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f158d0ac640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15881018f0 0x7f158819c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:26.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f158d0ac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15881022b0 0x7f158819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:26.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f158d0ac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f158810c460 con 0x7f15881022b0 2024-03-16T09:18:26.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f158d0ac640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f158810c2e0 con 0x7f15881018f0 2024-03-16T09:18:26.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f158d0ac640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f158810c5e0 con 0x7f1588100700 2024-03-16T09:18:26.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f1586d9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1588100700 0x7f158806a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:26.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f158759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15881022b0 0x7f158819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:26.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f1586d9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1588100700 0x7f158806a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.112:58460/0 (socket says 172.21.15.112:58460) 2024-03-16T09:18:26.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f1586d9d640 1 -- 172.21.15.112:0/525421764 learned_addr learned my addr 172.21.15.112:0/525421764 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:18:26.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f157ffff640 1 --2- 172.21.15.112:0/525421764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15881018f0 0x7f158819c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:26.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f1586d9d640 1 -- 172.21.15.112:0/525421764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15881018f0 msgr2=0x7f158819c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:26.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f1586d9d640 1 --2- 172.21.15.112:0/525421764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15881018f0 0x7f158819c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f1586d9d640 1 -- 172.21.15.112:0/525421764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15881022b0 msgr2=0x7f158819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:26.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f1586d9d640 1 --2- 172.21.15.112:0/525421764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15881022b0 0x7f158819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f1586d9d640 1 -- 172.21.15.112:0/525421764 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158819f810 con 0x7f1588100700 2024-03-16T09:18:26.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f158759e640 1 --2- 172.21.15.112:0/525421764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15881022b0 0x7f158819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:18:26.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.228+0000 7f1586d9d640 1 --2- 172.21.15.112:0/525421764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1588100700 0x7f158806a940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f157800e9c0 tx=0x7f157800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:26.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.229+0000 7f1584d99640 1 -- 172.21.15.112:0/525421764 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157800cd80 con 0x7f1588100700 2024-03-16T09:18:26.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.229+0000 7f1584d99640 1 -- 172.21.15.112:0/525421764 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f15780035f0 con 0x7f1588100700 2024-03-16T09:18:26.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.229+0000 7f158d0ac640 1 -- 172.21.15.112:0/525421764 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158819fb00 con 0x7f1588100700 2024-03-16T09:18:26.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.229+0000 7f1584d99640 1 -- 172.21.15.112:0/525421764 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578010670 con 0x7f1588100700 2024-03-16T09:18:26.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.229+0000 7f158d0ac640 1 -- 172.21.15.112:0/525421764 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f158819ffd0 con 0x7f1588100700 2024-03-16T09:18:26.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.230+0000 7f158d0ac640 1 -- 172.21.15.112:0/525421764 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1588107a20 con 0x7f1588100700 2024-03-16T09:18:26.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.231+0000 7f1584d99640 1 -- 172.21.15.112:0/525421764 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7f15780037a0 con 0x7f1588100700 2024-03-16T09:18:26.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.231+0000 7f1584d99640 1 --2- 172.21.15.112:0/525421764 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f15740a2800 0x7f15740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:26.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.231+0000 7f157ffff640 1 --2- 172.21.15.112:0/525421764 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f15740a2800 0x7f15740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:26.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.231+0000 7f1584d99640 1 -- 172.21.15.112:0/525421764 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f15780cee20 con 0x7f1588100700 2024-03-16T09:18:26.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.232+0000 7f157ffff640 1 --2- 172.21.15.112:0/525421764 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f15740a2800 0x7f15740a4cc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f157002c900 tx=0x7f1570005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:26.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.235+0000 7f1584d99640 1 -- 172.21.15.112:0/525421764 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f157809bed0 con 0x7f1588100700 2024-03-16T09:18:26.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:26 smithi194 ceph-mon[36729]: pgmap v435: 97 pgs: 97 active+clean; 54 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 735 op/s 2024-03-16T09:18:26.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.415+0000 7f158d0ac640 1 -- 172.21.15.112:0/525421764 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f158802d420 con 0x7f1588100700 2024-03-16T09:18:26.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.415+0000 7f1584d99640 1 -- 172.21.15.112:0/525421764 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f157809b7d0 con 0x7f1588100700 2024-03-16T09:18:26.418 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-03-16T09:18:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.417+0000 7f158d0ac640 1 -- 172.21.15.112:0/525421764 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f15740a2800 msgr2=0x7f15740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.417+0000 7f158d0ac640 1 --2- 172.21.15.112:0/525421764 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f15740a2800 0x7f15740a4cc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f157002c900 tx=0x7f1570005fd0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.417+0000 7f158d0ac640 1 -- 172.21.15.112:0/525421764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1588100700 msgr2=0x7f158806a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.417+0000 7f158d0ac640 1 --2- 172.21.15.112:0/525421764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1588100700 0x7f158806a940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f157800e9c0 tx=0x7f157800ee80 comp rx=0 tx=0).stop 2024-03-16T09:18:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.417+0000 7f158d0ac640 1 -- 172.21.15.112:0/525421764 shutdown_connections 2024-03-16T09:18:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.417+0000 7f158d0ac640 1 --2- 172.21.15.112:0/525421764 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f15740a2800 0x7f15740a4cc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.417+0000 7f158d0ac640 1 --2- 172.21.15.112:0/525421764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15881022b0 0x7f158819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.417+0000 7f158d0ac640 1 --2- 172.21.15.112:0/525421764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15881018f0 0x7f158819c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.417+0000 7f158d0ac640 1 --2- 172.21.15.112:0/525421764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1588100700 0x7f158806a940 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.417+0000 7f158d0ac640 1 -- 172.21.15.112:0/525421764 >> 172.21.15.112:0/525421764 conn(0x7f15880fbc90 msgr2=0x7f1588104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.417+0000 7f158d0ac640 1 -- 172.21.15.112:0/525421764 shutdown_connections 2024-03-16T09:18:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.417+0000 7f158d0ac640 1 -- 172.21.15.112:0/525421764 wait complete. 2024-03-16T09:18:26.430 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:26 smithi112 ceph-mon[32307]: pgmap v435: 97 pgs: 97 active+clean; 54 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 735 op/s 2024-03-16T09:18:26.432 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T09:18:26.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:26 smithi159 ceph-mon[36739]: pgmap v435: 97 pgs: 97 active+clean; 54 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 735 op/s 2024-03-16T09:18:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.534+0000 7f2eaa8c6640 1 -- 172.21.15.112:0/1116280811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea41018d0 msgr2=0x7f2ea4101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.534+0000 7f2eaa8c6640 1 --2- 172.21.15.112:0/1116280811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea41018d0 0x7f2ea4101d50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009900 tx=0x7f2e9802bc30 comp rx=0 tx=0).stop 2024-03-16T09:18:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.534+0000 7f2eaa8c6640 1 -- 172.21.15.112:0/1116280811 shutdown_connections 2024-03-16T09:18:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.534+0000 7f2eaa8c6640 1 --2- 172.21.15.112:0/1116280811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ea4102290 0x7f2ea4108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.534+0000 7f2eaa8c6640 1 --2- 172.21.15.112:0/1116280811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea41018d0 0x7f2ea4101d50 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.534+0000 7f2eaa8c6640 1 --2- 172.21.15.112:0/1116280811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ea41006e0 0x7f2ea4100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.534+0000 7f2eaa8c6640 1 -- 172.21.15.112:0/1116280811 >> 172.21.15.112:0/1116280811 conn(0x7f2ea40fbc70 msgr2=0x7f2ea40fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.534+0000 7f2eaa8c6640 1 -- 172.21.15.112:0/1116280811 shutdown_connections 2024-03-16T09:18:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.534+0000 7f2eaa8c6640 1 -- 172.21.15.112:0/1116280811 wait complete. 2024-03-16T09:18:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.534+0000 7f2eaa8c6640 1 Processor -- start 2024-03-16T09:18:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.534+0000 7f2eaa8c6640 1 -- start start 2024-03-16T09:18:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2eaa8c6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ea41006e0 0x7f2ea4104600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2eaa8c6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ea41018d0 0x7f2ea4102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2eaa8c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea4102290 0x7f2ea41031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2eaa8c6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea410c440 con 0x7f2ea4102290 2024-03-16T09:18:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2eaa8c6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea410c2c0 con 0x7f2ea41018d0 2024-03-16T09:18:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2eaa8c6640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea410c5c0 con 0x7f2ea41006e0 2024-03-16T09:18:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2ea8e63640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea4102290 0x7f2ea41031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2ea8e63640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea4102290 0x7f2ea41031f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52268/0 (socket says 172.21.15.112:52268) 2024-03-16T09:18:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2ea8e63640 1 -- 172.21.15.112:0/2887457867 learned_addr learned my addr 172.21.15.112:0/2887457867 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:18:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2ea3fff640 1 --2- 172.21.15.112:0/2887457867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ea41006e0 0x7f2ea4104600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:26.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2ea37fe640 1 --2- 172.21.15.112:0/2887457867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ea41018d0 0x7f2ea4102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:26.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2ea8e63640 1 -- 172.21.15.112:0/2887457867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ea41006e0 msgr2=0x7f2ea4104600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:26.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2ea8e63640 1 --2- 172.21.15.112:0/2887457867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ea41006e0 0x7f2ea4104600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2ea8e63640 1 -- 172.21.15.112:0/2887457867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ea41018d0 msgr2=0x7f2ea4102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:26.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2ea8e63640 1 --2- 172.21.15.112:0/2887457867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ea41018d0 0x7f2ea4102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2ea8e63640 1 -- 172.21.15.112:0/2887457867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea4103a10 con 0x7f2ea4102290 2024-03-16T09:18:26.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2ea8e63640 1 --2- 172.21.15.112:0/2887457867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea4102290 0x7f2ea41031f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00eb10 tx=0x7f2e9c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:26.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2ea17fa640 1 -- 172.21.15.112:0/2887457867 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c004b20 con 0x7f2ea4102290 2024-03-16T09:18:26.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2eaa8c6640 1 -- 172.21.15.112:0/2887457867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea41a1d00 con 0x7f2ea4102290 2024-03-16T09:18:26.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.535+0000 7f2ea17fa640 1 -- 172.21.15.112:0/2887457867 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2e9c004cc0 con 0x7f2ea4102290 2024-03-16T09:18:26.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.536+0000 7f2ea17fa640 1 -- 172.21.15.112:0/2887457867 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c0107f0 con 0x7f2ea4102290 2024-03-16T09:18:26.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.536+0000 7f2eaa8c6640 1 -- 172.21.15.112:0/2887457867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea41a21d0 con 0x7f2ea4102290 2024-03-16T09:18:26.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.536+0000 7f2eaa8c6640 1 -- 172.21.15.112:0/2887457867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea41a2400 con 0x7f2ea4102290 2024-03-16T09:18:26.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.538+0000 7f2ea17fa640 1 -- 172.21.15.112:0/2887457867 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7f2e9c005020 con 0x7f2ea4102290 2024-03-16T09:18:26.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.538+0000 7f2ea17fa640 1 --2- 172.21.15.112:0/2887457867 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2e8c0a2800 0x7f2e8c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:26.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.538+0000 7f2ea3fff640 1 --2- 172.21.15.112:0/2887457867 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2e8c0a2800 0x7f2e8c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:26.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.538+0000 7f2ea17fa640 1 -- 172.21.15.112:0/2887457867 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f2e9c0cf000 con 0x7f2ea4102290 2024-03-16T09:18:26.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.539+0000 7f2ea3fff640 1 --2- 172.21.15.112:0/2887457867 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2e8c0a2800 0x7f2e8c0a4cc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4101730 tx=0x7f2e94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:26.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.542+0000 7f2ea17fa640 1 -- 172.21.15.112:0/2887457867 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2e9c09c0b0 con 0x7f2ea4102290 2024-03-16T09:18:26.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.715+0000 7f2eaa8c6640 1 -- 172.21.15.112:0/2887457867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2ea41a2790 con 0x7f2ea4102290 2024-03-16T09:18:26.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.716+0000 7f2ea17fa640 1 -- 172.21.15.112:0/2887457867 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f2e9c09b9b0 con 0x7f2ea4102290 2024-03-16T09:18:26.719 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-03-16T09:18:26.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.718+0000 7f2eaa8c6640 1 -- 172.21.15.112:0/2887457867 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2e8c0a2800 msgr2=0x7f2e8c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:26.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.718+0000 7f2eaa8c6640 1 --2- 172.21.15.112:0/2887457867 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2e8c0a2800 0x7f2e8c0a4cc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4101730 tx=0x7f2e94005f90 comp rx=0 tx=0).stop 2024-03-16T09:18:26.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.718+0000 7f2eaa8c6640 1 -- 172.21.15.112:0/2887457867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea4102290 msgr2=0x7f2ea41031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:26.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.718+0000 7f2eaa8c6640 1 --2- 172.21.15.112:0/2887457867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea4102290 0x7f2ea41031f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00eb10 tx=0x7f2e9c00c2d0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.718+0000 7f2eaa8c6640 1 -- 172.21.15.112:0/2887457867 shutdown_connections 2024-03-16T09:18:26.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.718+0000 7f2eaa8c6640 1 --2- 172.21.15.112:0/2887457867 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f2e8c0a2800 0x7f2e8c0a4cc0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.718+0000 7f2eaa8c6640 1 --2- 172.21.15.112:0/2887457867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea4102290 0x7f2ea41031f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.718+0000 7f2eaa8c6640 1 --2- 172.21.15.112:0/2887457867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ea41018d0 0x7f2ea4102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.718+0000 7f2eaa8c6640 1 --2- 172.21.15.112:0/2887457867 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ea41006e0 0x7f2ea4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.718+0000 7f2eaa8c6640 1 -- 172.21.15.112:0/2887457867 >> 172.21.15.112:0/2887457867 conn(0x7f2ea40fbc70 msgr2=0x7f2ea40fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:26.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.718+0000 7f2eaa8c6640 1 -- 172.21.15.112:0/2887457867 shutdown_connections 2024-03-16T09:18:26.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.718+0000 7f2eaa8c6640 1 -- 172.21.15.112:0/2887457867 wait complete. 2024-03-16T09:18:26.732 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T09:18:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.834+0000 7f0500377640 1 -- 172.21.15.112:0/1843871903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f81018e0 msgr2=0x7f04f8101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.834+0000 7f0500377640 1 --2- 172.21.15.112:0/1843871903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f81018e0 0x7f04f8101d60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009900 tx=0x7f04e802bc30 comp rx=0 tx=0).stop 2024-03-16T09:18:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.834+0000 7f0500377640 1 -- 172.21.15.112:0/1843871903 shutdown_connections 2024-03-16T09:18:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.834+0000 7f0500377640 1 --2- 172.21.15.112:0/1843871903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04f81022a0 0x7f04f8108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.834+0000 7f0500377640 1 --2- 172.21.15.112:0/1843871903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f81018e0 0x7f04f8101d60 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.834+0000 7f0500377640 1 --2- 172.21.15.112:0/1843871903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04f81006f0 0x7f04f8100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.834+0000 7f0500377640 1 -- 172.21.15.112:0/1843871903 >> 172.21.15.112:0/1843871903 conn(0x7f04f80fbc80 msgr2=0x7f04f80fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.834+0000 7f0500377640 1 -- 172.21.15.112:0/1843871903 shutdown_connections 2024-03-16T09:18:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.834+0000 7f0500377640 1 -- 172.21.15.112:0/1843871903 wait complete. 2024-03-16T09:18:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.835+0000 7f0500377640 1 Processor -- start 2024-03-16T09:18:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.835+0000 7f0500377640 1 -- start start 2024-03-16T09:18:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.835+0000 7f0500377640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04f81006f0 0x7f04f8197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.835+0000 7f0500377640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f81018e0 0x7f04f81980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.835+0000 7f0500377640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04f81022a0 0x7f04f819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.835+0000 7f0500377640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f04f810c450 con 0x7f04f81018e0 2024-03-16T09:18:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.835+0000 7f0500377640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f04f810c2d0 con 0x7f04f81006f0 2024-03-16T09:18:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.835+0000 7f0500377640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f04f810c5d0 con 0x7f04f81022a0 2024-03-16T09:18:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.835+0000 7f04fd912640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f81018e0 0x7f04f81980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.835+0000 7f04fd912640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f81018e0 0x7f04f81980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52290/0 (socket says 172.21.15.112:52290) 2024-03-16T09:18:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.835+0000 7f04fd912640 1 -- 172.21.15.112:0/1710751687 learned_addr learned my addr 172.21.15.112:0/1710751687 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:18:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.835+0000 7f04fe113640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04f81006f0 0x7f04f8197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.835+0000 7f04fe914640 1 --2- 172.21.15.112:0/1710751687 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04f81022a0 0x7f04f819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.835+0000 7f04fd912640 1 -- 172.21.15.112:0/1710751687 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04f81022a0 msgr2=0x7f04f819f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.835+0000 7f04fd912640 1 --2- 172.21.15.112:0/1710751687 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04f81022a0 0x7f04f819f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.835+0000 7f04fd912640 1 -- 172.21.15.112:0/1710751687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04f81006f0 msgr2=0x7f04f8197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.835+0000 7f04fd912640 1 --2- 172.21.15.112:0/1710751687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04f81006f0 0x7f04f8197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:26.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.835+0000 7f04fd912640 1 -- 172.21.15.112:0/1710751687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f819f7d0 con 0x7f04f81018e0 2024-03-16T09:18:26.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.835+0000 7f04fe113640 1 --2- 172.21.15.112:0/1710751687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04f81006f0 0x7f04f8197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:18:26.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.836+0000 7f04fd912640 1 --2- 172.21.15.112:0/1710751687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f81018e0 0x7f04f81980f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009230 tx=0x7f04e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:26.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.836+0000 7f04ef7fe640 1 -- 172.21.15.112:0/1710751687 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8043020 con 0x7f04f81018e0 2024-03-16T09:18:26.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.836+0000 7f04ef7fe640 1 -- 172.21.15.112:0/1710751687 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f04e8004250 con 0x7f04f81018e0 2024-03-16T09:18:26.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.836+0000 7f0500377640 1 -- 172.21.15.112:0/1710751687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f819fa60 con 0x7f04f81018e0 2024-03-16T09:18:26.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.836+0000 7f04ef7fe640 1 -- 172.21.15.112:0/1710751687 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8003b50 con 0x7f04f81018e0 2024-03-16T09:18:26.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.836+0000 7f0500377640 1 -- 172.21.15.112:0/1710751687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f819ffb0 con 0x7f04f81018e0 2024-03-16T09:18:26.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.837+0000 7f0500377640 1 -- 172.21.15.112:0/1710751687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f8068a20 con 0x7f04f81018e0 2024-03-16T09:18:26.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.838+0000 7f04ef7fe640 1 -- 172.21.15.112:0/1710751687 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7f04e8046020 con 0x7f04f81018e0 2024-03-16T09:18:26.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.838+0000 7f04ef7fe640 1 --2- 172.21.15.112:0/1710751687 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f04e40a2800 0x7f04e40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:26.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.838+0000 7f04fe113640 1 --2- 172.21.15.112:0/1710751687 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f04e40a2800 0x7f04e40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:26.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.839+0000 7f04ef7fe640 1 -- 172.21.15.112:0/1710751687 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f04e80f0f40 con 0x7f04f81018e0 2024-03-16T09:18:26.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.839+0000 7f04fe113640 1 --2- 172.21.15.112:0/1710751687 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f04e40a2800 0x7f04e40a4cc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f04f8101740 tx=0x7f04f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:26.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:26.842+0000 7f04ef7fe640 1 -- 172.21.15.112:0/1710751687 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f04e804d020 con 0x7f04f81018e0 2024-03-16T09:18:27.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.024+0000 7f0500377640 1 -- 172.21.15.112:0/1710751687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f04f81a03e0 con 0x7f04f81018e0 2024-03-16T09:18:27.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.024+0000 7f04ef7fe640 1 -- 172.21.15.112:0/1710751687 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f04e80bdff0 con 0x7f04f81018e0 2024-03-16T09:18:27.028 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-03-16T09:18:27.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.026+0000 7f0500377640 1 -- 172.21.15.112:0/1710751687 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f04e40a2800 msgr2=0x7f04e40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:27.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.026+0000 7f0500377640 1 --2- 172.21.15.112:0/1710751687 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f04e40a2800 0x7f04e40a4cc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f04f8101740 tx=0x7f04f4005f90 comp rx=0 tx=0).stop 2024-03-16T09:18:27.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.027+0000 7f0500377640 1 -- 172.21.15.112:0/1710751687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f81018e0 msgr2=0x7f04f81980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:27.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.027+0000 7f0500377640 1 --2- 172.21.15.112:0/1710751687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f81018e0 0x7f04f81980f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009230 tx=0x7f04e8004ab0 comp rx=0 tx=0).stop 2024-03-16T09:18:27.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.027+0000 7f0500377640 1 -- 172.21.15.112:0/1710751687 shutdown_connections 2024-03-16T09:18:27.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.027+0000 7f0500377640 1 --2- 172.21.15.112:0/1710751687 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f04e40a2800 0x7f04e40a4cc0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:27.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.027+0000 7f0500377640 1 --2- 172.21.15.112:0/1710751687 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f04f81022a0 0x7f04f819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:27.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.027+0000 7f0500377640 1 --2- 172.21.15.112:0/1710751687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04f81018e0 0x7f04f81980f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:27.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.027+0000 7f0500377640 1 --2- 172.21.15.112:0/1710751687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04f81006f0 0x7f04f8197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:27.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.027+0000 7f0500377640 1 -- 172.21.15.112:0/1710751687 >> 172.21.15.112:0/1710751687 conn(0x7f04f80fbc80 msgr2=0x7f04f80fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:27.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.027+0000 7f0500377640 1 -- 172.21.15.112:0/1710751687 shutdown_connections 2024-03-16T09:18:27.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.027+0000 7f0500377640 1 -- 172.21.15.112:0/1710751687 wait complete. 2024-03-16T09:18:27.041 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-16T09:18:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.141+0000 7fb1278f8640 1 -- 172.21.15.112:0/1548738222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1201022d0 msgr2=0x7fb120108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.141+0000 7fb1278f8640 1 --2- 172.21.15.112:0/1548738222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1201022d0 0x7fb120108b60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb118009ca0 tx=0x7fb11802c0f0 comp rx=0 tx=0).stop 2024-03-16T09:18:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.141+0000 7fb1278f8640 1 -- 172.21.15.112:0/1548738222 shutdown_connections 2024-03-16T09:18:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.141+0000 7fb1278f8640 1 --2- 172.21.15.112:0/1548738222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1201022d0 0x7fb120108b60 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.141+0000 7fb1278f8640 1 --2- 172.21.15.112:0/1548738222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb120101910 0x7fb120101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.141+0000 7fb1278f8640 1 --2- 172.21.15.112:0/1548738222 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb120100720 0x7fb120100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.141+0000 7fb1278f8640 1 -- 172.21.15.112:0/1548738222 >> 172.21.15.112:0/1548738222 conn(0x7fb1200fbc90 msgr2=0x7fb1200fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.141+0000 7fb1278f8640 1 -- 172.21.15.112:0/1548738222 shutdown_connections 2024-03-16T09:18:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.141+0000 7fb1278f8640 1 -- 172.21.15.112:0/1548738222 wait complete. 2024-03-16T09:18:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.141+0000 7fb1278f8640 1 Processor -- start 2024-03-16T09:18:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.141+0000 7fb1278f8640 1 -- start start 2024-03-16T09:18:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb1278f8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb120100720 0x7fb120197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb1278f8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb120101910 0x7fb120198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb1278f8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1201022d0 0x7fb12019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb1278f8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb12010d5e0 con 0x7fb1201022d0 2024-03-16T09:18:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb1278f8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb12010d460 con 0x7fb120101910 2024-03-16T09:18:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb1278f8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb12010d760 con 0x7fb120100720 2024-03-16T09:18:27.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb125e95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1201022d0 0x7fb12019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:27.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb125e95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1201022d0 0x7fb12019f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52300/0 (socket says 172.21.15.112:52300) 2024-03-16T09:18:27.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb125e95640 1 -- 172.21.15.112:0/1447446242 learned_addr learned my addr 172.21.15.112:0/1447446242 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:18:27.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb125694640 1 --2- 172.21.15.112:0/1447446242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb120100720 0x7fb120197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:27.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb124e93640 1 --2- 172.21.15.112:0/1447446242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb120101910 0x7fb120198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:27.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb125e95640 1 -- 172.21.15.112:0/1447446242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb120100720 msgr2=0x7fb120197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:27.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb125e95640 1 --2- 172.21.15.112:0/1447446242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb120100720 0x7fb120197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:27.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb125e95640 1 -- 172.21.15.112:0/1447446242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb120101910 msgr2=0x7fb120198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:27.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb125e95640 1 --2- 172.21.15.112:0/1447446242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb120101910 0x7fb120198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:27.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb125e95640 1 -- 172.21.15.112:0/1447446242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12019f7e0 con 0x7fb1201022d0 2024-03-16T09:18:27.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb125e95640 1 --2- 172.21.15.112:0/1447446242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1201022d0 0x7fb12019f170 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb11802c700 tx=0x7fb1180346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:27.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb1167fc640 1 -- 172.21.15.112:0/1447446242 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118039040 con 0x7fb1201022d0 2024-03-16T09:18:27.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb1167fc640 1 -- 172.21.15.112:0/1447446242 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb118044070 con 0x7fb1201022d0 2024-03-16T09:18:27.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb1278f8640 1 -- 172.21.15.112:0/1447446242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb12019fa70 con 0x7fb1201022d0 2024-03-16T09:18:27.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb1167fc640 1 -- 172.21.15.112:0/1447446242 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1180043c0 con 0x7fb1201022d0 2024-03-16T09:18:27.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.142+0000 7fb1278f8640 1 -- 172.21.15.112:0/1447446242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb12019ff60 con 0x7fb1201022d0 2024-03-16T09:18:27.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.143+0000 7fb1278f8640 1 -- 172.21.15.112:0/1447446242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1201079e0 con 0x7fb1201022d0 2024-03-16T09:18:27.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.144+0000 7fb1167fc640 1 -- 172.21.15.112:0/1447446242 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7fb118004560 con 0x7fb1201022d0 2024-03-16T09:18:27.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.145+0000 7fb1167fc640 1 --2- 172.21.15.112:0/1447446242 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb10c0a2800 0x7fb10c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:18:27.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.145+0000 7fb1167fc640 1 -- 172.21.15.112:0/1447446242 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fb1180f1690 con 0x7fb1201022d0 2024-03-16T09:18:27.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.145+0000 7fb125694640 1 --2- 172.21.15.112:0/1447446242 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb10c0a2800 0x7fb10c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:18:27.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.145+0000 7fb125694640 1 --2- 172.21.15.112:0/1447446242 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb10c0a2800 0x7fb10c0a4cc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb120101770 tx=0x7fb110005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:18:27.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.148+0000 7fb1167fc640 1 -- 172.21.15.112:0/1447446242 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb1180be860 con 0x7fb1201022d0 2024-03-16T09:18:27.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:27 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/55257129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:18:27.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:27 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/525421764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:18:27.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:27 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2887457867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:18:27.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:27 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1710751687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:18:27.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.375+0000 7fb1278f8640 1 -- 172.21.15.112:0/1447446242 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fb12002d330 con 0x7fb1201022d0 2024-03-16T09:18:27.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.383+0000 7fb1167fc640 1 -- 172.21.15.112:0/1447446242 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+739838 (secure 0 0 0) 0x7fb1180be160 con 0x7fb1201022d0 2024-03-16T09:18:27.396 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:18:27.396 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:18:27.396 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 20, 2024-03-16T09:18:27.396 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.396 INFO:teuthology.orchestra.run.smithi112.stdout: "active_gid": 14152, 2024-03-16T09:18:27.396 INFO:teuthology.orchestra.run.smithi112.stdout: "active_name": "x", 2024-03-16T09:18:27.397 INFO:teuthology.orchestra.run.smithi112.stdout: "active_addrs": { 2024-03-16T09:18:27.397 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-03-16T09:18:27.397 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.397 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-03-16T09:18:27.397 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6800", 2024-03-16T09:18:27.397 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 2628870229 2024-03-16T09:18:27.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.397 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.397 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-03-16T09:18:27.397 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6801", 2024-03-16T09:18:27.397 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 2628870229 2024-03-16T09:18:27.397 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.397 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-03-16T09:18:27.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.398 INFO:teuthology.orchestra.run.smithi112.stdout: "active_addr": "172.21.15.112:6801/2628870229", 2024-03-16T09:18:27.398 INFO:teuthology.orchestra.run.smithi112.stdout: "active_change": "2024-03-16T09:06:30.793104+0000", 2024-03-16T09:18:27.398 INFO:teuthology.orchestra.run.smithi112.stdout: "active_mgr_features": 4540701547738038271, 2024-03-16T09:18:27.398 INFO:teuthology.orchestra.run.smithi112.stdout: "available": true, 2024-03-16T09:18:27.398 INFO:teuthology.orchestra.run.smithi112.stdout: "standbys": [ 2024-03-16T09:18:27.398 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.398 INFO:teuthology.orchestra.run.smithi112.stdout: "gid": 24112, 2024-03-16T09:18:27.398 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "z", 2024-03-16T09:18:27.398 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr_features": 4540701547738038271, 2024-03-16T09:18:27.398 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-03-16T09:18:27.398 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.398 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-03-16T09:18:27.398 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.399 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.399 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.399 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-03-16T09:18:27.399 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-03-16T09:18:27.399 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.399 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.399 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.399 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-03-16T09:18:27.399 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.399 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.399 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.399 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-03-16T09:18:27.399 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.399 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.399 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.400 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.400 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.400 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.400 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.400 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.400 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.400 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.400 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.400 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.400 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.400 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.400 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.400 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.401 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.401 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.401 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.401 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.401 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.401 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.401 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.401 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.401 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.401 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.401 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.401 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.401 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.402 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.402 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.402 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.402 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.402 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.402 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.402 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.402 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.402 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.402 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.402 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.402 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.403 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.403 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.403 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.403 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.403 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.403 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.403 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.403 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.403 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.403 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.403 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.403 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.403 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.403 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.403 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.404 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.404 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.404 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.404 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.404 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.404 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.404 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.404 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.404 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.404 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.404 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.404 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.404 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.405 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-03-16T09:18:27.405 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-03-16T09:18:27.405 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.405 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.405 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.405 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.405 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.405 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.405 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.405 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to", 2024-03-16T09:18:27.405 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.405 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.405 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.406 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-03-16T09:18:27.406 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-03-16T09:18:27.406 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.406 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.406 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.406 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-03-16T09:18:27.406 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.406 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.406 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.406 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-03-16T09:18:27.406 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.406 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.407 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.407 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-03-16T09:18:27.407 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-03-16T09:18:27.407 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.407 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.407 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.407 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.407 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.407 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.407 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.407 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-03-16T09:18:27.407 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.407 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.407 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.408 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-03-16T09:18:27.408 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-03-16T09:18:27.408 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.408 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.408 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.408 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.408 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.408 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.408 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.408 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-03-16T09:18:27.408 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.408 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.408 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.408 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-03-16T09:18:27.409 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-03-16T09:18:27.409 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.409 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.409 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.409 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-03-16T09:18:27.409 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.409 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.409 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.409 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-03-16T09:18:27.409 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.409 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.409 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.409 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-03-16T09:18:27.409 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-03-16T09:18:27.410 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.410 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.410 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.410 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.410 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.410 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.410 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.410 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-03-16T09:18:27.410 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.410 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.410 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.411 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-03-16T09:18:27.411 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-03-16T09:18:27.411 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.411 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.411 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.411 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.411 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.411 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.411 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.411 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-16T09:18:27.411 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.411 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.411 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.411 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-03-16T09:18:27.412 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-03-16T09:18:27.412 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.412 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.412 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.412 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.412 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.412 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.412 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.412 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-03-16T09:18:27.412 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.412 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.412 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.412 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.412 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.412 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.413 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-03-16T09:18:27.413 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.413 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.413 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.413 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-03-16T09:18:27.413 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-03-16T09:18:27.413 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.413 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.413 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.413 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.413 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.413 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.413 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.413 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-03-16T09:18:27.413 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.414 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.414 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.414 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-03-16T09:18:27.414 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-03-16T09:18:27.414 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.414 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.414 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.414 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-03-16T09:18:27.414 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.414 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.414 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.414 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-03-16T09:18:27.414 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-16T09:18:27.415 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.415 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.415 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-03-16T09:18:27.415 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-03-16T09:18:27.415 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-03-16T09:18:27.415 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.415 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.415 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-03-16T09:18:27.415 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-03-16T09:18:27.415 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-03-16T09:18:27.415 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.415 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-16T09:18:27.415 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-16T09:18:27.416 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.416 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.416 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-03-16T09:18:27.416 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-03-16T09:18:27.416 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-03-16T09:18:27.416 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.416 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.416 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-03-16T09:18:27.416 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-03-16T09:18:27.416 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-03-16T09:18:27.416 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.416 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-16T09:18:27.416 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.416 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.417 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.417 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-03-16T09:18:27.417 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-03-16T09:18:27.417 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.417 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.417 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.417 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-03-16T09:18:27.417 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.417 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.417 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.417 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-16T09:18:27.417 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-16T09:18:27.417 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.418 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.418 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-03-16T09:18:27.418 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-03-16T09:18:27.418 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:27.418 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.418 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.418 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-03-16T09:18:27.418 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-03-16T09:18:27.418 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-03-16T09:18:27.418 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.418 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-03-16T09:18:27.418 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-16T09:18:27.418 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.419 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.419 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-03-16T09:18:27.419 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-03-16T09:18:27.419 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.419 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.419 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.419 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-03-16T09:18:27.419 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.419 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.419 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.419 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-03-16T09:18:27.419 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-16T09:18:27.419 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.420 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.420 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-03-16T09:18:27.420 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-03-16T09:18:27.420 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-03-16T09:18:27.420 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.420 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.420 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-03-16T09:18:27.420 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-03-16T09:18:27.420 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-03-16T09:18:27.420 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.420 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-16T09:18:27.420 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-16T09:18:27.420 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.421 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.421 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.421 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.421 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.421 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.421 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.421 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.421 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.421 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.421 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.421 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.421 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.421 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.421 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.422 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.422 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.422 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.422 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.422 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.422 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.422 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.422 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.422 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.422 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.422 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.422 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.422 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.423 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.423 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.423 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.423 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.423 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.423 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.423 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.423 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.423 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.423 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.423 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.423 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.423 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.424 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.424 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.424 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.424 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.424 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.424 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.424 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.424 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.424 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.424 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.424 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.424 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.424 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.424 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.425 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.425 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.425 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.425 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.425 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.425 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.425 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.425 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.425 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.425 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.425 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.425 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.425 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.426 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-03-16T09:18:27.426 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-03-16T09:18:27.426 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:27.426 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.426 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.426 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-03-16T09:18:27.426 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.426 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.426 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.426 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-16T09:18:27.426 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.426 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.426 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.427 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-03-16T09:18:27.427 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-03-16T09:18:27.427 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.427 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.427 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.427 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-03-16T09:18:27.427 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.427 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.427 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.427 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-03-16T09:18:27.427 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-03-16T09:18:27.427 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-03-16T09:18:27.427 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-03-16T09:18:27.427 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-03-16T09:18:27.428 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.428 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-03-16T09:18:27.428 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.428 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.428 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.428 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-03-16T09:18:27.428 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-03-16T09:18:27.428 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.428 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.428 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.428 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.428 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.428 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.428 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.429 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-16T09:18:27.429 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.429 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.429 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.429 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-03-16T09:18:27.429 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-03-16T09:18:27.429 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.429 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.429 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.429 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-03-16T09:18:27.429 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.429 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.429 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.429 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-16T09:18:27.429 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.429 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.430 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.430 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-03-16T09:18:27.430 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-03-16T09:18:27.430 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.430 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.430 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.430 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-03-16T09:18:27.430 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-03-16T09:18:27.430 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.430 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.430 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-16T09:18:27.430 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-16T09:18:27.430 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.431 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.431 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-03-16T09:18:27.431 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-03-16T09:18:27.431 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-03-16T09:18:27.431 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.431 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.431 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:27.431 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.431 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.431 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.431 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-16T09:18:27.431 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.431 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.431 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.431 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.431 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.432 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.432 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-03-16T09:18:27.432 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.432 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.432 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.432 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-03-16T09:18:27.432 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-03-16T09:18:27.432 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:27.432 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.432 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.432 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-03-16T09:18:27.432 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.432 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.432 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.432 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-16T09:18:27.433 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.433 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.433 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.433 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-03-16T09:18:27.433 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-03-16T09:18:27.433 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.433 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.433 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.433 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-03-16T09:18:27.433 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.433 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.433 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.433 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-16T09:18:27.433 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.433 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.434 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.434 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-03-16T09:18:27.434 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-03-16T09:18:27.434 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.434 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.434 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.434 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-03-16T09:18:27.434 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.434 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.434 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.434 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-16T09:18:27.434 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.435 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.435 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.435 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-03-16T09:18:27.435 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-03-16T09:18:27.435 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.435 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.435 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.435 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.435 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.435 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.435 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.435 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-16T09:18:27.435 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-16T09:18:27.435 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.435 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.436 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-03-16T09:18:27.436 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-03-16T09:18:27.436 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.436 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.436 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.436 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-03-16T09:18:27.436 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.436 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.436 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.436 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-16T09:18:27.436 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.436 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.436 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.437 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-03-16T09:18:27.437 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-03-16T09:18:27.437 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:27.437 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.437 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.437 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-03-16T09:18:27.437 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.437 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.437 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.437 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-16T09:18:27.437 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.437 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.437 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.438 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-03-16T09:18:27.438 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-03-16T09:18:27.438 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.438 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.438 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.438 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.438 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.438 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.438 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.438 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-03-16T09:18:27.438 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-03-16T09:18:27.438 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-03-16T09:18:27.438 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.438 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-16T09:18:27.439 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.439 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.439 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.439 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-03-16T09:18:27.439 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-03-16T09:18:27.439 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.439 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.439 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.439 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.439 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.439 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.439 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.441 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-16T09:18:27.441 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.441 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.441 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.441 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-03-16T09:18:27.441 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-03-16T09:18:27.441 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.441 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.441 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.441 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.441 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.441 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.442 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.442 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-16T09:18:27.442 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.442 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.442 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.442 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-03-16T09:18:27.442 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-03-16T09:18:27.442 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.442 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.442 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.442 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.442 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.442 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.443 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.443 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-16T09:18:27.443 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.443 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.443 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.443 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-03-16T09:18:27.443 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-03-16T09:18:27.443 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.443 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.443 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.443 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-16T09:18:27.443 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.443 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.443 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.444 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-03-16T09:18:27.444 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.444 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.444 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.444 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-03-16T09:18:27.444 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-03-16T09:18:27.444 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.444 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.444 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.444 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-16T09:18:27.444 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.444 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.444 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.444 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-03-16T09:18:27.445 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.445 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.445 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.445 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-03-16T09:18:27.445 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-03-16T09:18:27.445 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.445 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.445 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.445 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-16T09:18:27.445 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.445 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.445 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.445 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-03-16T09:18:27.445 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.445 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.446 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.446 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-03-16T09:18:27.446 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-03-16T09:18:27.446 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.446 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.446 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.446 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-16T09:18:27.446 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.446 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.446 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.446 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-03-16T09:18:27.446 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.446 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.446 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.447 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-03-16T09:18:27.447 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-03-16T09:18:27.447 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.447 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.447 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.447 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-16T09:18:27.447 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.447 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.447 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.447 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-03-16T09:18:27.447 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.447 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.447 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.448 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-03-16T09:18:27.448 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-03-16T09:18:27.448 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.448 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.448 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.448 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-16T09:18:27.448 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.448 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.448 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.448 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-03-16T09:18:27.448 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.448 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.448 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.448 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-03-16T09:18:27.449 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-03-16T09:18:27.449 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.449 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.449 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.449 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-16T09:18:27.449 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.449 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.449 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.449 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-03-16T09:18:27.449 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.449 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.449 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.449 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-03-16T09:18:27.449 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-03-16T09:18:27.449 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.450 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.450 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.450 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-16T09:18:27.450 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.450 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.450 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.450 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-03-16T09:18:27.450 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.450 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.450 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.450 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-03-16T09:18:27.450 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-03-16T09:18:27.450 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.450 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.451 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.451 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-16T09:18:27.451 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.451 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.451 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.451 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-03-16T09:18:27.451 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.451 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.451 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.451 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-03-16T09:18:27.451 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-03-16T09:18:27.451 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.451 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.451 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.451 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-16T09:18:27.452 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.452 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.452 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.452 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-03-16T09:18:27.452 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.452 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.452 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.452 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-03-16T09:18:27.452 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-03-16T09:18:27.452 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.452 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.452 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.452 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-16T09:18:27.452 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.453 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.453 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.453 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-03-16T09:18:27.453 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.453 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.453 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.453 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-03-16T09:18:27.453 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-03-16T09:18:27.453 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.453 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.453 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.453 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-16T09:18:27.453 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.453 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.453 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.454 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-03-16T09:18:27.454 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.454 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.454 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.454 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-03-16T09:18:27.454 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-03-16T09:18:27.454 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.454 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.454 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.454 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-16T09:18:27.454 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.454 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.454 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.455 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-03-16T09:18:27.455 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.455 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.455 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.455 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-03-16T09:18:27.455 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-03-16T09:18:27.455 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.455 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.455 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.455 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-16T09:18:27.455 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.455 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.455 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.456 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-03-16T09:18:27.456 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.456 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.456 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.456 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-03-16T09:18:27.456 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-03-16T09:18:27.456 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.456 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.456 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.456 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-16T09:18:27.456 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.456 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.457 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.457 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-03-16T09:18:27.457 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.457 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.457 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.457 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-03-16T09:18:27.457 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-03-16T09:18:27.457 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.457 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.457 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.457 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.457 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.457 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.458 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.458 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-03-16T09:18:27.458 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.458 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.458 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.458 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-03-16T09:18:27.458 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-03-16T09:18:27.458 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.458 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.458 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.458 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-03-16T09:18:27.458 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.459 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.459 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.459 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-16T09:18:27.459 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.459 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.459 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.459 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-03-16T09:18:27.459 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-03-16T09:18:27.459 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.459 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.459 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.459 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-03-16T09:18:27.459 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.460 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.460 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.460 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-16T09:18:27.460 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.460 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.460 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.460 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-03-16T09:18:27.460 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-03-16T09:18:27.460 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.460 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.460 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.460 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-03-16T09:18:27.460 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.460 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.461 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.461 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-16T09:18:27.461 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.461 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.461 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.461 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-03-16T09:18:27.461 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-03-16T09:18:27.461 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.461 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.461 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.461 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-03-16T09:18:27.461 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.461 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.462 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.462 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-03-16T09:18:27.462 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.462 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.462 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.462 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-03-16T09:18:27.462 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-03-16T09:18:27.462 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.462 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.462 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.462 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.462 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.463 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.463 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.463 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-16T09:18:27.463 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.463 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.463 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.463 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-03-16T09:18:27.463 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-03-16T09:18:27.463 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.463 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.463 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.463 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-03-16T09:18:27.463 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.463 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.464 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.464 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-03-16T09:18:27.464 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.464 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.464 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.464 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-03-16T09:18:27.464 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-03-16T09:18:27.464 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.464 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.464 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.464 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-03-16T09:18:27.464 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.464 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.464 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.465 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-03-16T09:18:27.465 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.465 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.465 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.465 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-03-16T09:18:27.465 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-03-16T09:18:27.465 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.465 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.465 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.465 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.465 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.465 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.465 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.465 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-16T09:18:27.465 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.466 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.466 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.466 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-03-16T09:18:27.466 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-03-16T09:18:27.466 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.466 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.466 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.466 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.466 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.466 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.466 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.466 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-16T09:18:27.466 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.466 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.467 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.467 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.467 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.467 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.467 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.467 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.467 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.467 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.467 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.467 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.467 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.467 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.467 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.467 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.468 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.468 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.468 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.468 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.468 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.468 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.468 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.468 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-03-16T09:18:27.468 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-03-16T09:18:27.468 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.468 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.468 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.468 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.468 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.468 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.469 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.469 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-16T09:18:27.469 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.469 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.469 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.469 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.469 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.469 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.469 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.469 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.469 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.469 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.469 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.469 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.469 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-16T09:18:27.470 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.470 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.470 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.470 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.470 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.470 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.470 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.470 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.470 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.470 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.470 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.470 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.471 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.471 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.471 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.471 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.471 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.471 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.471 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.471 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.471 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.471 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.471 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.471 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.471 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.471 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.471 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.472 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.472 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.472 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.472 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.472 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.472 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.472 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.472 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.472 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.472 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-16T09:18:27.472 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-16T09:18:27.472 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.472 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.472 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.473 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.473 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.473 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.473 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.473 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-16T09:18:27.473 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.473 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.473 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.473 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-16T09:18:27.473 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-16T09:18:27.473 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.473 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.473 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.473 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-03-16T09:18:27.474 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.474 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.474 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.474 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-16T09:18:27.474 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.474 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.474 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.474 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-03-16T09:18:27.474 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-03-16T09:18:27.474 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.474 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.474 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.475 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:27.475 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.475 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.475 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.475 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-03-16T09:18:27.475 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.475 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.475 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.475 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-03-16T09:18:27.475 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-03-16T09:18:27.475 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.475 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.475 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.475 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:27.476 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.476 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.476 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.476 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-16T09:18:27.476 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.476 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.476 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.476 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-03-16T09:18:27.476 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-03-16T09:18:27.476 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.476 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.476 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.476 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.476 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.476 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.477 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.477 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-03-16T09:18:27.477 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.477 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.477 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.477 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-03-16T09:18:27.477 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-03-16T09:18:27.477 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.477 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.477 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.477 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-03-16T09:18:27.477 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.477 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.477 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.478 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-03-16T09:18:27.478 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-03-16T09:18:27.478 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.478 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-03-16T09:18:27.478 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.478 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.478 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.478 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-03-16T09:18:27.478 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-03-16T09:18:27.478 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.478 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.478 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.478 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-03-16T09:18:27.478 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.479 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.479 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.479 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-16T09:18:27.479 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.479 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.479 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.479 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-03-16T09:18:27.479 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-03-16T09:18:27.479 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.479 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.479 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.479 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-16T09:18:27.479 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.480 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.480 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.480 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-16T09:18:27.480 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.480 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.480 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.480 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-03-16T09:18:27.480 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-03-16T09:18:27.480 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.480 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.480 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.480 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.480 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.480 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.481 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.481 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-16T09:18:27.481 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.481 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.481 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.481 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-03-16T09:18:27.481 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-03-16T09:18:27.481 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.481 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.481 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.481 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.481 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.481 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.481 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.482 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-16T09:18:27.482 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.482 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.482 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.482 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-03-16T09:18:27.482 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-03-16T09:18:27.482 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.482 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.482 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.482 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.482 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.482 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.483 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.483 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-16T09:18:27.483 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.483 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.483 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.483 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-03-16T09:18:27.483 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-03-16T09:18:27.483 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.483 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.483 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.483 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.483 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.483 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.483 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.483 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-16T09:18:27.484 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.484 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.484 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.484 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-03-16T09:18:27.484 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-03-16T09:18:27.484 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.484 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.484 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.484 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.484 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.484 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.484 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.485 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-16T09:18:27.485 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.485 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.485 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.485 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-03-16T09:18:27.485 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-03-16T09:18:27.485 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.485 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.485 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.485 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-03-16T09:18:27.485 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.485 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.485 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.485 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-03-16T09:18:27.485 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.486 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.486 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.486 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-03-16T09:18:27.486 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-03-16T09:18:27.486 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.486 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.486 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.486 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.486 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.486 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.486 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.486 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-16T09:18:27.486 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.486 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.487 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.487 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-03-16T09:18:27.487 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-03-16T09:18:27.487 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.487 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.487 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.487 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.487 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.487 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.487 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.487 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-16T09:18:27.487 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.487 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.487 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.488 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-03-16T09:18:27.488 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-03-16T09:18:27.488 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.488 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.488 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.488 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.488 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.488 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.488 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.488 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-16T09:18:27.488 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.488 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.488 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.488 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-03-16T09:18:27.489 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-03-16T09:18:27.489 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.489 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.489 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.489 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.489 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.489 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.489 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.489 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-03-16T09:18:27.489 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.489 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.489 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.489 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-03-16T09:18:27.489 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-03-16T09:18:27.490 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.490 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.490 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.490 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.490 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.490 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.490 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.490 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-16T09:18:27.490 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.490 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.490 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.490 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-03-16T09:18:27.491 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-03-16T09:18:27.491 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.491 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.491 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.491 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.491 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.491 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.491 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.491 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-16T09:18:27.491 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.491 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.491 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.491 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.491 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.492 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.492 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-03-16T09:18:27.492 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.492 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.492 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.492 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.492 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.492 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.492 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.492 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.492 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.492 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.492 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.492 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.492 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.493 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.493 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.493 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.493 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.493 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.493 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.493 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.493 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.493 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.493 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.493 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.493 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.493 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.493 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.493 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.494 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.494 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.494 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.494 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.494 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.494 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.494 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.494 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.494 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.494 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.494 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.494 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.495 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.495 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.495 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.495 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.495 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.495 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.495 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.495 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.495 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.495 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.495 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.495 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.495 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.495 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.496 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.496 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.496 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.496 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.496 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.496 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.496 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.496 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.496 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.496 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.496 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.496 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.496 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.496 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.497 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.497 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.497 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-03-16T09:18:27.497 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-03-16T09:18:27.497 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.497 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.497 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.497 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-03-16T09:18:27.497 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.497 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.497 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.497 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-16T09:18:27.497 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.497 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.497 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.498 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-03-16T09:18:27.498 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-03-16T09:18:27.498 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.498 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.498 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.498 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-03-16T09:18:27.498 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.498 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.498 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.498 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-16T09:18:27.498 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.498 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.498 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.498 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.499 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.499 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.499 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-03-16T09:18:27.499 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.499 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.499 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.499 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-16T09:18:27.499 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-16T09:18:27.499 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.499 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.499 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.499 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:27.499 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.499 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.500 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.500 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.500 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.500 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.500 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.500 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-03-16T09:18:27.500 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-16T09:18:27.500 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.500 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.500 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.500 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.500 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.500 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.500 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.500 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.501 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.501 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.501 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.501 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-16T09:18:27.501 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-16T09:18:27.501 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.501 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.501 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.501 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.501 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.501 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.501 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.501 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.501 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.502 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.502 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.502 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-03-16T09:18:27.502 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-03-16T09:18:27.502 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.502 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.502 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.502 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.502 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.502 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.502 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.502 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.502 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.502 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.503 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.503 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-16T09:18:27.503 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-16T09:18:27.503 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.503 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.503 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.503 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.503 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.503 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.503 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.503 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.503 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.503 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.503 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.504 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-03-16T09:18:27.504 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-16T09:18:27.504 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.504 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.504 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.504 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.504 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.504 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.504 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.504 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.504 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.504 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.504 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-16T09:18:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-16T09:18:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-16T09:18:27.505 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-16T09:18:27.506 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.506 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.506 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.506 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.506 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.506 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.506 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.506 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.506 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.506 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.506 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.506 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-16T09:18:27.507 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-16T09:18:27.507 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.507 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.507 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.507 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.507 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.507 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.507 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.507 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.507 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.507 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.507 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.507 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-16T09:18:27.507 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-16T09:18:27.507 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.508 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.508 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.508 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.508 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.508 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.508 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.508 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.508 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.508 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.508 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.508 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-16T09:18:27.508 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-16T09:18:27.508 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.509 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.509 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.509 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.509 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.509 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.509 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.509 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.509 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.509 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.509 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.509 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-16T09:18:27.509 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-16T09:18:27.509 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.510 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.510 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.510 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.510 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.510 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.510 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.510 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.510 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.510 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.510 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.510 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-16T09:18:27.510 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-16T09:18:27.511 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.511 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.511 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.511 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.511 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.511 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.511 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.511 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.511 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.511 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.511 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.511 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-16T09:18:27.511 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-16T09:18:27.511 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.512 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.512 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.512 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.512 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.512 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.512 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.512 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.512 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.512 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.512 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.512 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-03-16T09:18:27.512 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-16T09:18:27.512 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.512 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.513 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.513 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-03-16T09:18:27.513 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.513 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.513 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.513 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.513 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.513 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.513 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.513 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-16T09:18:27.513 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-16T09:18:27.513 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.513 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.514 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.514 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.514 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.514 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.514 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.514 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.514 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.514 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.514 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.514 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-03-16T09:18:27.514 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-03-16T09:18:27.514 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.515 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.515 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.515 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.515 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.515 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.515 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.515 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.515 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.515 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.515 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.515 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-03-16T09:18:27.515 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-16T09:18:27.515 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.515 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.516 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.516 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-03-16T09:18:27.516 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.516 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.516 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.516 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.516 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.516 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.516 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.516 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-16T09:18:27.516 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-16T09:18:27.516 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.516 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.517 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.517 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.517 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.517 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.517 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.517 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.517 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.517 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.517 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.517 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-16T09:18:27.517 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-16T09:18:27.517 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.517 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.517 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.518 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.518 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.518 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.518 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.518 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.518 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.518 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.518 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.518 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-16T09:18:27.518 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-16T09:18:27.518 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.518 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.518 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.519 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.519 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.519 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.519 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.519 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.519 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.519 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.519 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.519 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-16T09:18:27.519 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-16T09:18:27.519 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.519 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.519 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.520 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.520 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.520 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.520 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.520 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.520 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.520 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.520 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.520 INFO:teuthology.orchestra.run.smithi112.stdout: "MULTICLUSTER_CONFIG": { 2024-03-16T09:18:27.520 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-16T09:18:27.520 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.520 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.520 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.521 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "{}", 2024-03-16T09:18:27.521 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.521 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.521 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.521 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.521 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.521 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.521 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.521 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-03-16T09:18:27.521 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-16T09:18:27.521 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.521 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.521 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.522 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.522 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.522 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.522 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.522 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.522 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.522 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.522 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.522 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-16T09:18:27.522 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-16T09:18:27.522 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.522 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.523 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.523 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.523 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.523 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.523 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.523 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.523 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.523 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.523 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.523 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-16T09:18:27.523 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-16T09:18:27.523 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.523 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.524 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.524 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.524 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.524 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.524 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.524 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.524 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.524 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.524 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.524 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-16T09:18:27.524 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-16T09:18:27.524 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.524 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-16T09:18:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-16T09:18:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.525 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-16T09:18:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-16T09:18:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.526 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-16T09:18:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-16T09:18:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.527 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-16T09:18:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-16T09:18:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.528 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-16T09:18:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-16T09:18:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.529 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-03-16T09:18:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-16T09:18:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.530 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-16T09:18:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-16T09:18:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-16T09:18:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.531 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-16T09:18:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-16T09:18:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.532 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-16T09:18:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-16T09:18:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-03-16T09:18:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.533 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-16T09:18:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-16T09:18:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-03-16T09:18:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.534 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-03-16T09:18:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-16T09:18:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.535 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-16T09:18:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-16T09:18:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-03-16T09:18:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.536 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-03-16T09:18:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-16T09:18:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.537 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.538 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-03-16T09:18:27.538 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-16T09:18:27.538 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.538 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.538 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.538 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.538 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.538 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.538 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.538 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.538 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.538 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.538 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.539 INFO:teuthology.orchestra.run.smithi112.stdout: "UNSAFE_TLS_v1_2": { 2024-03-16T09:18:27.539 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-16T09:18:27.539 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.539 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.539 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.539 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.539 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.539 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.539 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.539 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.539 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.539 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.539 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.540 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-16T09:18:27.540 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-16T09:18:27.540 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.540 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.540 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.540 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-03-16T09:18:27.540 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.540 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.540 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.540 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.540 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.540 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.540 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.541 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-16T09:18:27.541 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-16T09:18:27.541 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.541 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.541 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.541 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:27.541 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.541 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.541 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.541 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.541 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.541 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.541 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.541 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-16T09:18:27.542 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-16T09:18:27.542 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.542 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.542 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.542 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-03-16T09:18:27.542 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.542 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.542 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.542 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.542 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.542 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.542 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.542 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-03-16T09:18:27.542 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-03-16T09:18:27.543 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.543 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.543 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.543 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.543 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.543 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.543 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.543 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.543 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.543 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.543 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.543 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-03-16T09:18:27.543 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-03-16T09:18:27.543 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.544 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.544 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.544 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.544 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.544 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.544 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.544 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.544 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.544 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.544 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.544 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-03-16T09:18:27.544 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-03-16T09:18:27.544 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.544 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.544 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.545 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.545 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.545 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.545 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.545 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-03-16T09:18:27.545 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.545 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.545 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.545 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-03-16T09:18:27.545 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-03-16T09:18:27.545 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.545 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.545 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.546 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-03-16T09:18:27.546 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.546 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.546 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.546 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.546 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.546 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.546 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.546 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-03-16T09:18:27.546 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-03-16T09:18:27.546 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.546 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.546 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.547 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.547 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.547 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.547 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.547 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.547 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.547 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.547 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.547 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.547 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.547 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.547 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.547 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.548 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.548 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.548 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.548 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.548 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.548 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.548 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.548 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.548 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.548 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.548 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.548 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.548 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.548 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.548 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.549 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.549 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.549 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.549 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.549 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.549 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.549 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.549 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.549 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.549 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.549 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.549 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.549 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.550 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.550 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.550 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.550 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.550 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.550 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.550 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.550 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.550 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.550 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.550 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.550 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.550 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.551 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.551 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.551 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.551 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.551 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.551 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.551 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.551 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.551 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.551 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.551 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.551 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.551 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.551 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.551 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.552 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.552 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.552 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.552 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.552 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.552 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-03-16T09:18:27.552 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-03-16T09:18:27.552 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.552 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.552 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.552 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.552 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.552 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.552 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.553 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-03-16T09:18:27.553 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.553 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.553 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.553 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-03-16T09:18:27.553 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-03-16T09:18:27.553 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.553 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.553 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.553 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.553 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.553 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.553 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.553 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.554 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.554 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.554 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.554 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-03-16T09:18:27.554 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-03-16T09:18:27.554 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.554 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.554 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.554 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-03-16T09:18:27.554 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.554 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.554 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.555 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.555 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.555 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.555 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.555 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-03-16T09:18:27.555 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-03-16T09:18:27.555 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.555 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.555 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.555 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-03-16T09:18:27.555 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.555 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.555 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.555 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.555 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.556 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.556 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.556 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-03-16T09:18:27.556 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-03-16T09:18:27.556 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.556 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.556 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.556 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.556 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.556 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.556 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.556 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.556 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.556 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.557 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.557 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-03-16T09:18:27.557 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-03-16T09:18:27.557 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.557 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.557 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.557 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-03-16T09:18:27.557 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.557 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.557 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.557 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.557 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.557 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.557 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.558 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-03-16T09:18:27.558 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-03-16T09:18:27.558 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.558 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.558 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.558 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-03-16T09:18:27.558 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.558 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.558 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.558 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.558 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-03-16T09:18:27.559 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.559 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.559 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.559 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.559 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.559 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-03-16T09:18:27.559 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-03-16T09:18:27.559 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.559 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.559 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.559 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-03-16T09:18:27.559 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-03-16T09:18:27.559 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-03-16T09:18:27.559 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.559 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.560 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.560 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.560 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.560 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.560 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-03-16T09:18:27.560 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-03-16T09:18:27.560 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.560 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.560 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.560 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.560 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.560 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.560 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.560 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.561 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.561 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.561 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.561 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.561 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.561 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.561 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-03-16T09:18:27.561 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.561 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.561 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.561 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-03-16T09:18:27.561 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-03-16T09:18:27.561 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.561 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.562 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.562 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.562 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.562 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.562 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.562 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-03-16T09:18:27.562 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.562 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.562 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.562 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.562 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.562 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.562 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.563 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.563 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.563 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.563 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.563 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.563 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.563 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.563 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.563 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.563 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.563 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.563 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.563 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.563 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.563 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.564 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.564 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.564 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.564 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.564 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.564 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.564 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.564 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.564 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.564 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.564 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.564 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.564 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.564 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.564 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.565 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.565 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.565 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.565 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.565 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.565 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.565 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.565 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.565 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.565 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.565 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.565 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.565 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.565 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.566 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.566 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.566 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.566 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.566 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.566 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.566 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.566 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.566 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.566 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.566 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.566 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.566 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.566 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.567 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.567 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.567 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.567 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.567 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.567 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.567 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-03-16T09:18:27.567 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-03-16T09:18:27.567 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.567 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.567 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.567 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-03-16T09:18:27.567 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.567 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.568 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.568 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-16T09:18:27.568 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.568 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.568 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.568 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-03-16T09:18:27.568 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-03-16T09:18:27.568 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.568 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.568 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.568 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-03-16T09:18:27.568 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.568 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.568 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.568 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-16T09:18:27.569 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.569 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.569 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.569 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-03-16T09:18:27.569 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-03-16T09:18:27.569 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.569 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.569 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.569 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-03-16T09:18:27.569 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.569 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.569 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.569 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-03-16T09:18:27.569 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.570 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.570 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.570 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-03-16T09:18:27.570 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-03-16T09:18:27.570 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.570 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.570 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.570 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-03-16T09:18:27.570 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.570 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.570 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.570 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-16T09:18:27.570 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.571 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.571 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.571 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-03-16T09:18:27.571 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-03-16T09:18:27.571 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.571 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.571 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.571 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.571 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.571 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.571 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.571 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-16T09:18:27.571 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.572 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.572 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.572 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-03-16T09:18:27.572 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-03-16T09:18:27.572 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.572 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.572 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.572 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-03-16T09:18:27.572 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.572 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.572 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.572 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-03-16T09:18:27.572 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.572 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.573 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.573 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.573 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-03-16T09:18:27.573 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-03-16T09:18:27.573 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.573 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.573 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.573 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-03-16T09:18:27.573 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.573 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.573 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.573 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-16T09:18:27.573 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.573 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.573 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.574 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.574 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.574 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.574 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-03-16T09:18:27.574 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.574 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.574 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.574 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.574 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.574 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.574 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.574 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.574 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.575 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.575 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.575 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.575 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.575 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.575 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.575 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.575 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.575 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.575 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.575 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.575 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.575 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.575 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.575 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.576 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.576 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.576 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.576 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.576 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.576 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.576 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.576 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.576 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.576 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.576 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.576 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.576 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.576 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.577 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.577 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.577 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.577 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.577 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.577 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.577 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.577 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.577 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.577 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.577 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.577 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.577 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.577 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.577 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.577 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.578 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.578 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.578 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.578 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.578 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.578 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.578 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.578 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.578 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.578 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.578 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.578 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.578 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.578 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.578 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.579 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.579 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.579 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.579 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-03-16T09:18:27.579 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-03-16T09:18:27.579 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.579 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.579 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.579 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-03-16T09:18:27.579 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.579 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.579 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.579 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.579 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.580 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.580 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.580 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-03-16T09:18:27.580 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-03-16T09:18:27.580 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.580 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.580 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.580 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-03-16T09:18:27.580 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.580 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.580 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.580 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.580 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.580 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.581 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.581 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.581 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-03-16T09:18:27.581 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-03-16T09:18:27.581 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.581 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.581 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.581 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-03-16T09:18:27.581 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.581 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.581 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.581 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.581 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.581 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.582 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.582 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.582 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.582 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.582 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-03-16T09:18:27.582 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-03-16T09:18:27.582 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-03-16T09:18:27.582 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.582 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-03-16T09:18:27.582 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-03-16T09:18:27.582 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.582 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.582 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.583 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-03-16T09:18:27.583 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.583 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.583 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.583 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-16T09:18:27.583 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.583 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.583 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.583 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-03-16T09:18:27.583 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-03-16T09:18:27.583 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.583 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.583 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.583 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-03-16T09:18:27.584 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.584 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.584 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.584 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-16T09:18:27.584 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.584 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.584 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.584 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-03-16T09:18:27.584 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-03-16T09:18:27.584 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.584 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.584 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.584 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.585 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.585 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.585 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.585 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-03-16T09:18:27.585 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.585 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.585 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.585 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-03-16T09:18:27.585 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-03-16T09:18:27.585 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.585 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.585 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.585 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-03-16T09:18:27.585 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-03-16T09:18:27.585 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.586 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.586 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-16T09:18:27.586 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.586 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.586 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.586 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.586 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.586 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.586 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.586 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.586 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.586 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.586 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.586 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.587 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.587 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.587 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.587 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.587 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.587 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.587 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.587 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.587 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.587 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.587 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.587 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.587 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.587 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.587 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.587 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.588 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.588 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.588 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.588 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.588 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.588 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.588 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.588 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.588 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.588 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.588 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.588 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.588 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.588 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.588 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.589 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.589 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.589 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.589 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.589 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.589 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.589 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.589 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.589 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.589 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.589 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.589 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.589 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.589 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.589 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.590 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.590 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.590 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.590 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.590 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.590 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.590 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.590 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.590 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.590 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.590 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.590 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.590 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.590 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.591 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-03-16T09:18:27.591 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-03-16T09:18:27.591 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.591 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.591 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.591 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.591 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.591 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.591 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.591 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-03-16T09:18:27.591 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.591 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.591 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.591 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-03-16T09:18:27.592 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-03-16T09:18:27.592 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.592 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.592 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.592 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-03-16T09:18:27.592 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.592 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.592 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.592 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-03-16T09:18:27.592 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.592 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.592 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.593 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-03-16T09:18:27.593 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-03-16T09:18:27.593 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.593 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.593 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.593 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-03-16T09:18:27.593 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.593 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.593 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.593 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-16T09:18:27.593 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.593 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.593 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.594 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-03-16T09:18:27.594 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-03-16T09:18:27.594 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.594 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.594 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.594 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-03-16T09:18:27.594 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-03-16T09:18:27.594 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-03-16T09:18:27.594 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.594 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-16T09:18:27.594 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.594 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.594 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.595 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-03-16T09:18:27.595 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-03-16T09:18:27.595 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.595 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.595 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.595 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.595 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.595 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.595 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.595 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-03-16T09:18:27.595 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.595 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.595 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.596 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-03-16T09:18:27.596 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-03-16T09:18:27.596 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.596 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.596 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.596 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-03-16T09:18:27.596 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.596 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.596 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.596 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-16T09:18:27.596 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.596 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.597 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.597 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.597 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.597 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.597 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-03-16T09:18:27.597 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.597 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.597 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.597 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.597 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.597 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.597 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.597 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.597 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.598 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.598 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.598 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.598 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.598 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.598 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.598 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.598 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.598 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.598 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.598 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.598 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.598 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.599 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.599 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.599 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.599 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.599 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.599 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.599 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.599 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.599 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.599 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.599 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.599 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.599 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.600 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.600 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.600 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.600 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.600 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.600 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.600 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.600 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.600 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.600 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.600 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.600 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.600 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.600 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.600 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.601 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.601 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.601 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.601 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.601 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.601 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.601 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.601 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.601 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.601 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.601 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.601 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.601 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.601 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.601 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.601 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.602 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.602 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.602 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.602 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.602 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.602 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.602 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.602 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.602 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.602 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-03-16T09:18:27.602 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.602 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.602 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.602 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.602 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.603 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.603 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.603 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.603 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.603 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.603 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.603 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.603 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.603 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.603 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.603 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.603 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.603 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.603 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.604 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.604 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.604 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.604 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.604 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.604 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.604 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.604 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.604 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.604 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.604 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.604 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.604 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.604 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.605 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.605 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.605 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.605 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.605 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.605 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.605 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.605 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.605 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.605 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.605 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.605 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.605 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.606 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.606 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.606 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.606 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.606 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.606 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.606 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.606 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.606 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.606 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.606 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.606 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.606 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.606 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.607 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.607 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.607 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.607 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.607 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.607 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.607 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.607 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.607 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.607 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.607 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.607 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.607 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-03-16T09:18:27.607 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.608 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.608 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.608 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-03-16T09:18:27.608 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-03-16T09:18:27.608 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.608 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.608 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.608 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-03-16T09:18:27.608 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.608 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.608 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.608 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-16T09:18:27.608 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.608 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.608 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.609 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-03-16T09:18:27.609 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-03-16T09:18:27.609 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.609 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.609 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.609 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:27.609 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-03-16T09:18:27.609 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.609 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.609 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-16T09:18:27.609 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.609 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.609 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.610 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.610 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.610 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.610 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.610 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.610 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.610 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.610 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.610 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.610 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.610 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.610 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.610 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.610 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.610 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.611 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.611 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.611 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.611 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.611 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.611 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.611 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.611 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.611 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.611 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.611 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.611 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.611 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.612 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.612 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.612 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.612 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.612 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.612 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.612 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.612 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.612 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.612 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.612 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.612 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.612 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.612 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.613 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.613 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.613 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.613 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.613 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.613 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.613 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.613 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.613 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.613 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.613 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.613 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.613 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.613 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.614 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.614 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.614 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.614 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.614 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.614 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.614 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.614 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.614 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.614 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.614 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.614 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.614 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.615 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.615 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-03-16T09:18:27.615 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.615 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.615 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.615 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-03-16T09:18:27.615 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-03-16T09:18:27.615 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.615 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.615 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.615 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-03-16T09:18:27.615 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.615 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.615 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.616 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-03-16T09:18:27.616 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-16T09:18:27.616 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.616 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.616 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.616 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.616 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.616 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.616 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.616 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.616 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.616 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.616 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.617 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.619 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.620 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-03-16T09:18:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-03-16T09:18:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-03-16T09:18:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-16T09:18:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.621 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-03-16T09:18:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-03-16T09:18:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-03-16T09:18:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-03-16T09:18:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.622 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-03-16T09:18:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-03-16T09:18:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-03-16T09:18:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-03-16T09:18:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.623 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-03-16T09:18:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-03-16T09:18:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-03-16T09:18:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.624 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-03-16T09:18:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-03-16T09:18:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-03-16T09:18:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-16T09:18:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-16T09:18:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.625 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-03-16T09:18:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.626 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.628 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.629 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.630 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.631 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.631 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.631 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.631 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.631 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.631 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.631 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.631 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-03-16T09:18:27.631 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.631 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.631 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.631 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.631 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.631 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.632 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.632 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.632 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.632 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.632 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.632 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.632 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.632 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.632 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.632 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.632 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.632 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.632 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.632 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.632 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.632 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.632 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.633 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.633 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.633 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.633 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.633 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.633 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.633 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.633 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.633 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.633 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.633 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.633 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.633 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.634 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.634 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.634 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.634 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.634 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.634 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.634 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.634 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.634 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.634 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.634 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.634 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.634 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.634 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.634 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.635 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.635 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.635 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.635 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.635 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.635 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.635 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.635 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.635 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.635 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.635 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.635 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.635 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.635 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.635 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.636 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.636 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.636 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.636 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.636 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.636 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.636 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.636 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-03-16T09:18:27.636 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.636 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.636 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.636 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.636 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.636 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.636 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.637 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.637 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.637 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.637 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.637 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.637 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.637 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.637 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.637 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.637 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.637 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.637 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.637 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.637 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.637 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.638 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.638 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.638 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.638 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.638 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.638 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.638 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.638 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.638 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.638 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.638 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.638 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.638 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.639 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.639 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.639 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.639 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.639 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.639 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.639 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.639 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.639 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.639 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.639 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.639 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.639 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.639 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.639 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.640 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.640 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.640 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.640 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.640 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.640 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.640 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.640 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.640 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.640 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.640 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.640 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.640 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.640 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.640 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.641 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.641 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.641 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.641 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.641 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.641 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.641 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.641 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.641 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-03-16T09:18:27.641 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.641 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.641 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.641 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-03-16T09:18:27.641 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-03-16T09:18:27.641 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.642 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.642 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.642 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.642 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.642 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.642 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.642 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-16T09:18:27.642 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.642 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.642 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.642 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.642 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.642 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.642 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.643 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.643 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.643 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.643 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.643 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.643 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.643 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.643 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.643 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.643 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.643 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.643 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.643 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.643 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.643 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.643 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.644 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.644 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.644 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.644 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.644 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.644 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.644 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.644 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.644 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.644 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.644 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.644 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.644 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.645 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.645 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.645 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.645 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.645 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.645 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.645 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.645 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.645 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.645 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.645 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.645 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.645 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.645 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.645 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.645 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.646 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.646 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.646 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.646 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.646 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.646 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.646 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.646 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.646 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.646 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.646 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.646 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.646 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.646 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.646 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.647 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.647 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.647 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.647 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.647 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-03-16T09:18:27.647 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-03-16T09:18:27.647 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.647 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.647 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.647 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.647 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.647 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.647 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.647 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-03-16T09:18:27.647 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-03-16T09:18:27.647 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-03-16T09:18:27.648 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.648 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-03-16T09:18:27.648 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.648 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.648 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.648 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.648 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.648 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.648 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.648 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-03-16T09:18:27.648 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.648 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.648 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.648 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.648 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.648 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.649 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.649 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.649 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.649 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.649 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.649 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.649 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.649 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.649 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.649 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.649 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.649 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.649 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.649 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.649 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.650 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.650 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.650 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.650 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.650 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.650 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.650 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.650 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.650 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.650 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.650 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.650 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.650 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.650 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.651 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.651 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.651 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.651 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.651 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.651 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.651 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.651 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.651 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.651 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.651 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.651 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.651 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.651 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.651 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.651 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.652 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.652 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.652 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.652 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.652 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.652 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.652 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.652 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.652 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.652 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.652 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.652 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.652 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.652 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.652 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.652 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.653 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.653 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.653 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.653 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.653 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.653 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.653 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.653 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-03-16T09:18:27.653 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.653 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.653 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.653 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.653 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.654 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.654 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.654 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.654 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.654 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.654 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.654 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.654 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.654 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.654 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.654 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.654 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.654 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.654 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.655 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.655 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.655 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.655 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.655 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.655 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.655 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.655 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.655 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.655 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.655 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.655 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.655 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.655 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.655 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.656 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.656 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.656 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.656 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.656 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.656 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.656 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.656 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.656 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.656 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.656 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.656 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.656 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.656 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.656 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.657 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.657 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.657 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.657 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.657 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.657 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.657 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.657 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.657 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.657 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.657 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.657 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.657 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.657 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.658 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.658 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.658 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.658 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.658 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.658 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.658 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.658 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.658 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.658 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.658 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.658 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-03-16T09:18:27.658 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.658 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.659 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.659 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.659 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.659 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.659 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.659 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.659 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.659 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.659 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.659 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.659 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.659 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.659 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.659 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.659 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.660 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.660 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.660 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.660 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.660 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.660 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.660 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.660 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.660 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.660 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.660 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.660 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.660 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.660 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.660 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.661 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.661 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.661 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.661 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.661 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.661 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.661 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.661 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.661 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.661 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.661 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.661 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.662 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.662 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.662 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.662 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.662 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.662 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.662 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.662 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.662 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.662 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.662 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.662 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.662 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.663 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.663 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.663 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.663 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.663 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.663 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.663 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.663 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.663 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.663 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.663 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.663 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.663 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.664 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-03-16T09:18:27.664 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-03-16T09:18:27.664 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.664 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.664 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.664 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-03-16T09:18:27.664 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.664 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.664 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.664 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.664 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.664 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.664 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.664 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.665 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-03-16T09:18:27.665 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-03-16T09:18:27.665 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:27.665 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.665 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.665 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-03-16T09:18:27.665 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-03-16T09:18:27.665 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.665 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.665 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-03-16T09:18:27.665 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-16T09:18:27.665 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.665 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.665 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-03-16T09:18:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-03-16T09:18:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-03-16T09:18:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.666 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-16T09:18:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-03-16T09:18:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-03-16T09:18:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.667 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.668 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.669 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.670 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.671 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.672 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-03-16T09:18:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-03-16T09:18:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-03-16T09:18:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-03-16T09:18:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-03-16T09:18:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-03-16T09:18:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-03-16T09:18:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-03-16T09:18:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.674 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-03-16T09:18:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-03-16T09:18:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-03-16T09:18:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.675 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-03-16T09:18:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-03-16T09:18:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.676 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-16T09:18:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-16T09:18:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.677 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.678 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.678 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.678 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.678 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.678 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.678 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.678 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.678 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.678 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.678 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.678 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.678 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.679 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.679 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.679 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.679 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.679 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.679 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.679 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.679 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.679 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.679 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.679 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.679 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.679 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.679 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.680 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.680 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.680 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.680 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.680 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.680 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.680 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.680 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.680 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.680 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.680 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.680 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.680 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.680 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.680 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.681 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.681 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.681 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.681 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.681 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.681 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.681 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.681 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.681 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.681 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.681 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.681 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.681 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.681 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.681 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.682 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.682 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.682 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-03-16T09:18:27.682 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-03-16T09:18:27.682 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.682 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.682 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.682 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.682 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.682 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.682 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.682 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.682 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.682 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.683 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.683 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.683 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-16T09:18:27.683 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-16T09:18:27.683 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.683 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.683 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.683 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-03-16T09:18:27.683 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.683 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.683 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.683 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.683 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.683 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.684 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.684 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-03-16T09:18:27.684 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-03-16T09:18:27.684 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:27.684 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.684 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.684 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-03-16T09:18:27.684 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.684 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.684 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.684 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.684 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.684 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.684 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.685 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-03-16T09:18:27.685 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-03-16T09:18:27.685 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.685 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.685 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.685 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-03-16T09:18:27.685 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.685 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.685 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.685 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-16T09:18:27.685 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.685 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.685 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.685 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.685 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-03-16T09:18:27.686 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-03-16T09:18:27.686 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.686 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.686 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.686 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-03-16T09:18:27.686 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.686 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.686 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.686 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-16T09:18:27.686 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.686 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.686 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.687 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-03-16T09:18:27.687 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-03-16T09:18:27.687 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.687 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.687 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.687 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-03-16T09:18:27.687 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.687 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.687 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.687 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.687 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.687 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.687 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.688 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-03-16T09:18:27.688 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-03-16T09:18:27.688 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.688 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.688 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.688 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-03-16T09:18:27.688 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.688 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.688 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.688 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-03-16T09:18:27.688 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-03-16T09:18:27.688 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.688 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.688 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.688 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.689 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.689 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-03-16T09:18:27.689 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-03-16T09:18:27.689 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.689 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.689 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.689 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-03-16T09:18:27.689 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-03-16T09:18:27.689 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-03-16T09:18:27.689 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.689 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.689 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.689 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.690 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.690 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.690 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.690 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.690 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.690 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-03-16T09:18:27.690 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.690 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.690 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.690 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.690 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.690 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.690 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.690 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.691 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.691 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.691 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.691 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.691 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.691 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.691 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.691 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.691 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.691 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.691 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.691 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.691 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.691 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.691 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.692 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.692 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.692 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.692 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.692 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.692 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.692 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.692 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.692 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.692 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.692 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.692 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.692 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.693 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.693 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.693 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.693 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.693 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.693 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.693 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.693 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.693 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.693 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.693 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.693 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.693 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.693 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.693 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.693 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.694 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.694 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.694 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.694 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.694 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.694 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.694 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.694 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.694 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.694 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.694 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.694 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.695 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.695 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.695 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.695 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.695 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.695 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-03-16T09:18:27.695 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-03-16T09:18:27.695 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.695 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.695 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.695 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:27.695 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.695 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.695 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.695 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.696 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.696 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.696 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.696 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-03-16T09:18:27.696 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-03-16T09:18:27.696 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.696 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.696 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.696 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.696 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.696 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.696 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.696 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.696 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.696 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.697 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.697 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-03-16T09:18:27.697 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-03-16T09:18:27.697 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.697 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.697 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.697 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.697 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.697 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.697 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.697 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.697 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.697 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.697 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.698 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.698 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.698 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.698 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-03-16T09:18:27.698 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.698 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.698 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.698 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-03-16T09:18:27.698 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-03-16T09:18:27.698 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.698 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.698 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.698 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.698 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.699 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.699 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.699 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.699 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.699 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.699 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.699 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-03-16T09:18:27.699 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-03-16T09:18:27.699 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.699 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.699 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.699 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.699 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.699 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.700 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.700 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.700 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.700 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.700 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.700 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.700 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.700 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.700 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.701 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.701 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.701 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.701 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.701 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.701 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.701 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.701 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.701 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.701 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.701 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.701 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.701 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.701 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.702 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.702 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.702 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.702 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.702 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.702 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.702 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.702 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.702 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.702 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.702 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.702 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.703 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.703 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.703 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.703 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.703 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.703 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.703 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.703 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.703 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.703 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.703 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.703 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.703 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.703 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.703 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.704 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.704 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.704 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.704 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.704 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.704 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.704 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.704 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.704 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.704 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.704 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.704 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.704 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.704 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.705 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.705 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.705 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.705 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.705 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.705 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.705 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.705 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-03-16T09:18:27.705 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-03-16T09:18:27.705 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.705 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.705 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.705 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.706 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.706 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.706 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.706 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.706 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.706 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.706 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.706 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-03-16T09:18:27.706 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-03-16T09:18:27.706 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.706 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.706 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.706 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.707 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.707 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.707 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.707 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.707 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.707 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.707 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.707 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.707 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.707 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.707 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-03-16T09:18:27.707 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.707 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.707 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.708 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.708 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.708 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.708 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.708 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.708 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.708 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.708 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.708 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.708 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.708 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.708 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.708 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.708 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.708 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.708 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.709 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.709 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.709 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.709 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.709 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.709 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.709 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.709 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.709 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.709 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.709 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.709 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.709 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.710 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.710 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.710 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.710 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.710 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.710 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.710 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.710 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.710 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.710 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.710 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.710 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.711 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.711 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.711 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.711 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.711 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.711 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.711 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.711 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.711 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.711 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.711 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.711 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.711 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.711 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.711 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.712 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.712 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.712 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.712 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.712 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.712 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.712 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.712 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.712 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.712 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.712 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.712 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.712 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.712 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-03-16T09:18:27.713 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.713 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.713 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.713 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.713 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.713 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.713 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.713 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.713 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.713 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.713 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.713 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.713 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.713 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.713 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.713 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.714 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.714 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.714 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.714 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.714 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.714 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.714 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.714 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.714 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.714 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.714 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.714 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.715 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.715 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.715 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.715 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.715 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.715 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.715 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.715 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.715 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.715 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.715 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.715 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.715 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.715 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.716 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.716 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.716 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.716 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.716 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.716 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.716 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.716 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.716 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.716 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.716 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.716 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.716 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.716 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.716 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.717 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.717 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.717 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.717 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.717 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.717 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.717 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.717 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.717 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.717 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.717 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.717 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.717 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-03-16T09:18:27.717 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-03-16T09:18:27.718 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.718 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.718 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.718 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-03-16T09:18:27.718 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.718 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.718 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.718 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-16T09:18:27.718 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.718 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.718 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.718 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.718 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.719 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.719 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-03-16T09:18:27.719 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.719 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.719 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.719 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.719 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.719 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.719 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.719 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.719 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.719 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.719 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.719 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.719 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.719 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.720 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.720 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.720 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.720 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.720 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.720 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.720 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.720 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.720 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.720 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.720 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.720 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.720 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.720 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.721 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.721 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.721 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.721 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.721 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.721 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.721 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.721 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.721 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.721 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.721 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.721 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.721 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.721 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.721 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.722 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.722 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.722 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.722 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.722 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.722 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.722 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.722 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.722 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.722 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.722 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.722 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.722 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.723 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.723 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.723 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.723 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.723 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.723 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.723 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.723 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.723 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.723 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.723 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.723 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.723 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.724 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-03-16T09:18:27.724 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-03-16T09:18:27.724 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.724 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.724 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.724 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.724 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.724 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.724 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.724 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.724 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.724 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.724 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.724 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-03-16T09:18:27.724 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-03-16T09:18:27.725 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.725 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.725 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.725 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-03-16T09:18:27.725 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.725 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.725 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.725 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.725 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.725 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.725 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.725 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-03-16T09:18:27.725 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-03-16T09:18:27.725 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.726 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.726 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.726 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.726 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.726 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.726 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.726 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.726 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.726 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.726 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.726 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-03-16T09:18:27.726 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-03-16T09:18:27.727 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.727 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.727 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.727 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.727 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.727 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.727 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.727 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.727 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.727 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.727 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.727 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-03-16T09:18:27.727 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-03-16T09:18:27.728 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:27.728 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.728 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.728 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.728 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.728 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.728 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.728 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.728 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.728 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.728 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.728 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-03-16T09:18:27.729 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-03-16T09:18:27.729 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.729 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.729 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.729 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.729 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.729 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.729 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.729 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.729 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.729 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.729 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.729 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-03-16T09:18:27.730 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-03-16T09:18:27.730 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.730 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.730 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.730 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.730 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.730 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.730 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.730 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.730 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.730 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.730 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.731 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-03-16T09:18:27.731 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-03-16T09:18:27.731 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.731 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.731 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.731 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.731 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.731 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.731 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.731 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.731 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.731 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.731 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.731 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-03-16T09:18:27.732 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-03-16T09:18:27.732 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.732 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.732 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.732 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.732 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-03-16T09:18:27.732 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-03-16T09:18:27.732 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.732 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.732 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.732 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.732 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.732 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-03-16T09:18:27.733 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-03-16T09:18:27.733 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.733 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.733 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.733 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.733 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.733 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.733 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.733 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.733 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.733 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.733 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.733 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-03-16T09:18:27.734 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-03-16T09:18:27.734 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.734 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.734 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.734 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.734 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.734 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.734 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.734 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.734 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.734 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.734 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.735 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-03-16T09:18:27.735 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-03-16T09:18:27.735 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.735 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.735 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.735 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.735 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.735 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.735 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.735 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.735 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.735 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.735 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.735 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.736 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.736 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.736 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-03-16T09:18:27.736 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.736 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.736 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.736 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-03-16T09:18:27.736 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-03-16T09:18:27.736 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.736 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.736 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.736 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.736 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.736 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.737 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.737 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-03-16T09:18:27.737 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.737 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.737 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.737 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-03-16T09:18:27.737 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-03-16T09:18:27.737 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.737 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.737 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.737 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.737 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.737 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.738 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.738 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-03-16T09:18:27.738 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.738 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.738 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.738 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.738 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.738 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.738 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.738 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.738 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.738 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.738 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.739 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.739 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.739 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.739 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.739 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.739 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.739 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.739 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.739 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.739 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.739 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.739 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.739 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.739 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.739 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.740 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.740 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.740 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.740 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.740 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.740 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.740 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.740 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.740 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.740 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.740 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.740 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.740 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.740 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.740 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.740 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.741 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.741 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.741 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.741 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.741 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.741 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.741 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.741 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.741 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.741 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.741 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.741 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.741 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.741 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.741 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.742 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.742 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.742 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.742 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.742 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.742 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.742 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.742 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.742 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.742 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.742 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.742 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.742 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.743 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.743 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.743 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.743 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-03-16T09:18:27.743 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.743 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.743 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.743 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.743 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.743 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.743 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.743 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.743 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.743 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.743 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.744 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.744 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.744 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.744 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.744 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.744 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.744 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.744 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.744 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.744 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.744 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.744 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.744 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.744 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.745 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.745 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.745 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.745 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.745 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.745 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.745 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.745 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.745 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.745 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.745 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.745 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.745 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.745 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.745 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.746 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.746 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.746 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.746 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.746 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.746 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.746 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.746 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.746 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.746 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.746 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.746 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.746 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.746 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.746 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.747 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.747 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.747 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.747 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.747 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.747 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.747 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.747 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.747 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.747 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.747 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.747 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.747 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.747 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.747 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.748 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.748 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.748 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.748 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-03-16T09:18:27.748 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.748 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.748 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.748 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.748 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.748 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.748 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.748 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.748 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.748 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.748 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.749 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.749 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.749 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.749 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.749 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.749 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.749 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.749 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.749 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.749 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.749 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.749 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.749 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.749 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.749 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.749 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.750 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.750 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.750 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.750 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.750 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.750 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.750 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.750 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.750 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.750 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.750 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.750 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.750 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.751 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.751 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.751 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.751 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.751 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.751 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.751 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.751 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.751 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.751 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.751 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.751 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.751 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.751 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.751 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.752 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.752 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.752 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.752 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.752 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.752 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.752 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.752 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.752 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.752 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.752 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.752 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.752 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.752 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.752 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.752 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.753 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.753 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.753 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.753 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.753 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-03-16T09:18:27.753 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.753 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.753 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.753 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-03-16T09:18:27.753 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-03-16T09:18:27.753 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.753 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.753 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.753 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-16T09:18:27.753 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.754 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.754 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.754 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.754 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.754 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.754 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.754 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.754 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-03-16T09:18:27.754 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-03-16T09:18:27.754 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.754 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.754 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.755 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-03-16T09:18:27.755 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.755 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.755 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.755 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.755 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.755 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.755 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.755 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.755 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.755 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.755 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.755 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.755 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.756 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.756 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.756 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.756 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.756 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.756 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.756 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.756 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.756 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.756 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.756 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.756 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.756 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.756 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.756 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.757 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.757 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.757 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.757 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.757 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.757 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.757 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.757 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.757 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.757 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.757 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.757 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.757 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.757 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.758 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.758 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.758 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.758 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.758 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.758 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.758 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.758 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.758 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.758 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.758 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.758 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.758 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.758 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.759 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.759 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.759 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.759 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.759 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.759 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.759 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.759 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.759 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.759 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.759 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.759 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.759 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.759 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.760 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.760 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.760 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.760 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.760 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.760 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.760 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.760 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.760 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.760 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.760 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-03-16T09:18:27.760 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.760 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.760 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.760 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-03-16T09:18:27.761 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-03-16T09:18:27.761 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.761 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.761 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.761 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.761 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.761 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.761 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.761 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-16T09:18:27.761 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.761 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.761 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.761 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-03-16T09:18:27.761 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-03-16T09:18:27.762 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.762 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.762 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.762 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.762 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.762 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.762 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.762 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-16T09:18:27.762 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.762 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.762 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.762 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.762 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-03-16T09:18:27.763 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-03-16T09:18:27.763 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.763 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.763 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.763 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.763 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.763 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.763 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.763 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-16T09:18:27.763 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.763 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.763 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.763 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-03-16T09:18:27.763 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-03-16T09:18:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-16T09:18:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-03-16T09:18:27.764 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-03-16T09:18:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-16T09:18:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-03-16T09:18:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-03-16T09:18:27.765 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-03-16T09:18:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-03-16T09:18:27.766 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-03-16T09:18:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-03-16T09:18:27.767 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-16T09:18:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-03-16T09:18:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-03-16T09:18:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.768 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-03-16T09:18:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-03-16T09:18:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.769 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-03-16T09:18:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-03-16T09:18:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-03-16T09:18:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-03-16T09:18:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.770 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-03-16T09:18:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-03-16T09:18:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-03-16T09:18:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.771 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-03-16T09:18:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-03-16T09:18:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.772 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.773 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.774 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.775 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.776 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.777 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.777 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.777 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.777 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.777 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.777 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.777 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.777 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.777 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.777 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.777 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.777 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.777 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.777 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.778 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.778 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.778 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.778 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-03-16T09:18:27.778 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-03-16T09:18:27.778 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.778 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.778 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.778 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.778 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.778 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.778 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.778 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.779 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.779 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.779 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.779 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-03-16T09:18:27.779 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-03-16T09:18:27.779 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.779 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.779 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.779 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.779 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.779 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.779 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.779 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.779 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.780 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.780 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.780 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-03-16T09:18:27.780 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-03-16T09:18:27.780 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.780 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.780 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.780 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-16T09:18:27.780 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.780 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.780 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.780 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.780 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.781 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.781 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.781 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.781 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.781 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-03-16T09:18:27.781 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.781 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.781 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.781 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.781 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.781 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.781 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.781 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.782 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.782 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.782 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.782 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.782 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.782 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.782 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.782 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.782 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.782 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.782 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.782 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.782 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.782 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.783 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.783 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.783 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.783 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.783 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.783 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.783 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.783 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.783 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.783 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.783 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.783 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.783 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.784 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.784 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.784 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.784 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.784 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.784 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.784 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.784 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.784 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.784 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.784 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.784 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.784 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.784 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.784 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.785 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.785 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.785 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.785 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.785 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.785 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.785 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.785 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.785 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.785 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.785 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.785 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.785 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.785 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.786 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.786 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.786 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.786 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.786 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.786 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.786 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.786 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.786 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.786 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-03-16T09:18:27.786 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.786 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.787 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.787 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.787 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.787 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.787 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.787 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.787 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.787 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.787 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.787 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.787 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.787 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.787 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.787 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.787 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.788 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.788 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.788 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.788 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.788 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.788 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.788 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.788 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.788 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.788 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.788 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.788 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.788 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.788 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.788 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.789 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.789 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.789 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.789 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.789 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.789 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.789 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.789 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.789 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.789 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.789 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.789 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.789 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.789 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.789 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.790 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.790 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.790 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.790 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.790 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.790 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.790 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.790 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.790 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.790 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.790 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.790 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.790 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.791 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.791 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.791 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.791 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.791 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.791 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.791 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.791 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.791 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.791 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-03-16T09:18:27.791 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-03-16T09:18:27.791 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.791 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.791 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.791 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-03-16T09:18:27.792 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.792 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.792 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.792 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-16T09:18:27.792 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.792 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.792 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.792 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-03-16T09:18:27.792 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-03-16T09:18:27.792 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.792 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.792 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.792 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.792 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.792 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.793 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.793 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-03-16T09:18:27.793 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.793 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.793 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.793 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-03-16T09:18:27.793 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-03-16T09:18:27.793 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.793 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.793 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.793 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-03-16T09:18:27.793 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.794 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.794 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.794 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-16T09:18:27.794 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.794 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.794 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.794 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-03-16T09:18:27.794 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-03-16T09:18:27.794 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.794 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.794 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.794 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.794 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.795 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.795 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.795 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-16T09:18:27.795 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.795 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.795 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.795 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.795 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.795 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-03-16T09:18:27.795 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.795 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.795 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.795 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-03-16T09:18:27.796 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-03-16T09:18:27.796 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-03-16T09:18:27.796 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.796 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.796 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-03-16T09:18:27.796 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.796 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.796 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.796 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.796 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.796 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.796 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.796 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-03-16T09:18:27.796 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-03-16T09:18:27.796 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.797 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.797 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.797 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.797 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.797 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.797 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.797 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.797 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.797 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.797 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.797 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-03-16T09:18:27.797 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-03-16T09:18:27.797 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.797 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.797 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.798 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-03-16T09:18:27.798 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.798 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.798 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.798 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.798 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.798 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.798 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.798 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.798 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.798 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.798 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.798 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.799 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.799 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.799 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.799 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.799 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.799 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.799 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.799 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.799 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.799 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.799 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.799 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.799 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.799 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.799 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.800 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.800 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.800 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.800 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.800 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.800 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.800 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.800 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.800 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.800 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.800 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.800 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.800 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.800 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.801 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.801 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.801 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.801 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.801 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.801 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.801 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.801 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.801 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.801 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.801 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.801 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.801 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.801 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.801 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.801 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.802 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.802 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.802 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.802 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.802 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.802 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.802 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.802 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.802 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.802 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.802 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.802 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.802 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.802 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.803 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.803 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.803 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.803 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-03-16T09:18:27.803 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-03-16T09:18:27.803 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.803 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.803 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.803 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.803 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.803 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.803 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.803 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.803 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-03-16T09:18:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-03-16T09:18:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-03-16T09:18:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.804 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-03-16T09:18:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-03-16T09:18:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-16T09:18:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-03-16T09:18:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "gid": 24113, 2024-03-16T09:18:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "y", 2024-03-16T09:18:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr_features": 4540701547738038271, 2024-03-16T09:18:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-03-16T09:18:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-03-16T09:18:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.806 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-03-16T09:18:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-03-16T09:18:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-03-16T09:18:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-03-16T09:18:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.807 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.808 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.809 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.810 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.811 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-03-16T09:18:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-03-16T09:18:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.812 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to", 2024-03-16T09:18:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-03-16T09:18:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-03-16T09:18:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-03-16T09:18:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-03-16T09:18:27.813 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-03-16T09:18:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-03-16T09:18:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-03-16T09:18:27.814 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-03-16T09:18:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-03-16T09:18:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-03-16T09:18:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.815 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-03-16T09:18:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-03-16T09:18:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-03-16T09:18:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-03-16T09:18:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.816 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.817 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-03-16T09:18:27.817 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-03-16T09:18:27.817 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.817 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.817 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.817 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.817 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.817 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.817 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.817 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-03-16T09:18:27.817 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.817 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.817 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.817 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-03-16T09:18:27.818 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-03-16T09:18:27.818 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.818 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.818 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.818 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.818 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.818 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.818 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.818 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-16T09:18:27.818 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.818 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.818 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.818 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-03-16T09:18:27.819 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-03-16T09:18:27.819 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.819 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.819 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.819 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.819 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.819 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.819 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.819 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-03-16T09:18:27.819 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.819 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.819 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.819 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.819 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.820 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-03-16T09:18:27.820 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.820 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.820 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.820 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-03-16T09:18:27.820 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-03-16T09:18:27.820 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.820 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.820 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.820 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.820 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.820 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.820 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.821 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-03-16T09:18:27.821 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.821 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.821 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.821 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-03-16T09:18:27.821 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-03-16T09:18:27.821 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.821 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.821 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.821 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-03-16T09:18:27.821 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.821 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.821 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.822 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-03-16T09:18:27.822 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-16T09:18:27.822 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.822 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.822 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-03-16T09:18:27.822 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-03-16T09:18:27.822 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-03-16T09:18:27.822 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.822 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.822 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-03-16T09:18:27.822 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-03-16T09:18:27.822 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-03-16T09:18:27.822 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.822 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-16T09:18:27.823 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-16T09:18:27.823 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.823 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.823 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.823 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-03-16T09:18:27.823 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-03-16T09:18:27.823 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-03-16T09:18:27.823 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.823 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.823 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-03-16T09:18:27.823 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-03-16T09:18:27.823 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-03-16T09:18:27.823 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.823 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-16T09:18:27.824 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.824 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.824 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.824 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-03-16T09:18:27.824 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-03-16T09:18:27.824 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.824 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.824 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.824 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-03-16T09:18:27.824 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.824 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.824 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.824 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-16T09:18:27.824 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-16T09:18:27.825 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.825 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.825 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-03-16T09:18:27.825 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-03-16T09:18:27.825 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:27.825 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.825 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.825 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-03-16T09:18:27.825 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-03-16T09:18:27.825 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-03-16T09:18:27.825 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.825 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-03-16T09:18:27.825 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-16T09:18:27.826 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.826 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.826 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.826 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-03-16T09:18:27.826 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-03-16T09:18:27.826 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.826 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.826 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.826 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-03-16T09:18:27.826 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.826 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.826 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.826 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-03-16T09:18:27.826 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-16T09:18:27.826 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.827 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.827 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-03-16T09:18:27.827 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-03-16T09:18:27.827 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-03-16T09:18:27.827 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.827 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.827 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-03-16T09:18:27.827 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-03-16T09:18:27.827 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-03-16T09:18:27.827 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.827 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-16T09:18:27.827 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-16T09:18:27.827 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.827 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.828 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.828 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.828 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.828 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.828 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.828 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.828 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.828 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.828 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.828 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.828 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.828 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.828 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.829 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.829 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.829 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.829 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.829 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.829 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.829 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.829 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.829 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.829 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.829 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.829 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.829 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.830 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.830 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.830 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.830 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.830 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.830 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.830 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.830 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.830 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.830 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.830 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.830 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.830 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.830 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.831 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.831 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.831 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.831 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.831 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.831 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.831 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.831 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.831 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.831 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.831 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.831 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.831 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.832 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.832 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.832 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.832 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.832 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.832 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.832 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.832 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.832 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.832 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.832 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.832 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.832 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.833 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-03-16T09:18:27.833 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-03-16T09:18:27.833 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:27.833 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.833 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.833 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-03-16T09:18:27.833 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.833 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.833 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.833 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-16T09:18:27.833 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.833 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.833 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.833 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-03-16T09:18:27.834 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-03-16T09:18:27.834 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.834 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.834 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.834 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-03-16T09:18:27.834 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.834 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.834 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.834 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-03-16T09:18:27.834 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-03-16T09:18:27.834 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-03-16T09:18:27.834 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-03-16T09:18:27.834 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-03-16T09:18:27.835 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.835 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-03-16T09:18:27.835 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.835 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.835 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.835 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-03-16T09:18:27.835 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-03-16T09:18:27.835 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.835 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.835 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.835 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.835 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.835 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.836 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.836 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-16T09:18:27.836 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.836 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.836 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.836 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-03-16T09:18:27.836 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-03-16T09:18:27.836 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.836 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.836 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.836 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-03-16T09:18:27.836 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.836 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.837 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.837 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-16T09:18:27.837 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.837 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.837 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.837 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-03-16T09:18:27.837 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-03-16T09:18:27.837 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.837 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.837 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.837 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-03-16T09:18:27.837 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-03-16T09:18:27.838 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.838 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.838 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-16T09:18:27.838 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-16T09:18:27.838 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.838 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.838 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-03-16T09:18:27.838 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-03-16T09:18:27.838 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-03-16T09:18:27.838 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.838 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.838 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:27.838 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.839 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.839 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.839 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-16T09:18:27.839 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.839 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.839 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.839 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.839 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.839 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.839 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.839 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-03-16T09:18:27.839 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.839 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.840 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.840 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-03-16T09:18:27.840 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-03-16T09:18:27.840 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:27.840 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.840 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.840 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-03-16T09:18:27.840 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.840 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.840 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.840 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-16T09:18:27.840 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.840 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.840 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.841 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-03-16T09:18:27.841 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-03-16T09:18:27.841 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.841 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.841 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.841 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-03-16T09:18:27.841 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.841 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.841 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.841 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-16T09:18:27.841 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.841 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.841 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.842 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-03-16T09:18:27.842 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-03-16T09:18:27.842 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.842 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.842 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.842 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-03-16T09:18:27.842 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.842 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.842 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.842 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-16T09:18:27.842 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.842 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.842 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.843 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-03-16T09:18:27.843 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-03-16T09:18:27.843 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.843 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.843 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.843 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.843 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.843 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.843 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.843 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-16T09:18:27.843 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-16T09:18:27.843 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.844 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.844 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.844 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-03-16T09:18:27.844 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-03-16T09:18:27.844 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.844 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.844 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.844 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-03-16T09:18:27.844 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.844 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.844 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.844 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-16T09:18:27.844 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.844 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.845 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.845 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-03-16T09:18:27.845 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-03-16T09:18:27.845 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:27.845 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.845 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.845 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-03-16T09:18:27.845 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.845 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.845 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.845 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-16T09:18:27.845 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.845 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.845 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.846 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-03-16T09:18:27.846 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-03-16T09:18:27.846 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.846 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.846 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.846 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.846 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.846 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.846 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.846 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-03-16T09:18:27.846 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-03-16T09:18:27.846 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-03-16T09:18:27.846 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.847 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-16T09:18:27.847 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.847 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.847 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.847 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.847 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-03-16T09:18:27.847 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-03-16T09:18:27.847 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.847 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.847 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.847 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.847 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.847 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.849 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.849 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-16T09:18:27.849 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.849 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.849 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.849 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-03-16T09:18:27.849 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-03-16T09:18:27.849 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.849 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.849 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.849 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.850 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.850 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.850 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.850 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-16T09:18:27.850 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.850 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.850 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.850 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.850 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-03-16T09:18:27.850 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-03-16T09:18:27.850 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.850 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.850 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.850 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.851 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.851 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.851 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.851 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-16T09:18:27.851 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.851 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.851 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.851 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-03-16T09:18:27.851 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-03-16T09:18:27.851 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.851 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.851 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.851 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-16T09:18:27.851 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.852 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.852 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.852 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-03-16T09:18:27.852 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.852 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.852 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.852 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-03-16T09:18:27.852 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-03-16T09:18:27.852 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.852 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.852 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.852 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-16T09:18:27.852 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.852 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.853 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.853 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-03-16T09:18:27.853 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.853 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.853 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.853 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.853 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-03-16T09:18:27.853 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-03-16T09:18:27.853 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.853 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.853 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.853 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-16T09:18:27.853 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.853 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.853 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.854 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-03-16T09:18:27.854 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.854 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.854 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.854 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-03-16T09:18:27.854 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-03-16T09:18:27.854 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.854 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.854 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.854 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-16T09:18:27.854 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.854 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.854 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.855 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-03-16T09:18:27.855 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.855 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.855 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.855 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-03-16T09:18:27.855 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-03-16T09:18:27.855 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.855 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.855 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.855 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-16T09:18:27.855 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.855 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.855 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.855 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-03-16T09:18:27.855 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.856 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.856 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.856 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-03-16T09:18:27.856 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-03-16T09:18:27.856 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.856 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.856 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.856 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-16T09:18:27.856 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.856 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.856 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.856 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-03-16T09:18:27.856 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.857 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.857 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.857 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-03-16T09:18:27.857 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-03-16T09:18:27.857 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.857 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.857 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.857 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-16T09:18:27.857 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.857 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.857 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.857 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-03-16T09:18:27.857 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.857 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.858 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.858 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-03-16T09:18:27.858 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-03-16T09:18:27.858 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.858 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.858 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.858 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-16T09:18:27.858 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.858 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.858 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.858 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-03-16T09:18:27.858 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.858 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.858 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.859 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-03-16T09:18:27.859 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-03-16T09:18:27.859 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.859 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.859 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.859 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-16T09:18:27.859 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.859 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.859 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.859 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-03-16T09:18:27.859 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.859 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.859 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.859 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-03-16T09:18:27.860 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-03-16T09:18:27.860 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.860 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.860 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.860 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-16T09:18:27.860 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.860 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.860 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.860 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-03-16T09:18:27.860 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.860 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.860 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.860 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-03-16T09:18:27.860 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-03-16T09:18:27.861 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.861 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.861 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.861 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-16T09:18:27.861 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.861 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.861 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.861 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-03-16T09:18:27.861 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.861 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.861 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.861 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-03-16T09:18:27.861 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-03-16T09:18:27.861 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.861 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.862 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.862 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-16T09:18:27.862 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.862 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.862 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.862 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-03-16T09:18:27.862 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.862 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.862 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.862 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.862 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-03-16T09:18:27.862 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-03-16T09:18:27.862 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.862 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.863 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.863 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-16T09:18:27.863 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.863 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.863 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.863 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-03-16T09:18:27.863 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.863 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.863 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.863 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-03-16T09:18:27.863 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-03-16T09:18:27.863 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.863 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.863 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.864 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-16T09:18:27.864 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.864 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.864 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.864 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-03-16T09:18:27.864 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.864 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.864 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.864 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.864 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-03-16T09:18:27.864 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-03-16T09:18:27.864 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.864 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.864 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.864 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-16T09:18:27.864 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.865 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.865 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.865 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-03-16T09:18:27.865 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.865 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.865 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.865 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-03-16T09:18:27.865 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-03-16T09:18:27.865 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.865 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.865 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.865 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.865 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.865 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.866 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.866 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-03-16T09:18:27.866 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.866 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.866 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.866 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-03-16T09:18:27.866 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-03-16T09:18:27.866 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.866 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.866 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.866 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-03-16T09:18:27.866 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.866 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.867 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.867 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-16T09:18:27.867 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.867 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.867 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.867 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-03-16T09:18:27.867 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-03-16T09:18:27.867 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.867 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.867 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.867 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-03-16T09:18:27.867 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.867 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.867 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.867 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-16T09:18:27.867 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.868 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.868 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.868 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-03-16T09:18:27.868 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-03-16T09:18:27.868 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.868 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.868 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.868 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-03-16T09:18:27.868 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.868 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.868 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.868 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-16T09:18:27.868 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.868 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.869 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.869 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-03-16T09:18:27.869 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-03-16T09:18:27.869 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.869 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.869 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.869 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-03-16T09:18:27.869 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.869 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.869 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.869 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-03-16T09:18:27.869 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.869 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.869 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.870 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-03-16T09:18:27.870 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-03-16T09:18:27.870 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.870 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.870 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.870 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.870 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.870 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.870 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.870 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-16T09:18:27.870 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.870 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.870 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.870 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-03-16T09:18:27.871 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-03-16T09:18:27.871 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.871 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.871 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.871 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-03-16T09:18:27.871 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.871 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.871 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.871 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-03-16T09:18:27.871 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.871 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.871 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.871 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-03-16T09:18:27.871 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-03-16T09:18:27.872 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.872 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.872 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.872 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-03-16T09:18:27.872 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.872 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.872 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.872 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-03-16T09:18:27.872 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.872 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.872 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.872 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-03-16T09:18:27.872 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-03-16T09:18:27.873 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.873 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.873 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.873 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.873 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.873 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.873 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.873 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-16T09:18:27.873 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.873 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.873 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.873 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-03-16T09:18:27.873 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-03-16T09:18:27.873 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.874 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.874 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.874 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.874 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.874 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.874 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.874 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-16T09:18:27.874 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.874 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.874 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.874 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.874 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.875 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.875 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.875 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.875 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.875 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.875 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.875 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.875 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.875 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.875 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.875 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.875 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.875 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.875 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.876 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.876 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.876 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.876 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.876 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-03-16T09:18:27.876 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-03-16T09:18:27.876 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.876 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.876 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.876 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.876 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.876 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.876 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.876 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-16T09:18:27.877 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.877 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.877 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.877 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.877 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.877 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.877 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.877 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.877 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.877 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.877 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.877 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.877 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-16T09:18:27.877 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.878 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.878 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.878 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.878 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.878 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.878 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.878 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.878 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.878 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.878 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.878 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.878 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.878 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.879 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.879 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.879 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.879 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.879 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.879 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.879 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.879 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.879 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.879 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.879 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.879 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.879 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.879 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.880 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.880 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.880 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.880 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.880 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.880 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.880 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.880 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.880 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-16T09:18:27.880 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-16T09:18:27.880 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.880 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.880 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.880 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.881 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.881 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.881 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.881 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-16T09:18:27.881 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.881 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.881 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.881 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-16T09:18:27.881 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-16T09:18:27.881 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.881 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.881 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.881 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-03-16T09:18:27.882 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.882 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.882 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.882 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-16T09:18:27.882 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.882 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.882 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.882 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-03-16T09:18:27.882 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-03-16T09:18:27.882 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.882 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.882 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.883 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:27.883 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.883 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.883 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.883 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-03-16T09:18:27.883 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.883 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.883 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.883 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-03-16T09:18:27.883 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-03-16T09:18:27.883 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.883 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.883 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.883 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:27.884 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.884 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.884 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.884 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-16T09:18:27.884 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.884 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.884 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.884 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-03-16T09:18:27.884 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-03-16T09:18:27.884 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.884 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.884 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.884 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.885 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.885 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.885 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.885 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-03-16T09:18:27.885 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.885 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.885 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.885 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.885 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-03-16T09:18:27.885 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-03-16T09:18:27.885 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.885 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.885 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.885 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-03-16T09:18:27.885 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.886 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.886 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.886 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-03-16T09:18:27.886 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-03-16T09:18:27.886 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.886 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-03-16T09:18:27.886 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.886 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.886 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.886 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.886 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-03-16T09:18:27.886 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-03-16T09:18:27.886 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.886 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.887 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.887 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-03-16T09:18:27.887 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.887 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.887 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.887 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-16T09:18:27.887 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.887 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.887 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.887 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.887 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-03-16T09:18:27.887 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-03-16T09:18:27.887 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.887 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.888 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.888 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-16T09:18:27.888 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.888 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.888 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.888 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-16T09:18:27.888 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.888 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.888 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.888 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-03-16T09:18:27.888 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-03-16T09:18:27.888 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.888 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.888 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.889 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.889 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.889 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.889 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.889 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-16T09:18:27.889 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.889 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.889 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.889 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.889 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-03-16T09:18:27.889 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-03-16T09:18:27.889 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.889 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.889 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.890 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.890 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.890 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.890 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.890 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-16T09:18:27.890 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.890 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.890 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.890 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.890 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-03-16T09:18:27.890 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-03-16T09:18:27.890 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.890 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.890 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.891 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.891 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.891 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.891 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.891 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-16T09:18:27.891 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.891 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.891 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.891 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.891 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-03-16T09:18:27.891 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-03-16T09:18:27.891 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.891 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.891 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.892 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.892 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.892 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.892 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.892 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-16T09:18:27.892 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.892 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.892 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.892 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-03-16T09:18:27.892 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-03-16T09:18:27.892 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.892 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.892 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.892 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.893 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.893 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.893 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.893 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-16T09:18:27.893 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.893 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.893 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.893 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-03-16T09:18:27.893 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-03-16T09:18:27.893 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.893 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.893 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.894 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-03-16T09:18:27.894 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.894 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.894 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.894 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-03-16T09:18:27.894 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.894 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.894 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.894 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-03-16T09:18:27.894 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-03-16T09:18:27.894 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.894 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.895 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.895 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.895 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.895 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.895 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.895 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-16T09:18:27.895 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.895 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.895 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.895 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-03-16T09:18:27.895 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-03-16T09:18:27.895 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.895 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.896 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.896 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.896 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.896 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.896 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.896 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-16T09:18:27.896 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.896 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.896 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.896 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-03-16T09:18:27.896 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-03-16T09:18:27.896 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.896 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.897 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.897 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.897 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.897 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.897 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.897 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-16T09:18:27.897 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.897 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.897 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.897 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.897 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-03-16T09:18:27.897 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-03-16T09:18:27.897 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.897 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.898 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.898 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.898 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.898 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.898 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.898 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-03-16T09:18:27.898 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.898 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.898 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.898 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-03-16T09:18:27.898 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-03-16T09:18:27.898 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.898 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.899 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.899 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.899 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.899 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.899 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.899 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-16T09:18:27.899 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.899 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.899 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.899 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.899 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-03-16T09:18:27.899 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-03-16T09:18:27.899 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.899 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.899 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.900 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.900 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.900 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.900 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.900 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-16T09:18:27.900 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.900 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.900 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.900 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.900 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.900 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.900 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.900 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-03-16T09:18:27.900 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.900 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.901 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.901 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.901 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.901 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.901 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.901 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.901 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.901 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.901 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.901 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.901 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.901 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.901 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.901 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.902 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.902 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.902 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.902 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.902 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.902 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.902 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.902 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.902 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.902 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.902 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.902 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.902 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.902 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.902 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.903 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.903 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.903 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.903 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.903 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.903 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.903 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.903 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.903 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.903 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.903 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.903 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.903 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.903 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.903 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.904 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.904 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.904 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.904 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.904 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.904 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.904 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.904 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.904 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.904 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.904 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.904 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.905 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.905 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.905 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.905 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.905 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.905 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.905 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.905 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.905 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.905 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.905 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.905 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.905 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.906 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-03-16T09:18:27.906 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-03-16T09:18:27.906 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.906 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.906 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.906 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-03-16T09:18:27.906 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.906 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.906 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.906 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-16T09:18:27.906 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.906 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.906 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-03-16T09:18:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-03-16T09:18:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-03-16T09:18:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-16T09:18:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.907 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-03-16T09:18:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-16T09:18:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-16T09:18:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.908 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-03-16T09:18:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-16T09:18:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.909 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-16T09:18:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-16T09:18:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.910 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-03-16T09:18:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-03-16T09:18:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.911 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-16T09:18:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-16T09:18:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.912 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-03-16T09:18:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-16T09:18:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.913 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-16T09:18:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-16T09:18:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.914 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-16T09:18:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-16T09:18:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.915 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-16T09:18:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-16T09:18:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.916 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-16T09:18:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-16T09:18:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.917 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-16T09:18:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-16T09:18:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.918 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-16T09:18:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-16T09:18:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.920 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-16T09:18:27.920 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-16T09:18:27.920 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.920 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.920 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.920 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.920 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.920 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.920 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.920 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.920 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.920 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.920 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.921 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-16T09:18:27.921 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-16T09:18:27.921 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.921 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.921 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.921 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.921 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.921 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.921 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.921 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.921 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.921 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.921 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.922 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-03-16T09:18:27.922 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-16T09:18:27.922 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.922 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.922 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.922 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-03-16T09:18:27.922 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.922 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.922 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.922 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.922 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.922 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.922 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.922 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.923 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-16T09:18:27.923 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-16T09:18:27.923 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.923 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.923 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.923 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.923 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.923 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.923 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.923 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.923 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.923 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.923 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.924 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-03-16T09:18:27.924 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-03-16T09:18:27.924 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.924 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.924 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.924 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.924 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.924 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.924 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.924 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.924 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.924 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.924 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.925 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-03-16T09:18:27.925 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-16T09:18:27.925 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.925 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.925 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.925 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-03-16T09:18:27.925 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.925 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.925 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.925 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.925 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.925 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.925 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.925 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-16T09:18:27.926 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-16T09:18:27.926 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.926 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.926 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.926 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.926 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.926 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.926 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.926 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.926 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.926 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.926 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.926 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-16T09:18:27.927 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-16T09:18:27.927 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.927 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.927 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.927 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.927 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.927 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.927 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.927 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.927 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.927 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.927 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.927 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-16T09:18:27.928 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-16T09:18:27.928 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.928 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.928 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.928 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.928 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.928 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.928 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.928 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.928 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.928 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.928 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.928 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-16T09:18:27.928 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-16T09:18:27.929 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.929 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.929 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.929 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.929 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.929 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.929 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.929 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.929 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.929 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.929 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.929 INFO:teuthology.orchestra.run.smithi112.stdout: "MULTICLUSTER_CONFIG": { 2024-03-16T09:18:27.929 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-16T09:18:27.929 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.930 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.930 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.930 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "{}", 2024-03-16T09:18:27.930 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.930 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.930 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.930 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.930 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.930 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.930 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.930 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.930 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-03-16T09:18:27.931 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-16T09:18:27.931 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.931 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.931 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.931 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.931 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.931 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.931 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.931 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.931 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.931 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.931 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.931 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-16T09:18:27.931 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-16T09:18:27.932 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.932 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.932 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.932 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.932 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.932 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.932 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.932 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.932 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.932 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.932 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.932 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-16T09:18:27.933 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-16T09:18:27.933 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.933 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.933 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.933 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.933 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.933 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.933 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.933 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.933 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.933 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.933 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.933 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-16T09:18:27.934 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-16T09:18:27.934 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.934 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.934 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.934 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.934 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.934 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.934 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.934 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.934 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.934 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.934 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.935 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-16T09:18:27.935 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-16T09:18:27.935 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.935 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.935 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.935 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.935 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.935 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.935 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.935 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.935 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.935 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.935 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.935 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.935 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-16T09:18:27.936 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-16T09:18:27.936 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.936 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.936 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.936 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.936 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.936 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.936 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.936 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.936 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.936 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.936 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.936 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-16T09:18:27.937 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-16T09:18:27.937 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.937 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.937 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.937 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.937 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.937 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.937 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.937 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.937 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.937 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.937 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.937 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.937 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-16T09:18:27.937 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-16T09:18:27.937 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.938 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.938 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.938 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.938 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.938 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.938 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.938 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.938 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.938 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.938 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.938 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.938 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-16T09:18:27.938 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-16T09:18:27.938 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.939 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.939 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.939 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.939 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.939 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.939 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.939 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.939 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.939 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.939 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.939 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.939 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-03-16T09:18:27.939 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-16T09:18:27.939 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.939 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.940 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.940 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.940 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.940 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.940 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.940 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.940 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.940 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.940 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.940 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-16T09:18:27.940 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-16T09:18:27.940 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.940 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.940 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.941 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-16T09:18:27.941 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.941 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.941 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.941 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.941 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.941 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.941 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.941 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.941 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-16T09:18:27.941 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-16T09:18:27.941 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.941 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.941 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.941 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:27.942 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.942 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.942 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.942 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.942 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.942 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.942 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.942 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.942 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-16T09:18:27.942 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-16T09:18:27.942 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.942 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.942 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.943 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-03-16T09:18:27.943 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.943 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.943 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.943 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.943 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.943 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.943 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.943 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.943 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-16T09:18:27.943 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-16T09:18:27.943 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.943 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.943 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.943 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-03-16T09:18:27.944 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.944 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.944 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.944 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.944 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.944 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.944 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.944 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.944 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-03-16T09:18:27.944 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-16T09:18:27.944 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.944 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.944 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.944 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.945 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.945 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.945 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.945 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.945 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.945 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.945 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.945 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.945 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-16T09:18:27.945 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-16T09:18:27.945 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.945 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.945 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.945 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-03-16T09:18:27.946 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.946 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.946 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.946 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.946 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.946 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.946 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.946 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.946 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-03-16T09:18:27.946 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-16T09:18:27.946 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.946 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.946 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.946 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.947 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.947 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.947 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.947 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.947 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.947 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.947 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.947 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.947 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-03-16T09:18:27.947 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-16T09:18:27.947 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.947 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.947 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.947 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.948 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.948 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.948 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.948 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.948 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.948 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.948 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.948 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.948 INFO:teuthology.orchestra.run.smithi112.stdout: "UNSAFE_TLS_v1_2": { 2024-03-16T09:18:27.948 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-16T09:18:27.948 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.948 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.948 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.948 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.949 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.949 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.949 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.949 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.949 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.949 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.949 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.949 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.949 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-16T09:18:27.949 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-16T09:18:27.949 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.949 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.949 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.949 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-03-16T09:18:27.950 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.950 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.950 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.950 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.950 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.950 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.950 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.950 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-16T09:18:27.950 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-16T09:18:27.950 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.950 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.950 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.950 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:27.950 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.951 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.951 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.951 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.951 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.951 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.951 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.951 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-16T09:18:27.951 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-16T09:18:27.951 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.951 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.951 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.951 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-03-16T09:18:27.951 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.951 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.952 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.952 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.952 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.952 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.952 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.952 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-03-16T09:18:27.952 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-03-16T09:18:27.952 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.952 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.952 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.952 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.952 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.952 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.952 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.953 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.953 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.953 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.953 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.953 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-03-16T09:18:27.953 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-03-16T09:18:27.953 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.953 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.953 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.953 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.953 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.953 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.953 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.954 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.954 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.954 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.954 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.954 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-03-16T09:18:27.954 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-03-16T09:18:27.954 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.954 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.954 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.954 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.954 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.954 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.954 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.955 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-03-16T09:18:27.955 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.955 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.955 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.955 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-03-16T09:18:27.955 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-03-16T09:18:27.955 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.955 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.955 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.955 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-03-16T09:18:27.955 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.955 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.956 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.956 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.956 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.956 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.956 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.956 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-03-16T09:18:27.956 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-03-16T09:18:27.956 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.956 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.956 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.956 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.956 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.957 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.957 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.957 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.957 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.957 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.957 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.957 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.957 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.957 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.957 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.957 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.957 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.957 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.957 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.958 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.958 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.958 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.958 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.958 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.958 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.958 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.958 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.958 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.958 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.958 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.958 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.958 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.959 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.959 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.959 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.959 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.959 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.959 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.959 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.959 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.959 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.959 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.959 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.959 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.959 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.960 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.960 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.960 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.960 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.960 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.960 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.960 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.960 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.960 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.960 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.960 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.960 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.960 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.960 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.960 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.961 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.961 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.961 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.961 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.961 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.961 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.961 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.961 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.961 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.961 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.961 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.961 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.961 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.962 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.962 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.962 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.962 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.962 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-03-16T09:18:27.962 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-03-16T09:18:27.962 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.962 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.962 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.962 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.962 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.962 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.963 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.963 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-03-16T09:18:27.963 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.963 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.963 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.963 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-03-16T09:18:27.963 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-03-16T09:18:27.963 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.963 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.963 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.963 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.963 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.963 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.963 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.964 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.964 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.964 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.964 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.964 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-03-16T09:18:27.964 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-03-16T09:18:27.964 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.964 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.964 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.964 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-03-16T09:18:27.964 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.964 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.964 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.965 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.965 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.965 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.965 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.965 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.965 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-03-16T09:18:27.965 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-03-16T09:18:27.965 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.965 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.965 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.965 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-03-16T09:18:27.965 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.965 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.965 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.965 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.966 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.966 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.966 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.966 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-03-16T09:18:27.966 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-03-16T09:18:27.966 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.966 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.966 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.966 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.966 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.966 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.966 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.966 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.967 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.967 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.967 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.967 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.967 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-03-16T09:18:27.967 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-03-16T09:18:27.967 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.967 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.967 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.967 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-03-16T09:18:27.967 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.967 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.967 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.967 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.968 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.968 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.968 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.968 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-03-16T09:18:27.968 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-03-16T09:18:27.968 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.968 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.968 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.968 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-03-16T09:18:27.968 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.968 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.968 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.968 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.968 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-03-16T09:18:27.969 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.969 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.969 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.969 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.969 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.969 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.969 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-03-16T09:18:27.969 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-03-16T09:18:27.969 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.969 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.969 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.969 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-03-16T09:18:27.969 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-03-16T09:18:27.969 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-03-16T09:18:27.970 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.970 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.970 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.970 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.970 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.970 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-03-16T09:18:27.970 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-03-16T09:18:27.970 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.970 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.970 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.970 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.970 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.970 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.971 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.971 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.971 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.971 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.971 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.971 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.971 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.971 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.971 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.971 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-03-16T09:18:27.971 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.971 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.971 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.971 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-03-16T09:18:27.971 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-03-16T09:18:27.972 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.972 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.972 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.972 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.972 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.972 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.972 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.972 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-03-16T09:18:27.972 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.972 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.972 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.972 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.972 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.973 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.973 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.973 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.973 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.973 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.973 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.973 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.973 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.973 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.973 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.973 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.973 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.973 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.973 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.973 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.974 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.974 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.974 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.974 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.974 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.974 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.974 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.974 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.974 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.974 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.974 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.974 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.975 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.975 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.975 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.975 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.975 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.975 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.975 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.975 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.975 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.975 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.975 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.975 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.975 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.976 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.976 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.976 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.976 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.976 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.976 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.976 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.976 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.976 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.976 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.976 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.976 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.976 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.976 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.977 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.977 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.977 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.977 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.977 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.977 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.977 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.977 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.977 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.977 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.977 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.977 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-03-16T09:18:27.977 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-03-16T09:18:27.978 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.978 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.978 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.978 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-03-16T09:18:27.978 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.978 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.978 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.978 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-16T09:18:27.978 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.978 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.978 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.978 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.978 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-03-16T09:18:27.978 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-03-16T09:18:27.979 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.979 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.979 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.979 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-03-16T09:18:27.979 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.979 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.979 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.979 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-16T09:18:27.979 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.979 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.979 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.979 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-03-16T09:18:27.979 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-03-16T09:18:27.979 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.980 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.980 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.980 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-03-16T09:18:27.980 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.980 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.980 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.980 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-03-16T09:18:27.980 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.980 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.980 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.980 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-03-16T09:18:27.980 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-03-16T09:18:27.980 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.981 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.981 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.981 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-03-16T09:18:27.981 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.981 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.981 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.981 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-16T09:18:27.981 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.981 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.981 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.981 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-03-16T09:18:27.981 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-03-16T09:18:27.981 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.981 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.981 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.982 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:27.982 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.982 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.982 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.982 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-16T09:18:27.982 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.982 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.982 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.982 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-03-16T09:18:27.982 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-03-16T09:18:27.982 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.982 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.983 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.983 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-03-16T09:18:27.983 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.983 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.983 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.983 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-03-16T09:18:27.983 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.983 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.983 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.983 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.983 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-03-16T09:18:27.983 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-03-16T09:18:27.983 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.983 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.983 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.984 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-03-16T09:18:27.984 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.984 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.984 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.984 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-16T09:18:27.984 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.984 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.984 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.984 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.984 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.984 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.984 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-03-16T09:18:27.984 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:27.985 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:27.985 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.985 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.985 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.985 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.985 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.985 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.985 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.985 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.985 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.985 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.985 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.985 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.985 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.986 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.986 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.986 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.986 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.986 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.986 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.986 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.986 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.986 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.986 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.986 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.986 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.986 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.987 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.987 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.987 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.987 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.987 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.987 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.987 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.987 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.987 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:27.987 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:27.987 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.987 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.987 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.988 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:27.988 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.988 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.988 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.988 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.988 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.988 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.988 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.988 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.988 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.988 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.988 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.988 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.988 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.988 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.989 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:27.989 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:27.989 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.989 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.989 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.989 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.989 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.989 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.989 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.989 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.989 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.989 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.989 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.990 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-03-16T09:18:27.990 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-03-16T09:18:27.990 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.990 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.990 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.990 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-03-16T09:18:27.990 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.990 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.990 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.990 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.990 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.990 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.990 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.991 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-03-16T09:18:27.991 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-03-16T09:18:27.991 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.991 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.991 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.991 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-03-16T09:18:27.991 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.991 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.991 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.991 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.991 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.991 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.991 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.992 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-03-16T09:18:27.992 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-03-16T09:18:27.992 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.992 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.992 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.992 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-03-16T09:18:27.992 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.992 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.992 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.992 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.992 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.992 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.992 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.992 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.993 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:27.993 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.993 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:27.993 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-03-16T09:18:27.993 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-03-16T09:18:27.993 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-03-16T09:18:27.993 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:27.993 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-03-16T09:18:27.993 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-03-16T09:18:27.993 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:27.993 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.993 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.993 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-03-16T09:18:27.993 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.993 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.993 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.994 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-16T09:18:27.994 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.994 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.994 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.994 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-03-16T09:18:27.994 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-03-16T09:18:27.994 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.994 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.994 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.994 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-03-16T09:18:27.994 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.994 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.995 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.995 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-16T09:18:27.995 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.995 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.995 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.995 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-03-16T09:18:27.995 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-03-16T09:18:27.995 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.995 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.995 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.995 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.995 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.996 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.996 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.996 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-03-16T09:18:27.996 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.996 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.996 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.996 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-03-16T09:18:27.996 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-03-16T09:18:27.996 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:27.996 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.996 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:27.996 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-03-16T09:18:27.996 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-03-16T09:18:27.996 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.997 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.997 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-16T09:18:27.997 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.997 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.997 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.997 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:27.997 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:27.997 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:27.997 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.997 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.997 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:27.997 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.997 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.998 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:27.998 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:27.998 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:27.998 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:27.998 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:27.998 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:27.998 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:27.998 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:27.998 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.998 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.998 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.998 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.998 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:27.999 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:27.999 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:27.999 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:27.999 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:27.999 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:27.999 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:27.999 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:27.999 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:27.999 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:27.999 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:27.999 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:27.999 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:27.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:27.999 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.000 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.000 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.000 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.000 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.000 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.000 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.000 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.000 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.000 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.000 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.000 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.000 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.000 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.000 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.001 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.001 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.001 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.001 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.001 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.001 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.001 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.001 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.001 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.001 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.001 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.001 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.001 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.001 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.002 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.002 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.002 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.002 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.002 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-03-16T09:18:28.002 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-03-16T09:18:28.002 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.002 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.002 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.002 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.002 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.002 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.002 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.003 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-03-16T09:18:28.003 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.003 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.003 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.003 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-03-16T09:18:28.003 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-03-16T09:18:28.003 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.003 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.003 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.003 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-03-16T09:18:28.003 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.003 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.003 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.003 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-03-16T09:18:28.004 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.004 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.004 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.004 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.004 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-03-16T09:18:28.004 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-03-16T09:18:28.004 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.004 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.004 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.004 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-03-16T09:18:28.004 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.004 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.004 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.004 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-16T09:18:28.005 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.005 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.005 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.005 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-03-16T09:18:28.005 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-03-16T09:18:28.005 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.005 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.005 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.005 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-03-16T09:18:28.005 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-03-16T09:18:28.005 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-03-16T09:18:28.005 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.005 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-16T09:18:28.006 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.006 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.006 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.006 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-03-16T09:18:28.006 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-03-16T09:18:28.006 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.006 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.006 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.006 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.006 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.006 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.006 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.006 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-03-16T09:18:28.007 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.007 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.007 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.007 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.007 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-03-16T09:18:28.007 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-03-16T09:18:28.007 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.007 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.007 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.007 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-03-16T09:18:28.007 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.007 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.007 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.007 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-16T09:18:28.007 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.008 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.008 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.008 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.008 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.008 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.008 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-03-16T09:18:28.008 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.008 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.008 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.008 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.008 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.008 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.008 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.009 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.009 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.009 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.009 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.009 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.009 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.009 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.009 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.009 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.009 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.009 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.009 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.009 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.009 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.009 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.010 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.010 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.010 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.010 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.010 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.010 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.010 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.010 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.010 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.010 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.010 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.010 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.010 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.011 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.011 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.011 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.011 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.011 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.011 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.011 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.011 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.011 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.011 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.011 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.011 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.011 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.011 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.012 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.012 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.012 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.012 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.012 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.012 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.012 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.012 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.012 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.012 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.012 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.012 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.012 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.013 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.013 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.013 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.013 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.013 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.013 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.013 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.013 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.013 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.013 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.013 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-03-16T09:18:28.013 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.013 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.013 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.014 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.014 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.014 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.014 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.014 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.014 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.014 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.014 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.014 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.014 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.014 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.014 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.014 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.014 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.015 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.015 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.015 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.015 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.015 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.015 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.015 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.015 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.015 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.015 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.015 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.015 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.015 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.016 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.016 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.016 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.016 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.016 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.016 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.016 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.016 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.016 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.016 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.016 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.016 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.016 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.016 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.017 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.017 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.017 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.017 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.017 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.017 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.017 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.017 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.017 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.017 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.017 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.017 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.017 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.018 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.018 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.018 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.018 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.018 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.018 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.018 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.018 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.018 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.018 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.018 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.018 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.018 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.018 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.019 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.019 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-03-16T09:18:28.019 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.019 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.019 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.019 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-03-16T09:18:28.019 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-03-16T09:18:28.019 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.019 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.019 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.019 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-03-16T09:18:28.019 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.019 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.020 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.020 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-16T09:18:28.020 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.020 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.020 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.020 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.020 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-03-16T09:18:28.020 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-03-16T09:18:28.020 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.020 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.020 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.020 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:28.020 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-03-16T09:18:28.021 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.021 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.021 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-16T09:18:28.021 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.021 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.021 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.022 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.022 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.022 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.022 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.022 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.022 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.022 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.022 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.023 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.023 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.023 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.023 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.023 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.023 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.023 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.023 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.024 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.024 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.024 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.024 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.024 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.024 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.025 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.025 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.025 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.025 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.025 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.025 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.025 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.025 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.025 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.025 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.026 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.026 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.026 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.026 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.026 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.026 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.026 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.026 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.026 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.026 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.026 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.026 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.027 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.027 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.027 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.027 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.027 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.027 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.027 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.027 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.027 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.027 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.027 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.027 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.027 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.028 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.028 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.028 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.028 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.028 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.028 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.028 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.028 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.028 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.028 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.028 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.028 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.028 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-03-16T09:18:28.029 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.029 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.029 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.029 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-03-16T09:18:28.029 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-03-16T09:18:28.029 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.029 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.029 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.029 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-03-16T09:18:28.029 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.029 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.029 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.029 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-03-16T09:18:28.029 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-16T09:18:28.029 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.030 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.030 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.030 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.030 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.030 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.030 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.030 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.030 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.030 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.030 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.030 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.030 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.031 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.031 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.031 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.031 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.031 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.031 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.031 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.031 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.031 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.031 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.031 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.031 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.031 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.031 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.031 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.032 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.032 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.032 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.032 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.032 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.032 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.032 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.032 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.032 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.032 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.032 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.032 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.032 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.032 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.033 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.033 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.033 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.033 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.033 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.033 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.033 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.033 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.033 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.033 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.033 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.033 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.033 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.034 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.034 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.034 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.034 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.034 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.034 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.034 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.034 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.034 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.034 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.034 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.034 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.034 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.035 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-03-16T09:18:28.035 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-03-16T09:18:28.035 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.035 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.035 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.035 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.035 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.035 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.035 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.035 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-03-16T09:18:28.035 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-16T09:18:28.035 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.035 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.036 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-03-16T09:18:28.036 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-03-16T09:18:28.036 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.036 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.036 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.036 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-03-16T09:18:28.036 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.036 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.036 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.036 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-03-16T09:18:28.036 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.036 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.036 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.036 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-03-16T09:18:28.037 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-03-16T09:18:28.037 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.037 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.037 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.037 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-03-16T09:18:28.037 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.037 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.037 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.037 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-03-16T09:18:28.037 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.037 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.037 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.038 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-03-16T09:18:28.038 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-03-16T09:18:28.038 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.038 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.038 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.038 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.038 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.038 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.038 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.038 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-03-16T09:18:28.038 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.039 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.039 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.039 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-03-16T09:18:28.039 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-03-16T09:18:28.039 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.039 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.039 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.039 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-03-16T09:18:28.039 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.039 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.039 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.039 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-16T09:18:28.040 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-16T09:18:28.040 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.040 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.040 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.040 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.040 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.040 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-03-16T09:18:28.040 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.040 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.040 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.040 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.040 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.040 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.041 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.041 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.041 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.041 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.041 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.041 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.041 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.041 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.041 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.041 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.041 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.041 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.041 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.041 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.042 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.042 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.042 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.042 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.042 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.042 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.042 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.042 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.042 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.042 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.042 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.042 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.042 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.043 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.043 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.043 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.043 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.043 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.043 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.043 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.043 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.043 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.043 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.043 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.043 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.043 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.044 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.044 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.044 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.044 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.044 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.044 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.044 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.044 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.044 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.044 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.044 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.044 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.044 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.045 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.045 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.045 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.045 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.045 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.045 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.045 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.045 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.045 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.045 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.045 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.045 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.045 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.046 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-03-16T09:18:28.046 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.046 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.046 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.046 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.046 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.046 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.046 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.046 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.046 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.046 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.046 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.046 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.046 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.047 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.047 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.047 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.047 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.047 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.047 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.047 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.047 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.047 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.047 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.047 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.047 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.047 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.047 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.048 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.048 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.048 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.048 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.048 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.048 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.048 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.048 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.048 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.048 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.048 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.048 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.048 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.049 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.049 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.049 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.049 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.049 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.049 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.049 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.049 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.049 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.049 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.049 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.049 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.049 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.049 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.050 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.050 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.050 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.050 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.050 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.050 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.050 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.050 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.050 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.050 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.050 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.050 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.050 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.051 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.051 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.051 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.051 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.051 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.051 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-03-16T09:18:28.051 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.051 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.052 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.052 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.052 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.052 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.053 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.053 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.053 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.053 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.053 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.053 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.053 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.053 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.053 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.053 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.053 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.053 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.053 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.054 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.054 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.054 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.054 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.054 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.054 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.054 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.054 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.054 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.054 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.054 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.054 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.055 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.056 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.056 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.056 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.056 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.056 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.056 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.056 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.056 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.056 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.056 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.056 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.056 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.058 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.058 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-03-16T09:18:28.058 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.058 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.058 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.058 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-03-16T09:18:28.058 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-03-16T09:18:28.058 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.058 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.058 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.058 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.058 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.058 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.058 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-16T09:18:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.059 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.060 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.061 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.062 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.062 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.062 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.062 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.062 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.062 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.062 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.062 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.062 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.062 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.062 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.062 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.062 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.062 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.063 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.064 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.064 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-03-16T09:18:28.064 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-03-16T09:18:28.064 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.064 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.064 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.064 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.064 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.064 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.064 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.064 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-03-16T09:18:28.064 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-03-16T09:18:28.064 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-03-16T09:18:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-03-16T09:18:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-03-16T09:18:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.065 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.066 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.066 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.066 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.066 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.066 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.066 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.066 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.066 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.066 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.066 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.066 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.066 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.066 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.066 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.067 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.068 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.069 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.069 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.069 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.069 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.069 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.069 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.069 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.069 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.069 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.069 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.069 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.069 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.069 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-03-16T09:18:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.071 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.072 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.073 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.074 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.074 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.074 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.074 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.074 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.074 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.074 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.074 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.074 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.074 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.074 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.074 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.074 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.075 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.075 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.075 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.075 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.075 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.075 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.075 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.075 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.075 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.075 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.075 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.075 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.075 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.075 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-03-16T09:18:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.076 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.077 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.077 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.077 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.077 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.077 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.077 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.077 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.077 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.077 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.077 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.077 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.077 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.077 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.077 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.077 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.078 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.078 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.078 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.078 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.078 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.078 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.078 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.078 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.078 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.078 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.078 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.078 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.078 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.079 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.079 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.079 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.079 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.079 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.079 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.079 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.079 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.079 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.079 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.079 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.079 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.079 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.079 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.080 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.080 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.080 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.080 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.080 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.080 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.080 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.080 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.080 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.080 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.080 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.080 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-03-16T09:18:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-03-16T09:18:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.081 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.082 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-03-16T09:18:28.082 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.082 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.082 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.082 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.082 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.082 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.082 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.082 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-03-16T09:18:28.082 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-03-16T09:18:28.082 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:28.082 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-03-16T09:18:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-03-16T09:18:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-03-16T09:18:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-16T09:18:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-03-16T09:18:28.083 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-03-16T09:18:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-03-16T09:18:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-16T09:18:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.084 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-03-16T09:18:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-03-16T09:18:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.085 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.086 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.086 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.086 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.086 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.086 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.086 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.086 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.086 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.086 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.086 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.086 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.086 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.086 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.087 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.088 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.089 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-03-16T09:18:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-03-16T09:18:28.090 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-03-16T09:18:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-03-16T09:18:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-03-16T09:18:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-03-16T09:18:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.091 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-03-16T09:18:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-03-16T09:18:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-03-16T09:18:28.092 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-03-16T09:18:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-03-16T09:18:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.093 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.094 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.094 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-03-16T09:18:28.094 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-03-16T09:18:28.094 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.094 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.094 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.094 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.094 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.094 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.094 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.094 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-16T09:18:28.094 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-16T09:18:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.095 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.096 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.096 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.096 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.096 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.096 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.096 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.096 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.096 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.096 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.096 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.096 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.096 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.096 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.096 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.097 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.098 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.098 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.098 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.098 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.098 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.098 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.098 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.098 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.098 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.098 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.098 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.098 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.098 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.098 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-03-16T09:18:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-03-16T09:18:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-16T09:18:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-16T09:18:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-03-16T09:18:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.101 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-03-16T09:18:28.102 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-03-16T09:18:28.102 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:28.102 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.102 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.102 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-03-16T09:18:28.102 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.102 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.102 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.102 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.102 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.102 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.102 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-03-16T09:18:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-03-16T09:18:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-03-16T09:18:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-16T09:18:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.103 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-03-16T09:18:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-03-16T09:18:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-03-16T09:18:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-16T09:18:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-03-16T09:18:28.104 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-03-16T09:18:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-03-16T09:18:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-03-16T09:18:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-03-16T09:18:28.105 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.106 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.106 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.106 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-03-16T09:18:28.106 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.106 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.106 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.106 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-03-16T09:18:28.106 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-03-16T09:18:28.106 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.106 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.106 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.106 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.106 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-03-16T09:18:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-03-16T09:18:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-03-16T09:18:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-03-16T09:18:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-03-16T09:18:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.107 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-03-16T09:18:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.108 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.109 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.110 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.110 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.110 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.110 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.110 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.110 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.110 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.110 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.110 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.110 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.110 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.110 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.110 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.111 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.111 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.111 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.111 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.111 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.111 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.111 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.111 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.111 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.111 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.111 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.111 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.111 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.112 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-03-16T09:18:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-03-16T09:18:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.113 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.114 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.114 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.114 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.114 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.114 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-03-16T09:18:28.114 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-03-16T09:18:28.114 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.114 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.114 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.114 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.114 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.114 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.114 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.115 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.115 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.115 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.115 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.115 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-03-16T09:18:28.115 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-03-16T09:18:28.115 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.115 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.115 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.115 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.115 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.115 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.115 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-03-16T09:18:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-03-16T09:18:28.116 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-03-16T09:18:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-03-16T09:18:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-03-16T09:18:28.117 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.118 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.118 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.118 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.118 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.118 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.118 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.118 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.118 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.118 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.118 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.118 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.118 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.118 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.119 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.119 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.119 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.119 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.119 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.119 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.119 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.119 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.119 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.119 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.119 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.119 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.119 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.119 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.119 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.120 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.120 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.120 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.120 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.120 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.120 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.120 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.120 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.120 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.120 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.120 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.120 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.122 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.122 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.122 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.122 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.122 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.122 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.122 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.122 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.122 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.122 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.122 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.122 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.123 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.123 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.123 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.123 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.123 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.123 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.123 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.123 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.123 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.123 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.123 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-03-16T09:18:28.123 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-03-16T09:18:28.123 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.124 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.124 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.124 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.124 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.124 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.124 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.124 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.124 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.124 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.124 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.124 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-03-16T09:18:28.124 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-03-16T09:18:28.124 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.125 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.125 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.125 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.125 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.125 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.125 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.125 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.125 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.125 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.125 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.125 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.125 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.125 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.126 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-03-16T09:18:28.126 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.126 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.126 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.126 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.126 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.126 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.126 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.126 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.126 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.126 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.126 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.126 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.126 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.127 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.129 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.129 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.129 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.129 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.129 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.129 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.129 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.129 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.129 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.129 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.129 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.129 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.129 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.129 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.130 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.130 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.130 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.130 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.130 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.130 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.130 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.130 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.130 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.130 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.130 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.130 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.130 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-03-16T09:18:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.131 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.132 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.133 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.134 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.134 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.134 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.134 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.134 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.134 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.134 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.134 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.134 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.134 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.134 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.134 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.134 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.135 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-03-16T09:18:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-03-16T09:18:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.136 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-03-16T09:18:28.137 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.137 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.137 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.137 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-16T09:18:28.137 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.137 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.137 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.137 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.137 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.137 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.137 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-03-16T09:18:28.137 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.137 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.138 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.138 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.138 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.138 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.138 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.138 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.138 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.138 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.138 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.138 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.138 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.138 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.138 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.139 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.140 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.142 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.142 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.142 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.142 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.142 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.142 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.142 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.142 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.142 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.142 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.142 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.142 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.142 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-03-16T09:18:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-03-16T09:18:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-03-16T09:18:28.143 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-03-16T09:18:28.144 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.144 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.144 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.144 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-03-16T09:18:28.144 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.144 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.144 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.144 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.144 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.144 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.144 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.144 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-03-16T09:18:28.144 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-03-16T09:18:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-03-16T09:18:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-03-16T09:18:28.145 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.146 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.146 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.146 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.146 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.146 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.146 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.146 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.146 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.146 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.146 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.146 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-03-16T09:18:28.146 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-03-16T09:18:28.146 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:28.147 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.147 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.147 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.147 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.147 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.147 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.147 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.147 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.147 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.147 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.147 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-03-16T09:18:28.147 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-03-16T09:18:28.147 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-03-16T09:18:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-03-16T09:18:28.148 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-03-16T09:18:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-03-16T09:18:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.149 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.150 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.150 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.150 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.150 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.150 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.150 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.150 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.150 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.150 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.150 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-03-16T09:18:28.150 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-03-16T09:18:28.150 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-03-16T09:18:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-03-16T09:18:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-03-16T09:18:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-03-16T09:18:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.151 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-03-16T09:18:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-03-16T09:18:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.152 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-03-16T09:18:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-03-16T09:18:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.153 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.154 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.154 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.154 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.154 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.154 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.154 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.154 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.154 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.154 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.154 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.154 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-03-16T09:18:28.154 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.154 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.155 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.155 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-03-16T09:18:28.155 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-03-16T09:18:28.155 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.155 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.155 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.155 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.155 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.155 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.155 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.155 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-03-16T09:18:28.155 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.155 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.155 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-03-16T09:18:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-03-16T09:18:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-03-16T09:18:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.157 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.158 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.158 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.158 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.158 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.158 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.158 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.158 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.158 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.158 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.158 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.158 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.158 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.158 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.159 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-03-16T09:18:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.162 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.163 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.163 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.163 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.163 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.163 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.163 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.163 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.163 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.163 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.163 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.163 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.163 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.163 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.164 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.164 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.164 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.164 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.164 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.164 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.164 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.164 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.164 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.164 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.164 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.164 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.164 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.166 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.166 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.166 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.166 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.166 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.166 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.166 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.166 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.166 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.166 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.166 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.166 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.166 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-03-16T09:18:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.167 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.168 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.168 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.168 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.168 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.168 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.168 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.168 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.168 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.168 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.168 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.168 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.168 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.168 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.168 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.169 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.170 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.170 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.170 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.170 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.170 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.170 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.170 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.170 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.170 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.170 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.170 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.170 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.170 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.171 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.172 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.172 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.172 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.172 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.172 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.172 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.172 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.172 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.172 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.172 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.172 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-03-16T09:18:28.172 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.172 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-03-16T09:18:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-03-16T09:18:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-16T09:18:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.174 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-03-16T09:18:28.174 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-03-16T09:18:28.174 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.174 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.174 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.174 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-03-16T09:18:28.174 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.174 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.174 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.174 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.174 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.174 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.174 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.175 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.175 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.175 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.175 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.175 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.175 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.175 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.175 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.175 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.175 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.175 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.175 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.175 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.176 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.176 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.176 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.176 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.176 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.176 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.176 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.176 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.176 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.176 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.176 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.176 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.176 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.177 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.178 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.178 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.178 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.178 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.178 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.178 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.178 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.178 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.178 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.178 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.178 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.178 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.178 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.179 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-03-16T09:18:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-03-16T09:18:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-03-16T09:18:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.180 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-16T09:18:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-03-16T09:18:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-03-16T09:18:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.181 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.182 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.182 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.182 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-16T09:18:28.182 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.182 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.182 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.182 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-03-16T09:18:28.182 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-03-16T09:18:28.182 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.182 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.182 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.182 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.183 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.183 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.183 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.183 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-16T09:18:28.183 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.183 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.183 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.183 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-03-16T09:18:28.183 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-03-16T09:18:28.183 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.183 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.183 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.183 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-16T09:18:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-03-16T09:18:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-03-16T09:18:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.184 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.185 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.185 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.185 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.185 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-16T09:18:28.185 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.185 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.185 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.185 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-03-16T09:18:28.185 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-03-16T09:18:28.185 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.185 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.185 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.186 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.186 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.186 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.186 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.186 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.186 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.186 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.186 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.186 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-03-16T09:18:28.186 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-03-16T09:18:28.186 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.186 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.186 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-03-16T09:18:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-03-16T09:18:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.187 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-16T09:18:28.188 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.188 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.188 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.188 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.188 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.188 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.188 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.188 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-03-16T09:18:28.188 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-03-16T09:18:28.188 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.188 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.188 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.188 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-03-16T09:18:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-03-16T09:18:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-03-16T09:18:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-03-16T09:18:28.189 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.190 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.190 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.190 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.190 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.190 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.190 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-03-16T09:18:28.190 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-03-16T09:18:28.190 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.190 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.190 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.190 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-03-16T09:18:28.190 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.191 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.191 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.191 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.191 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.191 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.191 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.191 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-03-16T09:18:28.191 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-03-16T09:18:28.191 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.191 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.191 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-03-16T09:18:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-03-16T09:18:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.192 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.193 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.193 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.193 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.193 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.193 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.193 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.193 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.193 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.193 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.193 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.193 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.193 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.193 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.194 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.194 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.194 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.194 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.194 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.194 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.194 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.194 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.194 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.194 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.194 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.194 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.195 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.195 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.195 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.195 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.195 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.195 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.195 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.195 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.195 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.195 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.195 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.195 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.196 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.196 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.196 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.196 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.196 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.196 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.196 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.196 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.196 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.196 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.196 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.196 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.196 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.196 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.198 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.198 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.198 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.198 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.198 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.198 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.198 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.198 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.198 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.198 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.198 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.198 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-03-16T09:18:28.198 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-03-16T09:18:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-03-16T09:18:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-03-16T09:18:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-03-16T09:18:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-03-16T09:18:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.200 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-16T09:18:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-03-16T09:18:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.201 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.202 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.202 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.202 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.202 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.202 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.202 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.202 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.202 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.202 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.202 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.202 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.202 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.202 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.203 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.203 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.203 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.203 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.203 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.203 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.203 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.203 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.203 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.203 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.203 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.203 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.203 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.204 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.204 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.204 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.204 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.204 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.204 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.204 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.204 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.204 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.204 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.204 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.204 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.204 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.204 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.206 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.206 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.206 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.206 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.206 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.206 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.206 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.206 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.206 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.206 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.206 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.206 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.206 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.206 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.207 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.207 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.207 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-03-16T09:18:28.207 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.207 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.207 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.207 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.207 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.207 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.207 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.207 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.207 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.209 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.209 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.209 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.209 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.209 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.209 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.209 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.209 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.209 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.209 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.209 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.209 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.210 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.211 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.212 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.212 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.212 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.212 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.212 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.212 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.212 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.212 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-03-16T09:18:28.212 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-03-16T09:18:28.212 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.212 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.212 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.212 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-03-16T09:18:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-16T09:18:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-03-16T09:18:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-03-16T09:18:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.213 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.214 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.214 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.214 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-03-16T09:18:28.214 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.214 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.214 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.214 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-03-16T09:18:28.214 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-03-16T09:18:28.214 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.214 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.214 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.214 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-03-16T09:18:28.214 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.215 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.215 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.215 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-16T09:18:28.215 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.215 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.215 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.215 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-03-16T09:18:28.215 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-03-16T09:18:28.215 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.215 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.215 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.215 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.215 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.216 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.216 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.216 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-16T09:18:28.216 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.216 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.216 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.216 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.216 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.216 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.216 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-03-16T09:18:28.216 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.216 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-03-16T09:18:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-03-16T09:18:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-03-16T09:18:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-03-16T09:18:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.217 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-03-16T09:18:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-03-16T09:18:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-03-16T09:18:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-03-16T09:18:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-03-16T09:18:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.222 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.222 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.222 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.222 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.222 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.222 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.222 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.222 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.222 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.222 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.222 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.222 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.223 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.224 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.224 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.224 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.224 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.224 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.224 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.224 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.224 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.224 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.224 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.224 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.224 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.224 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-03-16T09:18:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-03-16T09:18:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.225 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-03-16T09:18:28.226 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-03-16T09:18:28.226 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.226 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.226 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.226 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-03-16T09:18:28.226 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.226 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.226 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.226 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.226 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.226 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.226 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-03-16T09:18:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-03-16T09:18:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-16T09:18:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.227 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-03-16T09:18:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: "modules": [ 2024-03-16T09:18:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-03-16T09:18:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: "dashboard", 2024-03-16T09:18:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: "iostat", 2024-03-16T09:18:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: "nfs", 2024-03-16T09:18:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: "restful" 2024-03-16T09:18:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-03-16T09:18:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.228 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-03-16T09:18:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-03-16T09:18:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-03-16T09:18:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-03-16T09:18:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-03-16T09:18:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.230 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.230 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.230 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.230 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.230 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.230 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.230 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.230 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.230 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.230 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.230 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.230 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.231 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.232 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.233 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.233 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.233 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.233 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.233 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.233 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.233 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.233 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.233 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.233 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.233 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.233 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.233 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.233 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.234 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.234 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.234 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.234 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.234 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.234 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.234 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.234 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.234 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.234 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.234 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.234 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.235 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.235 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-03-16T09:18:28.235 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-03-16T09:18:28.235 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.235 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.235 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.235 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.235 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.235 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.235 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.235 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to", 2024-03-16T09:18:28.235 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.235 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-03-16T09:18:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-03-16T09:18:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-03-16T09:18:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-03-16T09:18:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.236 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.237 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-03-16T09:18:28.237 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-03-16T09:18:28.237 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.237 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.237 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.237 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.237 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.237 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.237 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.237 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-03-16T09:18:28.237 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.237 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.237 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-03-16T09:18:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-03-16T09:18:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-03-16T09:18:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.238 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-03-16T09:18:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-03-16T09:18:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-03-16T09:18:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-03-16T09:18:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.239 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-03-16T09:18:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-03-16T09:18:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-03-16T09:18:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-03-16T09:18:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-03-16T09:18:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-16T09:18:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.241 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-03-16T09:18:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-03-16T09:18:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-03-16T09:18:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.242 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-03-16T09:18:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-03-16T09:18:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-03-16T09:18:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.243 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-03-16T09:18:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-03-16T09:18:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-03-16T09:18:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-03-16T09:18:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.244 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-03-16T09:18:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-16T09:18:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-03-16T09:18:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-03-16T09:18:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-03-16T09:18:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-03-16T09:18:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-03-16T09:18:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-03-16T09:18:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.245 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-16T09:18:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-16T09:18:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-03-16T09:18:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-03-16T09:18:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-03-16T09:18:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-03-16T09:18:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-03-16T09:18:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-03-16T09:18:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.246 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-16T09:18:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-03-16T09:18:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-03-16T09:18:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-03-16T09:18:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-16T09:18:28.247 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-16T09:18:28.248 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.248 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.248 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-03-16T09:18:28.248 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-03-16T09:18:28.248 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:28.248 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.248 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.248 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-03-16T09:18:28.248 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-03-16T09:18:28.248 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-03-16T09:18:28.248 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.248 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-03-16T09:18:28.248 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-16T09:18:28.249 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.249 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.249 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-03-16T09:18:28.249 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-03-16T09:18:28.249 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.249 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.249 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.249 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-03-16T09:18:28.249 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.249 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.249 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.249 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-03-16T09:18:28.249 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-16T09:18:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-03-16T09:18:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-03-16T09:18:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-03-16T09:18:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-03-16T09:18:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-03-16T09:18:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-03-16T09:18:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-16T09:18:28.250 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-16T09:18:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.251 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.252 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.253 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.253 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.253 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.253 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.253 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.253 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.253 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.253 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.253 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.253 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.253 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.253 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.254 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.254 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.254 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.254 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.254 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.254 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.254 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.254 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.254 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.254 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.254 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.254 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.255 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.255 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.255 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.255 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.255 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.255 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.255 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.255 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.255 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.255 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.255 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.255 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.256 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.256 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.256 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.256 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-03-16T09:18:28.256 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-03-16T09:18:28.256 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:28.256 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.256 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.256 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-03-16T09:18:28.256 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.256 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.256 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-16T09:18:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-03-16T09:18:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-03-16T09:18:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-03-16T09:18:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-03-16T09:18:28.257 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-03-16T09:18:28.258 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-03-16T09:18:28.258 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-03-16T09:18:28.258 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-03-16T09:18:28.258 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.258 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-03-16T09:18:28.258 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.258 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.258 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.258 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-03-16T09:18:28.258 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-03-16T09:18:28.258 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.258 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.258 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-16T09:18:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-03-16T09:18:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-03-16T09:18:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.259 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-03-16T09:18:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-16T09:18:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-03-16T09:18:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-03-16T09:18:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.260 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-03-16T09:18:28.261 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-03-16T09:18:28.261 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.261 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.261 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-16T09:18:28.261 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-16T09:18:28.261 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.261 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.261 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-03-16T09:18:28.261 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-03-16T09:18:28.261 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-03-16T09:18:28.261 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.261 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.262 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:28.262 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.262 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.262 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.262 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-16T09:18:28.262 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.262 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.262 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.262 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.262 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.262 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.262 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-03-16T09:18:28.262 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-03-16T09:18:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-03-16T09:18:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-03-16T09:18:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-16T09:18:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.263 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-03-16T09:18:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-03-16T09:18:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-03-16T09:18:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-16T09:18:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.264 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-03-16T09:18:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-03-16T09:18:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-03-16T09:18:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-16T09:18:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.266 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-03-16T09:18:28.266 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-03-16T09:18:28.266 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.266 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.266 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.266 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.266 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.266 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.266 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.266 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-16T09:18:28.266 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-16T09:18:28.266 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.266 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-03-16T09:18:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-03-16T09:18:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-03-16T09:18:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-16T09:18:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.268 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-03-16T09:18:28.268 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-03-16T09:18:28.268 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:28.268 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.268 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.268 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-03-16T09:18:28.268 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.268 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.268 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.268 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-16T09:18:28.268 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.268 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.268 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.269 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-03-16T09:18:28.269 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-03-16T09:18:28.269 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.269 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.269 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.269 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.269 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.269 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.269 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.269 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-03-16T09:18:28.269 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-03-16T09:18:28.269 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-03-16T09:18:28.269 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.269 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-16T09:18:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-03-16T09:18:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-03-16T09:18:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-16T09:18:28.270 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.271 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.271 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.271 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-03-16T09:18:28.271 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-03-16T09:18:28.271 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.271 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.271 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.271 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.271 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.271 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.271 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.271 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-16T09:18:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-03-16T09:18:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-03-16T09:18:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-16T09:18:28.272 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.273 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.273 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.273 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-03-16T09:18:28.273 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-03-16T09:18:28.273 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.273 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.273 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.273 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-16T09:18:28.273 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.273 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.273 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.273 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-03-16T09:18:28.273 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.274 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.274 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.274 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-03-16T09:18:28.274 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-03-16T09:18:28.274 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.274 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.274 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.274 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-16T09:18:28.274 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.274 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.274 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.274 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-03-16T09:18:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-03-16T09:18:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-03-16T09:18:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-16T09:18:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-03-16T09:18:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.275 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-03-16T09:18:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-03-16T09:18:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-16T09:18:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-03-16T09:18:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.277 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-03-16T09:18:28.277 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-03-16T09:18:28.277 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.277 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.277 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.277 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-16T09:18:28.277 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.277 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.277 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.277 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-03-16T09:18:28.277 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.277 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.277 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.277 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.278 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-03-16T09:18:28.278 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-03-16T09:18:28.278 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.278 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.278 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.278 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-16T09:18:28.278 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.278 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.278 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.278 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-03-16T09:18:28.278 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.278 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.278 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.278 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-03-16T09:18:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-03-16T09:18:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-16T09:18:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-03-16T09:18:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.279 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-03-16T09:18:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-03-16T09:18:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-16T09:18:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-03-16T09:18:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.280 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-03-16T09:18:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-03-16T09:18:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-16T09:18:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-03-16T09:18:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-03-16T09:18:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-03-16T09:18:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.281 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-16T09:18:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-03-16T09:18:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-03-16T09:18:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-03-16T09:18:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.282 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-16T09:18:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-03-16T09:18:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-03-16T09:18:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-03-16T09:18:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.283 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-16T09:18:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-03-16T09:18:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-03-16T09:18:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-03-16T09:18:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-16T09:18:28.284 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-03-16T09:18:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-03-16T09:18:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-03-16T09:18:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-16T09:18:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.285 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-03-16T09:18:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-03-16T09:18:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-03-16T09:18:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-16T09:18:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.286 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-03-16T09:18:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-03-16T09:18:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-03-16T09:18:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-03-16T09:18:28.287 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.288 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.288 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.288 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-03-16T09:18:28.288 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-03-16T09:18:28.288 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.288 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.288 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.288 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-03-16T09:18:28.288 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.288 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.288 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.288 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-16T09:18:28.288 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.288 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.289 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.289 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-03-16T09:18:28.289 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-03-16T09:18:28.289 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.289 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.289 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.289 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-03-16T09:18:28.289 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.289 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.289 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.289 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-16T09:18:28.289 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.290 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.290 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.290 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-03-16T09:18:28.290 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-03-16T09:18:28.290 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.290 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.290 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.290 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-03-16T09:18:28.290 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.290 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.290 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.290 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-16T09:18:28.290 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.290 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.291 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.291 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-03-16T09:18:28.291 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-03-16T09:18:28.291 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.291 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.291 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.291 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-03-16T09:18:28.291 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.291 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.291 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.291 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-03-16T09:18:28.291 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.291 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.291 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.292 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-03-16T09:18:28.292 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-03-16T09:18:28.292 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.292 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.292 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.292 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.292 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.292 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.292 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.292 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-16T09:18:28.292 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.292 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.292 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.293 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-03-16T09:18:28.293 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-03-16T09:18:28.293 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.293 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.293 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.293 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-03-16T09:18:28.293 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.293 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.293 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.293 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-03-16T09:18:28.293 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.293 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.293 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.293 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-03-16T09:18:28.294 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-03-16T09:18:28.294 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.294 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.294 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.294 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-03-16T09:18:28.294 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.294 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.294 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.294 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-03-16T09:18:28.294 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.294 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.294 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.295 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-03-16T09:18:28.295 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-03-16T09:18:28.295 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.295 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.295 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.295 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.295 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.295 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.295 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.295 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-16T09:18:28.295 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.295 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.295 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.295 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-03-16T09:18:28.296 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-03-16T09:18:28.296 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.296 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.296 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.296 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.296 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.296 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.296 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.296 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-16T09:18:28.296 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.296 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.296 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.296 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.296 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.296 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.297 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.297 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.297 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.297 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.297 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.297 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.297 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.297 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.297 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.297 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.297 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.297 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.297 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.297 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.297 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.298 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.298 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.298 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-03-16T09:18:28.298 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-03-16T09:18:28.298 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.298 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.298 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.298 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.298 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.298 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.298 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.298 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-16T09:18:28.298 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.299 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.299 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.299 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.299 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.299 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.299 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.299 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.299 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.299 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.299 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.299 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.299 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-16T09:18:28.299 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.300 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.300 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.300 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.300 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.300 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.300 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.300 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.300 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.300 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.300 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.300 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.300 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.300 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.300 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.300 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.300 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.301 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.301 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.301 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.301 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.301 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.301 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.301 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.301 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.301 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.301 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.301 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.301 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.301 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.301 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.301 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.302 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.302 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.302 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.302 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.302 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-16T09:18:28.302 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-16T09:18:28.302 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.302 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.302 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.302 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.302 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.302 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.302 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.303 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-16T09:18:28.303 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.303 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.303 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.303 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-16T09:18:28.303 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-16T09:18:28.303 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.303 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.303 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.303 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-03-16T09:18:28.303 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.303 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.303 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.303 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-16T09:18:28.303 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.304 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.304 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.304 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-03-16T09:18:28.304 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-03-16T09:18:28.304 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.304 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.304 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.304 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:28.304 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.304 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.304 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.304 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-03-16T09:18:28.304 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.304 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.304 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.305 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.305 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-03-16T09:18:28.305 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-03-16T09:18:28.305 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.305 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.305 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.305 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:28.305 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.305 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.305 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.305 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-16T09:18:28.305 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.305 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.305 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.306 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-03-16T09:18:28.306 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-03-16T09:18:28.306 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.306 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.306 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.306 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.306 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.306 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.306 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.306 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-03-16T09:18:28.306 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.306 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.306 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.307 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-03-16T09:18:28.307 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-03-16T09:18:28.307 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.307 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.307 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.307 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-03-16T09:18:28.307 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.307 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.307 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.307 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-03-16T09:18:28.307 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-03-16T09:18:28.307 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.307 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-03-16T09:18:28.307 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.308 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.308 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.308 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.308 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-03-16T09:18:28.308 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-03-16T09:18:28.308 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.308 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.308 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.308 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-03-16T09:18:28.308 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.308 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.308 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.308 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-16T09:18:28.308 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.308 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.308 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.309 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-03-16T09:18:28.309 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-03-16T09:18:28.309 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.309 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.309 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.309 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-16T09:18:28.309 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.309 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.309 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.309 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-16T09:18:28.309 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.309 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.309 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.310 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-03-16T09:18:28.310 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-03-16T09:18:28.310 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.310 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.310 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.310 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.310 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.310 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.310 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.310 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-16T09:18:28.310 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.310 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.310 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-03-16T09:18:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-03-16T09:18:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-16T09:18:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-03-16T09:18:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-03-16T09:18:28.312 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.312 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.312 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.312 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.312 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.312 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.312 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.312 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-16T09:18:28.312 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.312 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.312 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.312 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.312 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-03-16T09:18:28.312 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-03-16T09:18:28.312 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.312 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.313 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.313 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.313 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.313 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.313 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.313 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-16T09:18:28.313 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.313 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.313 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.313 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-03-16T09:18:28.313 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-03-16T09:18:28.313 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.313 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.313 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.313 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.314 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.314 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.314 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.314 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-16T09:18:28.314 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.314 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.314 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.314 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-03-16T09:18:28.314 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-03-16T09:18:28.314 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.314 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.314 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.315 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-03-16T09:18:28.315 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.315 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.315 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.315 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-03-16T09:18:28.315 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.315 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.315 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.315 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-03-16T09:18:28.315 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-03-16T09:18:28.315 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.315 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.315 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.315 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-16T09:18:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-03-16T09:18:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-03-16T09:18:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.317 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.317 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.317 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.317 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-16T09:18:28.317 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.317 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.317 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.317 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-03-16T09:18:28.317 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-03-16T09:18:28.317 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.317 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.317 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.317 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.318 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.318 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.318 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.318 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-16T09:18:28.318 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.318 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.318 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.318 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-03-16T09:18:28.318 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-03-16T09:18:28.318 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.318 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.318 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.319 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.319 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.319 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.319 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.319 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-03-16T09:18:28.319 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.319 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.319 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.319 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-03-16T09:18:28.319 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-03-16T09:18:28.319 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.319 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.319 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.320 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.320 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.320 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.320 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.320 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-16T09:18:28.320 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.320 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.320 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.320 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-03-16T09:18:28.320 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-03-16T09:18:28.320 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.320 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.320 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.321 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.321 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.321 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.321 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.321 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-16T09:18:28.321 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.321 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.321 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.321 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.321 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.321 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.321 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-03-16T09:18:28.321 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.322 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.322 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.322 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.322 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.322 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.322 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.322 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.322 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.322 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.322 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.322 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.322 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.322 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.322 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.323 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.323 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.323 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.323 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.323 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.323 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.323 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.323 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.323 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.323 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.323 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.323 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.323 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.324 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.324 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.324 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.324 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.324 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.324 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.324 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.324 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.324 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.324 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.324 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.324 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.324 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.325 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.325 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.325 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.325 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.325 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.325 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.325 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.325 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.325 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.325 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.325 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.325 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.325 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.325 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.325 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.326 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.326 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.326 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.326 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.326 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.326 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.326 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.326 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.326 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.326 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.326 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.326 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.326 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.327 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-03-16T09:18:28.327 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-03-16T09:18:28.327 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.327 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.327 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.327 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-03-16T09:18:28.327 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.327 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.327 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.327 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-16T09:18:28.327 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.327 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.327 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.328 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-03-16T09:18:28.328 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-03-16T09:18:28.328 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.328 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.328 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.328 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-03-16T09:18:28.328 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.328 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.328 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.328 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-16T09:18:28.328 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.328 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.328 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.328 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.328 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.329 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.329 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-03-16T09:18:28.329 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.329 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.329 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.329 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-16T09:18:28.329 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-16T09:18:28.329 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.329 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.329 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.329 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:28.329 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.329 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.329 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.329 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.330 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.330 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.330 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.330 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-03-16T09:18:28.330 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-16T09:18:28.330 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.330 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.330 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.330 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.330 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.330 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.330 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.331 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.331 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.331 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.331 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.331 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-16T09:18:28.331 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-16T09:18:28.331 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.331 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.331 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.331 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.331 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.331 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.331 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.331 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.331 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.332 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.332 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.332 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-03-16T09:18:28.332 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-03-16T09:18:28.332 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.332 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.332 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.332 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.332 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.332 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.332 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.332 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.333 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.333 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.333 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.333 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.333 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-16T09:18:28.333 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-16T09:18:28.333 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.333 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.333 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.333 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.333 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.333 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.333 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.333 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.333 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.334 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.334 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.334 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-03-16T09:18:28.334 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-16T09:18:28.334 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.334 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.334 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.334 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.334 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.334 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.334 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.334 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.335 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.335 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.335 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.335 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.335 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-16T09:18:28.335 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-16T09:18:28.335 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.335 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.335 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.335 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.335 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.335 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.335 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.335 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.335 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.335 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.336 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.336 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-16T09:18:28.336 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-16T09:18:28.336 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.336 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.336 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.336 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.336 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.336 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.336 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.336 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.336 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.336 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.336 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.337 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-16T09:18:28.337 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-16T09:18:28.337 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.337 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.337 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.337 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.337 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.337 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.337 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.337 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.337 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.337 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.337 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.337 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-16T09:18:28.338 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-16T09:18:28.338 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.338 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.338 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.338 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.338 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.338 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.338 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.338 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.338 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.338 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.338 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.338 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-16T09:18:28.339 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-16T09:18:28.339 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.339 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.339 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.339 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.339 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.339 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.339 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.339 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.339 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.339 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.339 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.339 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.339 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-16T09:18:28.339 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-16T09:18:28.339 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.340 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.340 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.340 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.340 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.340 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.340 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.340 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.340 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.340 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.340 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.340 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.340 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-16T09:18:28.340 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-16T09:18:28.340 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.340 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.340 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.341 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.341 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.341 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.341 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.341 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.341 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.341 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.341 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.341 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.341 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-16T09:18:28.341 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-16T09:18:28.341 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.341 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.341 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.341 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.342 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.342 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.342 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.342 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.342 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.342 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.342 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.342 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-03-16T09:18:28.342 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-16T09:18:28.342 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.342 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.342 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.342 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-03-16T09:18:28.342 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.343 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.343 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.343 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.343 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.343 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.343 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.343 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-16T09:18:28.343 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-16T09:18:28.343 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.343 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.343 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.343 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.343 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.343 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.344 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.344 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.344 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.344 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.344 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.344 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-03-16T09:18:28.344 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-03-16T09:18:28.344 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.344 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.344 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.344 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.344 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.344 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.344 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.344 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.345 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.345 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.345 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.345 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.345 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-03-16T09:18:28.345 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-16T09:18:28.345 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.345 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.345 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.345 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-03-16T09:18:28.345 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.345 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.345 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.345 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.345 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.346 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.346 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.346 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.346 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-16T09:18:28.346 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-16T09:18:28.346 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.346 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.346 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.346 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.346 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.346 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.346 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.346 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.346 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.346 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.347 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.347 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-16T09:18:28.347 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-16T09:18:28.347 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.347 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.347 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.347 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.347 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.347 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.347 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.347 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.347 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.348 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.348 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.348 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-16T09:18:28.348 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-16T09:18:28.348 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.348 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.348 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.348 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.348 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.348 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.348 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.348 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.348 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.349 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.349 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.349 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-16T09:18:28.349 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-16T09:18:28.349 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.349 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.349 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.349 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.349 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.349 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.349 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.349 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.349 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.350 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.350 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.350 INFO:teuthology.orchestra.run.smithi112.stdout: "MULTICLUSTER_CONFIG": { 2024-03-16T09:18:28.350 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-16T09:18:28.350 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.350 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.350 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.350 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "{}", 2024-03-16T09:18:28.350 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.350 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.350 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.350 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.350 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.351 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.351 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.351 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-03-16T09:18:28.351 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-16T09:18:28.351 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.351 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.351 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.351 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.351 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.351 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.351 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.351 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.352 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.352 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.352 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.352 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-16T09:18:28.352 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-16T09:18:28.352 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.352 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.352 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.352 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.352 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.352 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.352 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.352 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.352 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.353 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.353 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.353 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-16T09:18:28.353 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-16T09:18:28.353 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.353 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.353 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.353 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.353 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.353 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.353 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.353 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.353 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.354 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.354 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.354 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-16T09:18:28.354 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-16T09:18:28.354 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.354 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.354 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.354 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.354 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.354 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.354 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.354 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.354 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.355 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.355 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.355 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-16T09:18:28.355 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-16T09:18:28.355 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.355 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.355 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.355 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.355 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.355 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.355 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.355 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.355 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.356 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.356 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.356 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-16T09:18:28.356 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-16T09:18:28.356 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.356 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.356 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.356 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.356 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.356 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.356 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.356 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.356 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.357 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.357 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.357 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-16T09:18:28.357 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-16T09:18:28.357 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.357 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.357 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.357 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.357 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.357 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.357 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.357 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.358 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.358 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.358 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.358 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-16T09:18:28.358 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-16T09:18:28.358 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.358 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.358 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.358 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.358 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.358 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.358 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.358 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.359 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.359 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.359 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.359 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-16T09:18:28.359 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-16T09:18:28.359 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.359 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.359 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.359 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.359 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.359 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.359 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.359 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.359 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.360 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.360 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.360 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-03-16T09:18:28.360 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-16T09:18:28.360 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.360 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.360 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.360 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.360 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.360 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.360 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.360 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.360 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.360 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.360 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.361 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-16T09:18:28.361 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-16T09:18:28.361 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.361 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.361 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.361 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-16T09:18:28.361 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.361 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.361 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.361 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.361 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.361 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.361 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.362 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-16T09:18:28.362 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-16T09:18:28.362 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.362 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.362 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.362 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:28.362 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.362 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.362 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.362 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.362 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.362 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.362 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.363 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-16T09:18:28.363 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-16T09:18:28.363 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.363 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.363 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.363 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-03-16T09:18:28.363 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.363 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.363 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.363 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.363 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.363 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.363 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.364 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-16T09:18:28.364 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-16T09:18:28.364 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.364 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.364 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.364 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-03-16T09:18:28.364 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.364 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.364 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.364 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.364 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.364 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.364 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.364 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.365 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-03-16T09:18:28.365 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-16T09:18:28.365 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.365 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.365 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.365 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.365 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.365 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.365 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.365 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.365 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.365 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.365 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.365 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-16T09:18:28.366 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-16T09:18:28.366 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.366 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.366 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.366 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-03-16T09:18:28.366 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.366 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.366 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.366 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.366 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.366 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.366 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.367 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-03-16T09:18:28.367 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-16T09:18:28.367 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.367 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.367 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.367 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.367 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.367 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.367 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.367 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.367 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.367 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.367 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.367 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-03-16T09:18:28.368 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-16T09:18:28.368 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.368 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.368 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.368 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.368 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.368 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.368 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.368 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.368 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.368 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.368 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.368 INFO:teuthology.orchestra.run.smithi112.stdout: "UNSAFE_TLS_v1_2": { 2024-03-16T09:18:28.369 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-16T09:18:28.369 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.369 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.369 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.369 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.369 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.369 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.369 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.369 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.369 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.369 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.369 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.369 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-16T09:18:28.369 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-16T09:18:28.370 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.370 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.370 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.370 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-03-16T09:18:28.370 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.370 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.370 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.370 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.370 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.370 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.370 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.370 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-16T09:18:28.370 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-16T09:18:28.371 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.371 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.371 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.371 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:28.371 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.371 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.371 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.371 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.371 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.371 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.371 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.371 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-16T09:18:28.371 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-16T09:18:28.371 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.372 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.372 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.372 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-03-16T09:18:28.372 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.372 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.372 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.372 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.372 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.372 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.372 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.372 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-03-16T09:18:28.372 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-03-16T09:18:28.372 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.372 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.373 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.373 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.373 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.373 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.373 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.373 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.373 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.373 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.373 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.373 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-03-16T09:18:28.373 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-03-16T09:18:28.373 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.373 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.373 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.374 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.374 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.374 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.374 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.374 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.374 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.374 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.374 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.374 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-03-16T09:18:28.374 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-03-16T09:18:28.374 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.374 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.374 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.374 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.375 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.375 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.375 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.375 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-03-16T09:18:28.375 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.375 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.375 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.375 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-03-16T09:18:28.375 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-03-16T09:18:28.375 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.375 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.375 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.375 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-03-16T09:18:28.375 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.376 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.376 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.376 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.376 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.376 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.376 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.376 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-03-16T09:18:28.376 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-03-16T09:18:28.376 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.376 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.376 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.376 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.376 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.376 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.376 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.377 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.377 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.377 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.377 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.377 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.377 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.377 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.377 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.377 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.377 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.377 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.377 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.377 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.377 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.377 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.378 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.378 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.378 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.378 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.378 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.378 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.378 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.378 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.378 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.378 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.378 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.378 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.378 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.379 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.379 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.379 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.379 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.379 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.379 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.379 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.379 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.379 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.379 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.379 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.379 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.379 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.380 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.380 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.380 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.380 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.380 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.380 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.380 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.380 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.380 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.380 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.380 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.380 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.380 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.380 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.380 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.380 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.381 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.381 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.381 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.381 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.381 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.381 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.381 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.381 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.381 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.381 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.381 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.381 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.381 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.382 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-03-16T09:18:28.382 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-03-16T09:18:28.382 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.382 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.382 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.382 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.382 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.382 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.382 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.382 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-03-16T09:18:28.382 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.382 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.382 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.383 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-03-16T09:18:28.383 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-03-16T09:18:28.383 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.383 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.383 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.383 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.383 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.383 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.383 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.383 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.383 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.383 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.383 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.384 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-03-16T09:18:28.384 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-03-16T09:18:28.384 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.384 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.384 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.384 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-03-16T09:18:28.384 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.384 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.384 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.384 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.384 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.384 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.384 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.384 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-03-16T09:18:28.385 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-03-16T09:18:28.385 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.385 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.385 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.385 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-03-16T09:18:28.385 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.385 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.385 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.385 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.385 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.385 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.385 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.385 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-03-16T09:18:28.385 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-03-16T09:18:28.385 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.386 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.386 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.386 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.386 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.386 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.386 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.386 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.386 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.386 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.386 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.386 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-03-16T09:18:28.386 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-03-16T09:18:28.386 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.387 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.387 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.387 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-03-16T09:18:28.387 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.387 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.387 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.387 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.387 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.387 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.387 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.387 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-03-16T09:18:28.387 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-03-16T09:18:28.387 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.387 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.388 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.388 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-03-16T09:18:28.388 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.388 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.388 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.388 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.388 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-03-16T09:18:28.388 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.388 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.388 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.388 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.388 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.388 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-03-16T09:18:28.388 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-03-16T09:18:28.388 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.389 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.389 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.389 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-03-16T09:18:28.389 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-03-16T09:18:28.389 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-03-16T09:18:28.389 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.389 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.389 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.389 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.389 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.389 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-03-16T09:18:28.389 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-03-16T09:18:28.389 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.389 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.390 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.390 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.390 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.390 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.390 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.390 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.390 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.390 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.390 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.390 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.390 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.390 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.390 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-03-16T09:18:28.391 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.391 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.391 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.391 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-03-16T09:18:28.391 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-03-16T09:18:28.391 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.391 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.391 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.391 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.391 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.391 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.391 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.391 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-03-16T09:18:28.391 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.391 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.391 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.392 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.392 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.392 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.392 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.392 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.392 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.392 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.392 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.392 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.392 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.392 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.392 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.392 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.392 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.392 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.393 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.393 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.393 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.393 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.393 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.393 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.393 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.393 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.393 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.393 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.393 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.393 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.394 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.394 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.394 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.394 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.394 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.394 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.394 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.394 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.394 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.394 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.394 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.394 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.394 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.395 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.395 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.395 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.395 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.395 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.395 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.395 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.395 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.395 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.395 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.395 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.395 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.395 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.396 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.396 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.396 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.396 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.396 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.396 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.396 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.396 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.396 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.396 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.396 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.396 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.396 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.397 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-03-16T09:18:28.397 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-03-16T09:18:28.397 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.397 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.397 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.397 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-03-16T09:18:28.398 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.398 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.398 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.398 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-16T09:18:28.398 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.398 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.398 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.399 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-03-16T09:18:28.399 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-03-16T09:18:28.399 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.399 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.399 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.399 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-03-16T09:18:28.400 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.400 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.400 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.400 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-16T09:18:28.400 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.400 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.400 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.401 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-03-16T09:18:28.401 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-03-16T09:18:28.401 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.401 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.401 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.402 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-03-16T09:18:28.402 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.402 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.402 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.402 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-03-16T09:18:28.402 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.402 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.402 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.402 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-03-16T09:18:28.403 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-03-16T09:18:28.403 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.403 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.403 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.403 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-03-16T09:18:28.403 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.403 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.403 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.403 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-16T09:18:28.403 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.403 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.403 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.403 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-03-16T09:18:28.403 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-03-16T09:18:28.404 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.404 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.404 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.404 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.404 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.404 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.404 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.404 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-16T09:18:28.404 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.404 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.404 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.404 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-03-16T09:18:28.404 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-03-16T09:18:28.404 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.404 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.405 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.405 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-03-16T09:18:28.405 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.405 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.405 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.405 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-03-16T09:18:28.405 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.405 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.405 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.405 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-03-16T09:18:28.405 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-03-16T09:18:28.405 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.405 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.406 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.406 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-03-16T09:18:28.406 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.406 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.406 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.406 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-16T09:18:28.406 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.406 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.406 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.406 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.406 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.406 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.406 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-03-16T09:18:28.407 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.407 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.407 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.407 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.407 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.407 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.407 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.407 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.407 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.407 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.407 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.407 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.407 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.407 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.407 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.407 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.407 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.408 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.408 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.408 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.408 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.408 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.408 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.408 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.408 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.408 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.408 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.408 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.408 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.408 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.408 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.408 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.409 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.409 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.409 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.409 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.409 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.409 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.409 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.409 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.409 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.409 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.409 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.409 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.409 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.409 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.409 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.410 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.410 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.410 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.410 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.410 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.410 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.410 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.410 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.410 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.410 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.410 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.410 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.410 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.410 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.411 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.411 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.411 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.411 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.411 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.411 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.411 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.411 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.411 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-03-16T09:18:28.411 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-03-16T09:18:28.411 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.411 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.411 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.412 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-03-16T09:18:28.412 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.412 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.412 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.412 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.412 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.412 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.412 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.412 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-03-16T09:18:28.412 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-03-16T09:18:28.412 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.412 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.412 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.413 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-03-16T09:18:28.413 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.413 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.413 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.413 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.413 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.413 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.413 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.413 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-03-16T09:18:28.413 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-03-16T09:18:28.413 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.414 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.414 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.414 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-03-16T09:18:28.414 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.414 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.414 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.414 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.414 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.414 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.414 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.414 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.414 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.415 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.415 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-03-16T09:18:28.415 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-03-16T09:18:28.415 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-03-16T09:18:28.415 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.415 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-03-16T09:18:28.415 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-03-16T09:18:28.415 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.415 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.415 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.415 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-03-16T09:18:28.415 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.415 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.415 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.416 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-16T09:18:28.416 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.416 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.416 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.416 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-03-16T09:18:28.416 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-03-16T09:18:28.416 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.416 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.416 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.416 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-03-16T09:18:28.416 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.416 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.416 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.417 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-16T09:18:28.417 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.417 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.417 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.417 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-03-16T09:18:28.417 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-03-16T09:18:28.417 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.417 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.417 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.417 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.417 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.417 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.417 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.418 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-03-16T09:18:28.418 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.418 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.418 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.418 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-03-16T09:18:28.418 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-03-16T09:18:28.418 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.418 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.418 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.418 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-03-16T09:18:28.418 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-03-16T09:18:28.418 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.419 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.419 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-16T09:18:28.419 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.419 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.419 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.419 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.419 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.419 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.419 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.419 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.419 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.419 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.419 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.419 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.419 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.420 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.420 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.420 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.420 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.420 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.420 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.420 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.420 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.420 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.420 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.420 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.420 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.420 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.421 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.421 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.421 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.421 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.421 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.421 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.421 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.421 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.421 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.421 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.421 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.421 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.421 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.421 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.422 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.422 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.422 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.422 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.422 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.422 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.422 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.422 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.422 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.422 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.422 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.422 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.422 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.422 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.423 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.423 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.423 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.423 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.423 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.423 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.423 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.423 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.423 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.423 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.423 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.423 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.423 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.424 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.424 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.424 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-03-16T09:18:28.424 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-03-16T09:18:28.424 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.424 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.424 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.424 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.424 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.424 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.424 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.424 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-03-16T09:18:28.424 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.424 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.425 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.425 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-03-16T09:18:28.425 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-03-16T09:18:28.425 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.425 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.425 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.425 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-03-16T09:18:28.425 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.425 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.425 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.425 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-03-16T09:18:28.425 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.425 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.426 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.426 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-03-16T09:18:28.426 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-03-16T09:18:28.426 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.426 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.426 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.426 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-03-16T09:18:28.426 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.426 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.426 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.426 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-16T09:18:28.426 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.426 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.427 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.427 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-03-16T09:18:28.427 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-03-16T09:18:28.427 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.427 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.427 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.427 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-03-16T09:18:28.427 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-03-16T09:18:28.427 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-03-16T09:18:28.427 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.427 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-16T09:18:28.427 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.427 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.427 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.428 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-03-16T09:18:28.428 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-03-16T09:18:28.428 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.428 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.428 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.428 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.428 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.428 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.428 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.428 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-03-16T09:18:28.428 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.428 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.428 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.429 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-03-16T09:18:28.429 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-03-16T09:18:28.429 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.429 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.429 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.429 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-03-16T09:18:28.429 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.429 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.429 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.429 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-16T09:18:28.429 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.429 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.429 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.429 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.430 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.430 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.430 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-03-16T09:18:28.430 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.430 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.430 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.430 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.430 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.430 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.430 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.430 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.430 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.430 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.431 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.431 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.431 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.431 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.431 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.431 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.431 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.431 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.431 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.431 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.431 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.431 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.431 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.431 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.432 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.432 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.432 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.432 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.432 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.432 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.432 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.432 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.432 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.432 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.432 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.432 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.433 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.433 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.433 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.433 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.433 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.433 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.433 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.433 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.433 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.433 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.433 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.433 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.433 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.433 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.434 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.434 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.434 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.434 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.434 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.434 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.434 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.434 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.434 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.434 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.434 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.434 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.435 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.435 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.435 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.435 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.435 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.435 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.435 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.435 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.435 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.435 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.435 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-03-16T09:18:28.435 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.435 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.435 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.436 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.436 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.436 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.436 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.436 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.436 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.436 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.436 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.436 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.436 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.436 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.436 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.436 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.436 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.437 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.437 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.437 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.437 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.437 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.437 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.437 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.437 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.437 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.437 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.437 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.437 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.437 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.437 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.437 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.438 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.438 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.438 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.438 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.438 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.438 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.438 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.438 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.438 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.438 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.438 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.438 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.439 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.439 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.439 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.439 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.439 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.439 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.439 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.439 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.439 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.439 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.439 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.439 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.439 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.439 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.440 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.440 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.440 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.440 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.440 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.440 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.440 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.440 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.440 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.440 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.440 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.440 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.440 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.440 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.440 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-03-16T09:18:28.440 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.441 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.441 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.441 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-03-16T09:18:28.441 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-03-16T09:18:28.441 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.441 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.441 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.441 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-03-16T09:18:28.441 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.441 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.441 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.441 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-16T09:18:28.441 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.441 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.441 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.442 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-03-16T09:18:28.442 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-03-16T09:18:28.442 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.442 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.442 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.442 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:28.442 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-03-16T09:18:28.442 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.442 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.442 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-16T09:18:28.442 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.442 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.442 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.443 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.443 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.443 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.443 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.443 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.443 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.443 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.443 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.443 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.443 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.443 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.443 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.443 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.443 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.444 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.444 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.444 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.444 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.444 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.444 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.444 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.444 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.444 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.444 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.444 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.444 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.444 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.444 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.445 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.445 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.445 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.445 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.445 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.445 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.445 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.445 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.445 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.445 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.445 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.445 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.445 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.445 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.446 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.446 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.446 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.446 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.446 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.446 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.446 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.446 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.446 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.446 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.446 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.446 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.446 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.447 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.447 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.447 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.447 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.447 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.447 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.447 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.447 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.447 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.447 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.447 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.447 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.447 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.447 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.447 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-03-16T09:18:28.448 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.448 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.448 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.448 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-03-16T09:18:28.448 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-03-16T09:18:28.448 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.448 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.448 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.448 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-03-16T09:18:28.448 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.448 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.448 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.448 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-03-16T09:18:28.448 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-16T09:18:28.448 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.448 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.449 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.449 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.449 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.449 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.449 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.449 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.449 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.449 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.449 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.449 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.449 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.449 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.449 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.449 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.450 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.450 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.450 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.450 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.450 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.450 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.450 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.450 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.450 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.450 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.450 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.450 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.450 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.451 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.451 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.451 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.451 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.451 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.451 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.451 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.451 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.451 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.451 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.451 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.451 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.451 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.451 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.452 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.452 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.452 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.452 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.452 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.452 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.452 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.452 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.452 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.452 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.452 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.452 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.452 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.452 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.452 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.453 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.453 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.453 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.453 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.453 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.453 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.453 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.453 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.453 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.453 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.453 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-03-16T09:18:28.453 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-03-16T09:18:28.453 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.454 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.454 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.454 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.454 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.454 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.454 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.454 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-03-16T09:18:28.454 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-16T09:18:28.454 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.454 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.454 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-03-16T09:18:28.454 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-03-16T09:18:28.454 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.455 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.455 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.455 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-03-16T09:18:28.455 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.455 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.455 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.455 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-03-16T09:18:28.455 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.455 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.455 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.455 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-03-16T09:18:28.455 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-03-16T09:18:28.456 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.456 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.456 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.456 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-03-16T09:18:28.456 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.456 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.456 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.456 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-03-16T09:18:28.456 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.456 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.456 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.456 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-03-16T09:18:28.456 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-03-16T09:18:28.456 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.456 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.457 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.457 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.457 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.457 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.457 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.457 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-03-16T09:18:28.457 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.457 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.457 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.457 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-03-16T09:18:28.457 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-03-16T09:18:28.457 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.457 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.457 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.457 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-03-16T09:18:28.458 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.458 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.458 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.458 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-16T09:18:28.458 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-16T09:18:28.458 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.458 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.458 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.458 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.458 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.458 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-03-16T09:18:28.458 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.458 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.459 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.459 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.459 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.459 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.459 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.459 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.459 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.459 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.459 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.459 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.459 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.459 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.459 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.459 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.459 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.459 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.459 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.460 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.460 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.460 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.460 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.460 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.460 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.460 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.460 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.460 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.460 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.460 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.460 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.460 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.460 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.461 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.461 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.461 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.461 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.461 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.461 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.461 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.461 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.461 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.461 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.461 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.461 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.461 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.461 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.462 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.462 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.462 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.462 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.462 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.462 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.462 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.462 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.462 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.462 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.462 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.462 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.462 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.463 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.463 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.463 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.463 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.463 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.463 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.463 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.463 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.463 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.463 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.463 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.463 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.463 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-03-16T09:18:28.463 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.463 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.464 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.464 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.464 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.464 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.464 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.464 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.464 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.464 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.464 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.464 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.464 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.464 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.464 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.464 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.465 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.465 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.465 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.465 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.465 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.465 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.465 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.465 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.465 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.465 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.465 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.465 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.465 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.465 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.465 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.466 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.466 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.466 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.466 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.466 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.466 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.466 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.466 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.466 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.466 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.466 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.466 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.467 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.467 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.467 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.467 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.467 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.467 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.467 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.467 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.467 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.467 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.467 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.467 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.467 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.467 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.467 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.468 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.468 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.468 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.468 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.468 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.468 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.468 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.468 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.468 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.468 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.468 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.468 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.468 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.468 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.468 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-03-16T09:18:28.468 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.469 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.469 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.469 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.469 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.469 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.469 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.469 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.469 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.469 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.469 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.469 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.469 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.469 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.469 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.469 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.470 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.470 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.470 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.470 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.470 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.470 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.470 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.470 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.470 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.470 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.470 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.470 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.470 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.470 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.471 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.471 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.471 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.471 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.471 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.471 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.471 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.471 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.471 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.471 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.471 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.471 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.471 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.471 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.471 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.472 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.472 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.472 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.472 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.472 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.472 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.472 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.472 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.472 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.472 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.472 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.472 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.472 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.472 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.472 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.473 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.473 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.473 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.473 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.473 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.473 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.473 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.473 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.473 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.473 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.473 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.473 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.473 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-03-16T09:18:28.473 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.474 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.474 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.474 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-03-16T09:18:28.474 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-03-16T09:18:28.474 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.474 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.474 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.474 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.474 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.474 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.474 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.474 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-16T09:18:28.474 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.475 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.475 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.475 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.475 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.475 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.475 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.475 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.475 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.475 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.475 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.475 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.475 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.475 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.475 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.475 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.476 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.476 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.476 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.476 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.476 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.476 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.476 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.476 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.476 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.476 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.476 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.476 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.476 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.476 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.477 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.477 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.477 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.477 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.477 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.477 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.477 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.477 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.477 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.477 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.477 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.477 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.477 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.478 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.478 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.478 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.478 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.478 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.478 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.478 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.478 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.478 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.478 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.478 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.478 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.478 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.479 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.479 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.479 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.479 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.479 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.479 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.479 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.479 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.479 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.479 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.479 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.479 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.479 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.480 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-03-16T09:18:28.480 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-03-16T09:18:28.480 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.480 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.480 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.480 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.480 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.480 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.480 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.480 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-03-16T09:18:28.480 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-03-16T09:18:28.480 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-03-16T09:18:28.480 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.480 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-03-16T09:18:28.481 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.481 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.481 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.481 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.481 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.481 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.481 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-03-16T09:18:28.481 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.481 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.481 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.481 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.481 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.481 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.481 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.482 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.482 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.482 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.482 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.482 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.482 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.482 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.482 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.482 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.482 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.482 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.482 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.482 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.483 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.483 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.483 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.483 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.483 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.483 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.483 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.483 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.483 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.483 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.483 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.483 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.483 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.484 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.484 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.484 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.484 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.484 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.484 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.484 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.484 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.484 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.484 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.484 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.484 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.484 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.485 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.485 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.485 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.485 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.485 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.485 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.485 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.485 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.485 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.485 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.485 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.485 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.485 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.486 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.486 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.486 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.486 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.486 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.486 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.486 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.486 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.486 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.486 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.486 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.486 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.486 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.487 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-03-16T09:18:28.487 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.487 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.487 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.487 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.487 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.487 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.487 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.487 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.487 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.487 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.487 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.487 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.487 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.487 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.488 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.488 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.488 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.488 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.488 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.488 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.488 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.488 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.488 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.488 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.488 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.488 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.488 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.488 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.489 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.489 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.489 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.489 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.489 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.489 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.489 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.489 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.489 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.489 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.489 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.489 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.489 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.489 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.490 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.490 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.490 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.490 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.490 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.490 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.490 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.490 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.490 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.490 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.490 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.490 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.490 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.490 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.491 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.491 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.491 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.491 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.491 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.491 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.491 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.491 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.491 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.491 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.491 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.491 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.491 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.492 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.492 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.492 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.492 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-03-16T09:18:28.492 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.492 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.492 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.492 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.492 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.492 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.492 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.492 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.492 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.492 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.493 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.493 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.493 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.493 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.493 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.493 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.493 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.493 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.493 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.493 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.493 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.493 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.493 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.493 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.494 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.494 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.494 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.494 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.494 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.494 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.494 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.494 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.494 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.494 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.494 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.494 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.494 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.495 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.495 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.495 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.495 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.495 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.495 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.495 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.495 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.495 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.495 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.495 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.495 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.495 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.495 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.496 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.496 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.496 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.496 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.496 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.496 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.496 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.496 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.496 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.496 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.496 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.496 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.497 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.497 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.497 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.497 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.497 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.497 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.497 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.497 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-03-16T09:18:28.497 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-03-16T09:18:28.497 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.497 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.497 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.497 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-03-16T09:18:28.497 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.498 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.498 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.498 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.498 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.498 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.498 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.498 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-03-16T09:18:28.498 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-03-16T09:18:28.498 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:28.498 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.498 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.498 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-03-16T09:18:28.498 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-03-16T09:18:28.499 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.499 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.499 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-03-16T09:18:28.499 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-16T09:18:28.499 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.499 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.499 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.499 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.499 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.499 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-03-16T09:18:28.499 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.499 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.499 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.500 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-03-16T09:18:28.500 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-03-16T09:18:28.500 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.500 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.500 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.500 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.500 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.500 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.500 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.500 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-16T09:18:28.500 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.500 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.500 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.500 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-03-16T09:18:28.501 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-03-16T09:18:28.501 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.501 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.501 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.501 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.501 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.501 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.501 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.501 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.501 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.501 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.501 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.501 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.502 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.502 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.502 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.502 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.502 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.502 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.502 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.502 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.502 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.502 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.502 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.502 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.502 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.502 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.502 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.503 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.503 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.503 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.503 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.503 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.503 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.503 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.503 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.503 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.503 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.503 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.503 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.503 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.503 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.504 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.504 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.504 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.504 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.504 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.504 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.504 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.504 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.504 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.504 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.504 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.504 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.504 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.504 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.505 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.505 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.505 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.505 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.505 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.505 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.505 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.505 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.505 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.505 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.505 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.505 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.505 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.505 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.505 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.506 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.506 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.506 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.506 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.506 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.506 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.506 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.506 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-03-16T09:18:28.506 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-03-16T09:18:28.506 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.506 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.506 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.506 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-03-16T09:18:28.506 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.507 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.507 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.507 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-03-16T09:18:28.507 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.507 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.507 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.507 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-03-16T09:18:28.507 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-03-16T09:18:28.507 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.507 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.507 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.507 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-03-16T09:18:28.507 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.508 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.508 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.508 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-03-16T09:18:28.508 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.508 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.508 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.508 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.508 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.508 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.508 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-03-16T09:18:28.508 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.508 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.508 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.509 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-03-16T09:18:28.509 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-03-16T09:18:28.509 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.509 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.509 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.509 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.509 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.509 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.509 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.509 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.509 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.509 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.509 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.510 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-03-16T09:18:28.510 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-03-16T09:18:28.510 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.510 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.510 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.510 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.510 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.510 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.510 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.510 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-16T09:18:28.510 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-16T09:18:28.510 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.510 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.511 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.511 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.511 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.511 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.511 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.511 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.511 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.511 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.511 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.511 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.511 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.511 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.511 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.511 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.511 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.512 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.512 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.512 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.512 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.512 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.512 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.512 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.512 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.512 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.512 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.512 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.512 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.512 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.513 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.513 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.513 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.513 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.513 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.513 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.513 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.513 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.513 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.513 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.513 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.513 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.514 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.514 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.514 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.514 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.514 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.514 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.514 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.514 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.514 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.514 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.514 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.514 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.514 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.515 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.515 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.515 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.515 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.515 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.515 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.515 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.515 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.515 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.515 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.515 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.515 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.515 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.516 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-03-16T09:18:28.516 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-03-16T09:18:28.516 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.516 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.516 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.516 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.516 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.516 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.516 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.516 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.516 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.516 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.516 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.517 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-16T09:18:28.517 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-16T09:18:28.517 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.517 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.517 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.517 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-03-16T09:18:28.517 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.517 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.517 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.517 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.517 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.517 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.517 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.518 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-03-16T09:18:28.518 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-03-16T09:18:28.518 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:28.518 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.518 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.518 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-03-16T09:18:28.518 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.518 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.518 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.518 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.518 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.518 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.518 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-03-16T09:18:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-03-16T09:18:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-03-16T09:18:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-16T09:18:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.519 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-03-16T09:18:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-03-16T09:18:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-03-16T09:18:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-16T09:18:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-03-16T09:18:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-03-16T09:18:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-03-16T09:18:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-03-16T09:18:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-03-16T09:18:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-03-16T09:18:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-03-16T09:18:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-03-16T09:18:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.522 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-03-16T09:18:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-03-16T09:18:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-03-16T09:18:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-03-16T09:18:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-03-16T09:18:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.523 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-03-16T09:18:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.524 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.525 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.526 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.527 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.527 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.527 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.527 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.527 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.527 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.527 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.527 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.527 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.527 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.527 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.527 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.527 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.528 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.528 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.528 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.528 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.528 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.528 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.528 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.528 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.528 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.528 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.528 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.528 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.528 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.529 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.529 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.529 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.529 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.529 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.529 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.529 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.529 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.529 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.529 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.529 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-03-16T09:18:28.529 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-03-16T09:18:28.530 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.530 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.530 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.530 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-03-16T09:18:28.530 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.530 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.530 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.530 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.530 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.530 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.530 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.530 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-03-16T09:18:28.530 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-03-16T09:18:28.531 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.531 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.531 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.531 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.531 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.531 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.531 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.531 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.531 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.531 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.531 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.531 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-03-16T09:18:28.531 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-03-16T09:18:28.532 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.532 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.532 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.532 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.532 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.532 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.532 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.532 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.532 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.532 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.532 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.532 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.532 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.533 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.533 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-03-16T09:18:28.533 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.533 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.533 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.533 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-03-16T09:18:28.533 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-03-16T09:18:28.533 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.533 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.533 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.533 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.533 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.533 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.533 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.534 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.534 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.534 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.534 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.534 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-03-16T09:18:28.534 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-03-16T09:18:28.534 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.534 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.534 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.534 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.534 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.534 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.534 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.535 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.535 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.535 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.535 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.535 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.535 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.535 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.535 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.535 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.535 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.535 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.535 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.535 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.535 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.536 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.536 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.536 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.536 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.536 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.536 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.536 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.536 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.536 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.536 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.536 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.536 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.536 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.537 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.537 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.537 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.537 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.537 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.537 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.537 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.537 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.537 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.537 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.537 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.537 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.537 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.538 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.538 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.538 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.538 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.538 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.538 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.538 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.538 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.538 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.538 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.538 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.538 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.538 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.538 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.539 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.539 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.539 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.539 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.539 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.539 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.539 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.539 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.539 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.539 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.539 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.539 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.539 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.539 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.540 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.540 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.540 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-03-16T09:18:28.540 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-03-16T09:18:28.540 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.540 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.540 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.540 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.540 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.540 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.540 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.540 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.540 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.540 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.541 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.541 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-03-16T09:18:28.541 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-03-16T09:18:28.541 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.541 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.541 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.541 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.541 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.541 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.541 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.541 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.541 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.541 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.542 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.542 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.542 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.542 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.542 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-03-16T09:18:28.542 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.542 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.542 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.542 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.542 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.542 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.542 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.542 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.543 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.543 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.543 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.543 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.543 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.543 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.543 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.543 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.543 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.543 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.543 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.543 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.543 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.544 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.544 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.544 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.544 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.544 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.544 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.544 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.544 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.544 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.544 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.544 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.544 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.544 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.545 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.545 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.545 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.545 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.545 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.545 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.545 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.545 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.545 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.545 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.545 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.545 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.545 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.545 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.546 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.546 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.546 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.546 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.546 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.546 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.546 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.546 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.546 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.546 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.546 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.546 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.546 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.547 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.547 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.547 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.547 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.547 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.547 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.547 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.547 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.547 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.547 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.547 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.547 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-03-16T09:18:28.547 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.547 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.548 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.548 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.548 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.548 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.548 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.548 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.548 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.548 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.548 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.548 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.548 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.548 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.548 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.548 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.548 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.549 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.549 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.549 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.549 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.549 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.549 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.549 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.549 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.549 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.549 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.549 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.549 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.549 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.549 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.550 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.550 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.550 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.550 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.550 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.550 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.550 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.550 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.550 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.550 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.550 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.550 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.550 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.551 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.551 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.551 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.551 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.551 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.551 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.551 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.551 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.551 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.551 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.551 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.551 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.551 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.551 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.552 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.552 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.552 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.552 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.552 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.552 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.552 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.552 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.552 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.552 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.552 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.552 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-03-16T09:18:28.552 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-03-16T09:18:28.552 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.552 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.553 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.553 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-03-16T09:18:28.553 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.553 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.553 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.553 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-16T09:18:28.553 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.553 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.553 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.553 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.553 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.553 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.553 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-03-16T09:18:28.553 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.553 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.553 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.554 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.554 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.554 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.554 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.554 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.554 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.554 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.554 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.554 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.554 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.554 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.554 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.554 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.554 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.555 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.555 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.555 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.555 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.555 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.555 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.555 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.555 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.555 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.555 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.555 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.555 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.555 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.555 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.555 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.556 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.556 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.556 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.556 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.556 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.556 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.556 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.556 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.556 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.556 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.556 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.556 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.556 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.556 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.556 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.557 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.557 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.557 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.557 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.557 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.557 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.557 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.557 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.557 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.557 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.557 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.557 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.557 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.557 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.557 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.557 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.558 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.558 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.558 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.558 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.558 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.558 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.558 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-03-16T09:18:28.558 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-03-16T09:18:28.558 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.558 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.558 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.558 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.558 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.558 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.559 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.559 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.559 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.559 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.559 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.559 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-03-16T09:18:28.559 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-03-16T09:18:28.559 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.559 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.559 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.559 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-03-16T09:18:28.559 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.559 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.560 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.560 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.560 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.560 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.560 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.560 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.560 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-03-16T09:18:28.560 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-03-16T09:18:28.560 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.560 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.560 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.560 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.560 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.560 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.560 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.561 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.561 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.561 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.561 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.561 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-03-16T09:18:28.561 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-03-16T09:18:28.561 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.561 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.561 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.561 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.561 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.561 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.561 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.561 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.562 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.562 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.562 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.562 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.562 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-03-16T09:18:28.562 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-03-16T09:18:28.562 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-03-16T09:18:28.562 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.562 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.562 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.562 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.562 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.562 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.563 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.563 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.563 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.563 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.563 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-03-16T09:18:28.563 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-03-16T09:18:28.563 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.563 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.563 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.563 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.563 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.563 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.563 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.564 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.564 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.564 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.564 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.564 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-03-16T09:18:28.564 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-03-16T09:18:28.564 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.564 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.564 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.564 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.564 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.564 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.565 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.565 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.565 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.565 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.565 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.565 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-03-16T09:18:28.565 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-03-16T09:18:28.565 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.565 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.565 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.565 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.565 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.565 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.565 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.566 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.566 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.566 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.566 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.566 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.566 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-03-16T09:18:28.566 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-03-16T09:18:28.566 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.566 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.566 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.566 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.566 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-03-16T09:18:28.566 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-03-16T09:18:28.566 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.567 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.567 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.567 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.567 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.567 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-03-16T09:18:28.567 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-03-16T09:18:28.567 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.567 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.567 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.567 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.567 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.567 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.567 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.568 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.568 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.568 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.568 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.568 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-03-16T09:18:28.568 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-03-16T09:18:28.568 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.568 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.568 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.568 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.568 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.568 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.568 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.569 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.569 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.569 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.569 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.569 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-03-16T09:18:28.569 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-03-16T09:18:28.569 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.569 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.569 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.569 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.569 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.569 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.569 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.570 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.570 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.570 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.570 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.570 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.570 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.570 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.570 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-03-16T09:18:28.570 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.570 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.570 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.570 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-03-16T09:18:28.570 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-03-16T09:18:28.570 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.571 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.571 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.571 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.571 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.571 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.571 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.571 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-03-16T09:18:28.571 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.571 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.571 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.571 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-03-16T09:18:28.571 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-03-16T09:18:28.571 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.572 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.572 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.572 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.572 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.572 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.572 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.572 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-03-16T09:18:28.572 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.572 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.572 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.572 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.572 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.572 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.573 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.573 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.573 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.573 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.573 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.573 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.573 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.573 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.573 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.573 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.573 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.573 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.573 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.574 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.574 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.574 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.574 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.574 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.574 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.574 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.574 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.575 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.575 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.575 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.575 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.575 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.575 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.575 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.575 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.575 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.575 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.575 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.575 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.575 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.575 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.576 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.576 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.576 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.576 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.576 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.576 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.576 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.576 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.576 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.576 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.576 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.576 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.576 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.576 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.576 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.577 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.577 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.577 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.577 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.577 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.577 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.577 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.577 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.577 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.577 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.577 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.577 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.577 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.577 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.577 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.578 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.578 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.578 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.578 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-03-16T09:18:28.578 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.578 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.578 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.578 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.578 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.578 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.578 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.578 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.578 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.578 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.579 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.579 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.579 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.579 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.579 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.579 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.579 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.579 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.579 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.579 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.579 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.579 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.579 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.579 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.580 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.580 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.580 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.580 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.580 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.580 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.580 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.580 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.580 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.580 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.580 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.580 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.580 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.580 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.581 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.581 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.581 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.581 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.581 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.581 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.581 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.581 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.581 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.581 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.581 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.581 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.581 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.581 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.581 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.581 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.582 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.582 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.582 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.582 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.582 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.582 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.582 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.582 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.582 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.582 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.582 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.582 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.582 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.582 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.583 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.583 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.583 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.583 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.583 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-03-16T09:18:28.583 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.583 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.583 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.583 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.583 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.583 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.583 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.584 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.584 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.584 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.584 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.584 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.584 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.584 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.584 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.584 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.584 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.584 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.584 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.584 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.584 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.584 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.585 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.585 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.585 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.585 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.585 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.585 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.585 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.585 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.585 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.585 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.585 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.585 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.585 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.586 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.586 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.586 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.586 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.586 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.586 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.586 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.586 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.586 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.586 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.586 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.586 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.586 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.587 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.587 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.587 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.587 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.587 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.587 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.587 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.587 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.587 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.587 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.587 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.587 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.588 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.588 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.588 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.588 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.588 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.588 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.588 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.588 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.588 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.588 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.588 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.588 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.589 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.589 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-03-16T09:18:28.589 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.589 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.589 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.589 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-03-16T09:18:28.589 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-03-16T09:18:28.589 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.589 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.589 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.589 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-16T09:18:28.589 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.589 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.589 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.589 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.590 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.590 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.590 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.590 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.590 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-03-16T09:18:28.590 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-03-16T09:18:28.590 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.590 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.590 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.590 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-03-16T09:18:28.590 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.590 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.590 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.590 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.590 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.591 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.591 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.591 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.591 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.591 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.591 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.591 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.591 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.591 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.591 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.591 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.591 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.591 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.592 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.592 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.592 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.592 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.592 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.592 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.592 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.592 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.592 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.592 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.592 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.592 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.592 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.592 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.593 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.593 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.593 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.593 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.593 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.593 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.593 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.593 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.593 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.593 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.593 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.593 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.593 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.593 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.594 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.594 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.594 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.594 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.594 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.594 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.594 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.594 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.594 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.594 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.594 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.594 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.594 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.594 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.595 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.595 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.595 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.595 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.595 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.595 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.595 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.595 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.595 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.595 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.595 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.595 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.595 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.595 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.596 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.596 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.596 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-03-16T09:18:28.596 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.596 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.596 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.596 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-03-16T09:18:28.596 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-03-16T09:18:28.596 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.596 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.596 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.596 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.596 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.597 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.597 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.597 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-16T09:18:28.597 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.597 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.597 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.597 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-03-16T09:18:28.597 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-03-16T09:18:28.597 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.597 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.597 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.597 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.597 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.597 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.598 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.598 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-16T09:18:28.598 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.598 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.598 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.598 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.598 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-03-16T09:18:28.598 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-03-16T09:18:28.598 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.598 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.598 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.598 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.599 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.599 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.599 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.599 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-16T09:18:28.599 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.599 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.599 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.599 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.599 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-03-16T09:18:28.599 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-03-16T09:18:28.599 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.600 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.600 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.600 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.600 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.600 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.600 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.600 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-16T09:18:28.600 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.600 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.600 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.601 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.601 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-03-16T09:18:28.601 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-03-16T09:18:28.601 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.601 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.601 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.601 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.601 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.602 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.602 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.602 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-16T09:18:28.602 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.602 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.602 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.602 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.602 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-03-16T09:18:28.603 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-03-16T09:18:28.603 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.603 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.603 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.603 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.603 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.603 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.604 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.604 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.604 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.604 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.604 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.604 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-03-16T09:18:28.604 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-03-16T09:18:28.604 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.604 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.604 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.604 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.605 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.605 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.605 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.605 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.605 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.605 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.605 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.605 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-03-16T09:18:28.605 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-03-16T09:18:28.605 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.605 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.605 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.605 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-16T09:18:28.605 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.606 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.606 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.606 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.606 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.606 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.606 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.606 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-03-16T09:18:28.606 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-03-16T09:18:28.606 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.606 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.606 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.606 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.607 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.607 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.607 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.607 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.607 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.607 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.607 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.607 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-03-16T09:18:28.607 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-03-16T09:18:28.607 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.607 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.607 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.607 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-03-16T09:18:28.607 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-03-16T09:18:28.608 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.608 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.608 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.608 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.608 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.608 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.608 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-03-16T09:18:28.608 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-03-16T09:18:28.608 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.608 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.608 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.608 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-03-16T09:18:28.608 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.609 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.609 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.609 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.609 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.609 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.609 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.609 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-03-16T09:18:28.609 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-03-16T09:18:28.609 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.609 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.609 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.609 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.609 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.609 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.610 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.610 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.610 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.610 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.610 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.610 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.610 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-03-16T09:18:28.610 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-03-16T09:18:28.610 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.610 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.610 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.610 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.610 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.610 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.611 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.611 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.611 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.611 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.611 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.611 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.611 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.611 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.611 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.611 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.611 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.611 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.611 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.611 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.612 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.612 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.612 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.612 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.612 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.612 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.612 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.612 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.612 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.612 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.612 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.612 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.612 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.613 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.613 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.613 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.613 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.613 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.613 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.613 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.613 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.613 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.613 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.613 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.613 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.613 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.613 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.613 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.614 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.614 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.614 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.614 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.614 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.614 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.614 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.614 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.614 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.614 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.614 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.614 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.614 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.614 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.615 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.615 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.615 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.615 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.615 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.615 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.615 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.615 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.615 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.615 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.615 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.615 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.615 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.616 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.616 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.616 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-03-16T09:18:28.616 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-03-16T09:18:28.616 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.616 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.616 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.616 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.616 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.616 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.616 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.616 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.616 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.617 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.617 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.617 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-03-16T09:18:28.617 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-03-16T09:18:28.617 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.617 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.617 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.617 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.617 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.617 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.617 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.617 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.617 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.618 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.618 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.618 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-03-16T09:18:28.618 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-03-16T09:18:28.618 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.618 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.618 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.618 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-16T09:18:28.618 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.618 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.618 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.618 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.618 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.619 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.619 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.619 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.619 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.619 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-03-16T09:18:28.619 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.619 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.619 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.619 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.619 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.619 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.619 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.620 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.620 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.620 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.620 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.620 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.620 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.620 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.620 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.620 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.620 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.620 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.620 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.620 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.620 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.621 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.621 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.621 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.621 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.621 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.621 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.621 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.621 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.621 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.621 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.621 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.621 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.621 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.622 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.622 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.622 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.622 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.622 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.622 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.622 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.622 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.622 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.622 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.622 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.622 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.622 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.623 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.623 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.623 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.623 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.623 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.623 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.623 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.623 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.623 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.623 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.623 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.623 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.623 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.624 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.624 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.624 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.624 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.624 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.624 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.624 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.624 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.624 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.624 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.624 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.624 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.624 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.625 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.625 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-03-16T09:18:28.625 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.625 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.625 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.625 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.625 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.625 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.625 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.625 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.625 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.625 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.625 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.625 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.625 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.625 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.626 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.626 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.626 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.626 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.626 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.626 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.626 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.626 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.626 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.626 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.626 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.626 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.627 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.627 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.627 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.627 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.627 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.627 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.627 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.627 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.627 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.627 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.627 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.627 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.627 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.627 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.628 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.628 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.628 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.628 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.628 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.628 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.628 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.628 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.628 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.628 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.628 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.628 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.628 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.628 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.628 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.628 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.629 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.629 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.629 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.629 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.629 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.629 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.629 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.629 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.629 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.629 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.629 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.629 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.630 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.630 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-03-16T09:18:28.630 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-03-16T09:18:28.630 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.630 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.630 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.630 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-03-16T09:18:28.630 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.630 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.630 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.630 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-16T09:18:28.630 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.631 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.631 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.631 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-03-16T09:18:28.631 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-03-16T09:18:28.631 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.631 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.631 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.631 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.631 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.631 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.631 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.631 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-03-16T09:18:28.631 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.631 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.632 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.632 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-03-16T09:18:28.632 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-03-16T09:18:28.632 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.632 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.632 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.632 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-03-16T09:18:28.632 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.632 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.632 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.632 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-16T09:18:28.632 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.632 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.633 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.633 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-03-16T09:18:28.633 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-03-16T09:18:28.633 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.633 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.633 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.633 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-03-16T09:18:28.633 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.633 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.633 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.633 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-16T09:18:28.633 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.633 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.634 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.634 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.634 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.634 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.634 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.634 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-03-16T09:18:28.634 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-03-16T09:18:28.634 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-03-16T09:18:28.634 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-03-16T09:18:28.634 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-03-16T09:18:28.634 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-03-16T09:18:28.634 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-03-16T09:18:28.634 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.634 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.635 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-03-16T09:18:28.635 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.635 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.635 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.635 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.635 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.635 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.635 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.635 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.635 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-03-16T09:18:28.635 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-03-16T09:18:28.635 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.635 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.635 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.635 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.636 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.636 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.636 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.636 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.636 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.636 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.636 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.636 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-03-16T09:18:28.636 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-03-16T09:18:28.636 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-03-16T09:18:28.636 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.636 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.636 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-03-16T09:18:28.636 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.636 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.637 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.637 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.637 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.637 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.637 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.637 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-03-16T09:18:28.637 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-03-16T09:18:28.637 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.637 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.637 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.637 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.637 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.637 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.637 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.638 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.638 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.638 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.638 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.638 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.638 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.638 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.638 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.638 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.638 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.638 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.638 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-03-16T09:18:28.639 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-03-16T09:18:28.639 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.639 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.639 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.639 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.639 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.639 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.639 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.639 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.639 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.639 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.639 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.639 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-03-16T09:18:28.639 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-03-16T09:18:28.639 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.640 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.640 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.640 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-03-16T09:18:28.640 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.640 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.640 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-03-16T09:18:28.640 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-03-16T09:18:28.640 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-03-16T09:18:28.640 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-03-16T09:18:28.640 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-03-16T09:18:28.640 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-03-16T09:18:28.640 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-03-16T09:18:28.640 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.640 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.640 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.640 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.641 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.641 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-03-16T09:18:28.641 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-03-16T09:18:28.641 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-03-16T09:18:28.641 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.641 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-03-16T09:18:28.641 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-03-16T09:18:28.641 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.641 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.641 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.641 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.641 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.641 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.641 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.642 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-03-16T09:18:28.642 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-03-16T09:18:28.642 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.642 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.642 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.642 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-03-16T09:18:28.642 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.642 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.642 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.642 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.642 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.642 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.642 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.643 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-03-16T09:18:28.643 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-03-16T09:18:28.643 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-03-16T09:18:28.643 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.643 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.643 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-03-16T09:18:28.643 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.643 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.643 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.643 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.643 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.643 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.643 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.643 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.644 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-03-16T09:18:28.644 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-03-16T09:18:28.644 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-03-16T09:18:28.644 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-03-16T09:18:28.644 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-03-16T09:18:28.644 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-16T09:18:28.644 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-03-16T09:18:28.644 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-03-16T09:18:28.644 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-03-16T09:18:28.644 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-03-16T09:18:28.644 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-03-16T09:18:28.644 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-03-16T09:18:28.644 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-03-16T09:18:28.644 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.645 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.645 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.645 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.645 INFO:teuthology.orchestra.run.smithi112.stdout: "services": { 2024-03-16T09:18:28.645 INFO:teuthology.orchestra.run.smithi112.stdout: "dashboard": "https://172.21.15.112:8443/" 2024-03-16T09:18:28.645 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.645 INFO:teuthology.orchestra.run.smithi112.stdout: "always_on_modules": { 2024-03-16T09:18:28.645 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus": [ 2024-03-16T09:18:28.645 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-03-16T09:18:28.645 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-03-16T09:18:28.645 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-03-16T09:18:28.645 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-03-16T09:18:28.645 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-03-16T09:18:28.645 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-03-16T09:18:28.646 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-03-16T09:18:28.646 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-03-16T09:18:28.646 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-03-16T09:18:28.646 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-03-16T09:18:28.646 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.646 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific": [ 2024-03-16T09:18:28.646 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-03-16T09:18:28.646 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-03-16T09:18:28.646 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-03-16T09:18:28.646 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-03-16T09:18:28.646 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-03-16T09:18:28.646 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-03-16T09:18:28.646 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-03-16T09:18:28.646 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-03-16T09:18:28.647 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-03-16T09:18:28.647 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-03-16T09:18:28.647 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.647 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy": [ 2024-03-16T09:18:28.647 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-03-16T09:18:28.647 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-03-16T09:18:28.647 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-03-16T09:18:28.647 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-03-16T09:18:28.647 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-03-16T09:18:28.647 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-03-16T09:18:28.647 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-03-16T09:18:28.647 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-03-16T09:18:28.647 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-03-16T09:18:28.647 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-03-16T09:18:28.647 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.648 INFO:teuthology.orchestra.run.smithi112.stdout: "reef": [ 2024-03-16T09:18:28.648 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-03-16T09:18:28.648 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-03-16T09:18:28.648 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-03-16T09:18:28.648 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-03-16T09:18:28.648 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-03-16T09:18:28.648 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-03-16T09:18:28.648 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-03-16T09:18:28.648 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-03-16T09:18:28.648 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-03-16T09:18:28.648 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-03-16T09:18:28.648 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-03-16T09:18:28.648 INFO:teuthology.orchestra.run.smithi112.stdout: "squid": [ 2024-03-16T09:18:28.648 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-03-16T09:18:28.648 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-03-16T09:18:28.649 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-03-16T09:18:28.649 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-03-16T09:18:28.649 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-03-16T09:18:28.649 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-03-16T09:18:28.649 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-03-16T09:18:28.649 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-03-16T09:18:28.649 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-03-16T09:18:28.649 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-03-16T09:18:28.649 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-03-16T09:18:28.649 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.649 INFO:teuthology.orchestra.run.smithi112.stdout: "last_failure_osd_epoch": 3, 2024-03-16T09:18:28.649 INFO:teuthology.orchestra.run.smithi112.stdout: "active_clients": [ 2024-03-16T09:18:28.649 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.649 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-03-16T09:18:28.649 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-03-16T09:18:28.650 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.650 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-03-16T09:18:28.650 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-03-16T09:18:28.650 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 2870055140 2024-03-16T09:18:28.650 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.650 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-03-16T09:18:28.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.650 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.650 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "libcephsqlite", 2024-03-16T09:18:28.650 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-03-16T09:18:28.650 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.650 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-03-16T09:18:28.650 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-03-16T09:18:28.650 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 2554500034 2024-03-16T09:18:28.651 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.651 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-03-16T09:18:28.651 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.651 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.651 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-03-16T09:18:28.651 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-03-16T09:18:28.651 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.651 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-03-16T09:18:28.651 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-03-16T09:18:28.651 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1364363659 2024-03-16T09:18:28.651 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.651 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-03-16T09:18:28.651 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:28.651 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.651 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-03-16T09:18:28.651 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-03-16T09:18:28.652 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-03-16T09:18:28.652 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-03-16T09:18:28.652 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-03-16T09:18:28.652 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 679680737 2024-03-16T09:18:28.652 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.652 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-03-16T09:18:28.652 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:28.652 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-03-16T09:18:28.652 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:18:28.654 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:27 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/55257129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:18:28.654 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:27 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/525421764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:18:28.654 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:27 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2887457867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:18:28.654 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:27 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1710751687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:18:28.654 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:28 smithi159 ceph-mon[36739]: pgmap v436: 97 pgs: 97 active+clean; 53 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 334 op/s 2024-03-16T09:18:28.654 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:28 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1447446242' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-16T09:18:28.654 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:27 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/55257129' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:18:28.654 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:27 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/525421764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:18:28.654 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:27 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2887457867' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:18:28.655 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:27 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1710751687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:18:28.655 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:28 smithi194 ceph-mon[36729]: pgmap v436: 97 pgs: 97 active+clean; 53 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 334 op/s 2024-03-16T09:18:28.655 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:28 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1447446242' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-16T09:18:28.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.388+0000 7fb1278f8640 1 -- 172.21.15.112:0/1447446242 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb10c0a2800 msgr2=0x7fb10c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:28.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.388+0000 7fb1278f8640 1 --2- 172.21.15.112:0/1447446242 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb10c0a2800 0x7fb10c0a4cc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb120101770 tx=0x7fb110005f90 comp rx=0 tx=0).stop 2024-03-16T09:18:28.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.388+0000 7fb1278f8640 1 -- 172.21.15.112:0/1447446242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1201022d0 msgr2=0x7fb12019f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:18:28.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.388+0000 7fb1278f8640 1 --2- 172.21.15.112:0/1447446242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1201022d0 0x7fb12019f170 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb11802c700 tx=0x7fb1180346d0 comp rx=0 tx=0).stop 2024-03-16T09:18:28.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.388+0000 7fb1278f8640 1 -- 172.21.15.112:0/1447446242 shutdown_connections 2024-03-16T09:18:28.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.388+0000 7fb1278f8640 1 --2- 172.21.15.112:0/1447446242 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fb10c0a2800 0x7fb10c0a4cc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:28.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.388+0000 7fb1278f8640 1 --2- 172.21.15.112:0/1447446242 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1201022d0 0x7fb12019f170 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:28.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.388+0000 7fb1278f8640 1 --2- 172.21.15.112:0/1447446242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb120101910 0x7fb120198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:28.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.388+0000 7fb1278f8640 1 --2- 172.21.15.112:0/1447446242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb120100720 0x7fb120197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:18:28.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.389+0000 7fb1278f8640 1 -- 172.21.15.112:0/1447446242 >> 172.21.15.112:0/1447446242 conn(0x7fb1200fbc90 msgr2=0x7fb120104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:18:28.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.389+0000 7fb1278f8640 1 -- 172.21.15.112:0/1447446242 shutdown_connections 2024-03-16T09:18:28.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:18:27.389+0000 7fb1278f8640 1 -- 172.21.15.112:0/1447446242 wait complete. 2024-03-16T09:18:28.662 DEBUG:tasks.check_counter:Getting stats from a 2024-03-16T09:18:28.663 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-16T09:18:28.663 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:28 smithi112 ceph-mon[32307]: pgmap v436: 97 pgs: 97 active+clean; 53 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 334 op/s 2024-03-16T09:18:28.663 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:28 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1447446242' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-16T09:18:28.910 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:18:30.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:30 smithi112 ceph-mon[32307]: pgmap v437: 97 pgs: 97 active+clean; 38 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 385 op/s 2024-03-16T09:18:30.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:30 smithi194 ceph-mon[36729]: pgmap v437: 97 pgs: 97 active+clean; 38 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 385 op/s 2024-03-16T09:18:30.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:30 smithi159 ceph-mon[36739]: pgmap v437: 97 pgs: 97 active+clean; 38 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 385 op/s 2024-03-16T09:18:30.768 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:18:30.768 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T09:18:30.768 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 0, 2024-03-16T09:18:30.768 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2, 2024-03-16T09:18:30.768 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 0, 2024-03-16T09:18:30.768 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 0, 2024-03-16T09:18:30.768 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-03-16T09:18:30.768 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-03-16T09:18:30.768 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.001165835, 2024-03-16T09:18:30.769 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000000000, 2024-03-16T09:18:30.769 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000220847, 2024-03-16T09:18:30.769 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T09:18:30.769 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:30.769 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-03-16T09:18:30.769 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-03-16T09:18:30.769 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-03-16T09:18:30.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:30.769 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:30.769 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-03-16T09:18:30.769 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-03-16T09:18:30.769 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-03-16T09:18:30.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:30.769 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-16T09:18:30.770 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-03-16T09:18:30.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:30.770 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T09:18:30.770 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 94, 2024-03-16T09:18:30.770 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 87, 2024-03-16T09:18:30.770 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 716832, 2024-03-16T09:18:30.770 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 51360, 2024-03-16T09:18:30.770 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-03-16T09:18:30.770 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-03-16T09:18:30.770 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.015481610, 2024-03-16T09:18:30.770 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.004973795, 2024-03-16T09:18:30.770 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.012371906, 2024-03-16T09:18:30.770 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.001531005, 2024-03-16T09:18:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 87, 2024-03-16T09:18:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.005263939, 2024-03-16T09:18:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000060505 2024-03-16T09:18:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-03-16T09:18:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-03-16T09:18:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-03-16T09:18:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 716832, 2024-03-16T09:18:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 51360 2024-03-16T09:18:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:30.771 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T09:18:30.772 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 1, 2024-03-16T09:18:30.772 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 64, 2024-03-16T09:18:30.772 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 160, 2024-03-16T09:18:30.772 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 88976, 2024-03-16T09:18:30.772 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 3, 2024-03-16T09:18:30.772 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-03-16T09:18:30.772 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.005323318, 2024-03-16T09:18:30.772 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.003584274, 2024-03-16T09:18:30.772 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.001057648, 2024-03-16T09:18:30.772 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T09:18:30.772 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:30.772 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 62, 2024-03-16T09:18:30.772 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.002939579, 2024-03-16T09:18:30.772 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000047412 2024-03-16T09:18:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-03-16T09:18:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-03-16T09:18:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-03-16T09:18:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-16T09:18:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 88976 2024-03-16T09:18:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-03-16T09:18:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-03-16T09:18:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-03-16T09:18:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:30.773 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-03-16T09:18:30.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-03-16T09:18:30.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-03-16T09:18:30.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T09:18:30.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-03-16T09:18:30.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T09:18:30.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-03-16T09:18:30.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T09:18:30.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T09:18:30.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T09:18:30.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T09:18:30.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T09:18:30.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-03-16T09:18:30.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T09:18:30.774 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T09:18:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-03-16T09:18:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-03-16T09:18:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-03-16T09:18:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-03-16T09:18:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T09:18:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T09:18:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T09:18:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T09:18:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T09:18:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-03-16T09:18:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-03-16T09:18:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-03-16T09:18:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T09:18:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T09:18:30.775 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-03-16T09:18:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-03-16T09:18:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-03-16T09:18:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-03-16T09:18:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T09:18:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-03-16T09:18:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T09:18:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-03-16T09:18:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 148840, 2024-03-16T09:18:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 28, 2024-03-16T09:18:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-03-16T09:18:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-03-16T09:18:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-03-16T09:18:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-03-16T09:18:30.776 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-03-16T09:18:30.777 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-03-16T09:18:30.777 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-03-16T09:18:30.777 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-03-16T09:18:30.777 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-03-16T09:18:30.777 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-03-16T09:18:30.777 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T09:18:30.777 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-03-16T09:18:30.777 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-03-16T09:18:30.777 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-03-16T09:18:30.777 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-03-16T09:18:30.777 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-03-16T09:18:30.777 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-03-16T09:18:30.777 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-03-16T09:18:30.777 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-03-16T09:18:30.778 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-03-16T09:18:30.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:30.778 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T09:18:30.778 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-03-16T09:18:30.778 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-03-16T09:18:30.778 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-03-16T09:18:30.778 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 91, 2024-03-16T09:18:30.778 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 701170, 2024-03-16T09:18:30.778 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:30.778 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 91, 2024-03-16T09:18:30.778 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-03-16T09:18:30.778 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-03-16T09:18:30.778 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 91, 2024-03-16T09:18:30.778 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 701170, 2024-03-16T09:18:30.779 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-03-16T09:18:30.779 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-03-16T09:18:30.779 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-03-16T09:18:30.779 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-03-16T09:18:30.779 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:30.779 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:30.779 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:18:31.478 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-03-16T09:18:31.478 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-03-16T09:18:31.478 DEBUG:tasks.check_counter:Getting stats from d 2024-03-16T09:18:31.479 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-16T09:18:31.736 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:18:32.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:32 smithi112 ceph-mon[32307]: pgmap v438: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 319 op/s 2024-03-16T09:18:32.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:32 smithi194 ceph-mon[36729]: pgmap v438: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 319 op/s 2024-03-16T09:18:32.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:32 smithi159 ceph-mon[36739]: pgmap v438: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 319 op/s 2024-03-16T09:18:33.831 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:18:33.831 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T09:18:33.831 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 4, 2024-03-16T09:18:33.831 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 4, 2024-03-16T09:18:33.832 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 3024, 2024-03-16T09:18:33.832 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 448, 2024-03-16T09:18:33.832 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-03-16T09:18:33.832 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-03-16T09:18:33.832 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.001487310, 2024-03-16T09:18:33.832 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000037844, 2024-03-16T09:18:33.832 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000658473, 2024-03-16T09:18:33.832 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T09:18:33.832 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:33.832 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3, 2024-03-16T09:18:33.832 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000726290, 2024-03-16T09:18:33.832 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000242096 2024-03-16T09:18:33.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:33.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:33.833 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-03-16T09:18:33.833 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-03-16T09:18:33.833 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-03-16T09:18:33.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:33.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-16T09:18:33.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 448 2024-03-16T09:18:33.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:33.833 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T09:18:33.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 96, 2024-03-16T09:18:33.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 91, 2024-03-16T09:18:33.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 715440, 2024-03-16T09:18:33.833 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 54752, 2024-03-16T09:18:33.834 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-03-16T09:18:33.834 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-03-16T09:18:33.834 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.016667842, 2024-03-16T09:18:33.834 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.005083723, 2024-03-16T09:18:33.834 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.014045103, 2024-03-16T09:18:33.834 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.001684493, 2024-03-16T09:18:33.834 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:33.834 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 90, 2024-03-16T09:18:33.834 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.003675763, 2024-03-16T09:18:33.834 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000040841 2024-03-16T09:18:33.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:33.834 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:33.835 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-03-16T09:18:33.835 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-03-16T09:18:33.835 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-03-16T09:18:33.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:33.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 715440, 2024-03-16T09:18:33.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 54752 2024-03-16T09:18:33.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:33.835 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T09:18:33.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 1, 2024-03-16T09:18:33.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 68, 2024-03-16T09:18:33.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 160, 2024-03-16T09:18:33.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 89680, 2024-03-16T09:18:33.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 3, 2024-03-16T09:18:33.835 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-03-16T09:18:33.836 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.005323267, 2024-03-16T09:18:33.836 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.003531702, 2024-03-16T09:18:33.836 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.001130211, 2024-03-16T09:18:33.836 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T09:18:33.836 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:33.836 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 66, 2024-03-16T09:18:33.836 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.003274664, 2024-03-16T09:18:33.836 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000049616 2024-03-16T09:18:33.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:33.836 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:33.836 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-03-16T09:18:33.836 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-03-16T09:18:33.836 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-03-16T09:18:33.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:33.837 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-16T09:18:33.837 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 89680 2024-03-16T09:18:33.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:33.837 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-03-16T09:18:33.837 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-03-16T09:18:33.837 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-03-16T09:18:33.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:33.837 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-03-16T09:18:33.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-03-16T09:18:33.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-03-16T09:18:33.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T09:18:33.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-03-16T09:18:33.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T09:18:33.837 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-03-16T09:18:33.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T09:18:33.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T09:18:33.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T09:18:33.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T09:18:33.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T09:18:33.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-03-16T09:18:33.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T09:18:33.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T09:18:33.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-03-16T09:18:33.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-03-16T09:18:33.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-03-16T09:18:33.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-03-16T09:18:33.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T09:18:33.838 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T09:18:33.839 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T09:18:33.839 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T09:18:33.839 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T09:18:33.839 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-03-16T09:18:33.839 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-03-16T09:18:33.839 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-03-16T09:18:33.839 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T09:18:33.839 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T09:18:33.839 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-03-16T09:18:33.839 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-03-16T09:18:33.839 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-03-16T09:18:33.839 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-03-16T09:18:33.839 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T09:18:33.839 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-03-16T09:18:33.839 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T09:18:33.840 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-03-16T09:18:33.840 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 148903, 2024-03-16T09:18:33.840 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 29, 2024-03-16T09:18:33.840 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-03-16T09:18:33.840 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-03-16T09:18:33.840 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-03-16T09:18:33.840 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-03-16T09:18:33.840 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-03-16T09:18:33.840 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-03-16T09:18:33.840 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-03-16T09:18:33.840 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-03-16T09:18:33.840 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-03-16T09:18:33.840 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-03-16T09:18:33.840 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T09:18:33.841 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-03-16T09:18:33.841 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-03-16T09:18:33.841 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-03-16T09:18:33.841 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-03-16T09:18:33.841 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-03-16T09:18:33.841 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-03-16T09:18:33.841 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-03-16T09:18:33.841 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-03-16T09:18:33.841 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-03-16T09:18:33.841 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:33.841 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T09:18:33.841 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-03-16T09:18:33.841 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-03-16T09:18:33.841 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-03-16T09:18:33.842 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 97, 2024-03-16T09:18:33.842 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 701950, 2024-03-16T09:18:33.842 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:33.842 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 97, 2024-03-16T09:18:33.842 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-03-16T09:18:33.842 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-03-16T09:18:33.842 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 97, 2024-03-16T09:18:33.842 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 701950, 2024-03-16T09:18:33.842 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-03-16T09:18:33.842 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-03-16T09:18:33.842 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-03-16T09:18:33.843 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-03-16T09:18:33.843 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:33.843 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:33.843 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:18:34.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:34 smithi194 ceph-mon[36729]: pgmap v439: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 947 KiB/s wr, 273 op/s 2024-03-16T09:18:34.440 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:34 smithi112 ceph-mon[32307]: pgmap v439: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 947 KiB/s wr, 273 op/s 2024-03-16T09:18:34.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:34 smithi159 ceph-mon[36739]: pgmap v439: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 947 KiB/s wr, 273 op/s 2024-03-16T09:18:34.523 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-16T09:18:34.523 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-16T09:18:34.523 DEBUG:tasks.check_counter:Getting stats from g 2024-03-16T09:18:34.523 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-16T09:18:34.768 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:18:36.233 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:36 smithi112 ceph-mon[32307]: pgmap v440: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 559 KiB/s wr, 207 op/s 2024-03-16T09:18:36.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:36 smithi194 ceph-mon[36729]: pgmap v440: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 559 KiB/s wr, 207 op/s 2024-03-16T09:18:36.483 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:18:36.483 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T09:18:36.483 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 92, 2024-03-16T09:18:36.483 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 87, 2024-03-16T09:18:36.483 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 714352, 2024-03-16T09:18:36.483 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 52192, 2024-03-16T09:18:36.483 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-03-16T09:18:36.483 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-03-16T09:18:36.483 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.017307107, 2024-03-16T09:18:36.483 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.004955538, 2024-03-16T09:18:36.484 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.014526597, 2024-03-16T09:18:36.484 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.001487462, 2024-03-16T09:18:36.484 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:36.484 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 86, 2024-03-16T09:18:36.484 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.004375432, 2024-03-16T09:18:36.484 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000050877 2024-03-16T09:18:36.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:36.484 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:36.484 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-03-16T09:18:36.484 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-03-16T09:18:36.484 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-03-16T09:18:36.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:36.484 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 714352, 2024-03-16T09:18:36.484 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 52192 2024-03-16T09:18:36.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:36.485 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T09:18:36.485 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 5, 2024-03-16T09:18:36.485 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 68, 2024-03-16T09:18:36.485 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 3184, 2024-03-16T09:18:36.485 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 89776, 2024-03-16T09:18:36.485 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 3, 2024-03-16T09:18:36.485 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-03-16T09:18:36.485 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.005952806, 2024-03-16T09:18:36.485 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.004016730, 2024-03-16T09:18:36.485 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.001305308, 2024-03-16T09:18:36.485 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T09:18:36.485 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:36.486 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 67, 2024-03-16T09:18:36.486 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.004137338, 2024-03-16T09:18:36.486 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000061751 2024-03-16T09:18:36.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:36.486 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:36.486 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-03-16T09:18:36.486 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-03-16T09:18:36.486 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-03-16T09:18:36.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:36.486 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-03-16T09:18:36.486 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 89776 2024-03-16T09:18:36.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:36.486 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T09:18:36.486 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 0, 2024-03-16T09:18:36.487 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2, 2024-03-16T09:18:36.487 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 0, 2024-03-16T09:18:36.487 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 0, 2024-03-16T09:18:36.487 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-03-16T09:18:36.487 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-03-16T09:18:36.487 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.000349979, 2024-03-16T09:18:36.487 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000000000, 2024-03-16T09:18:36.487 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000300494, 2024-03-16T09:18:36.487 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T09:18:36.487 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:36.487 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-03-16T09:18:36.487 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-03-16T09:18:36.487 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-03-16T09:18:36.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:36.488 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:36.488 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-03-16T09:18:36.488 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-03-16T09:18:36.488 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-03-16T09:18:36.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:36.488 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-16T09:18:36.488 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-03-16T09:18:36.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:36.488 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-03-16T09:18:36.488 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-03-16T09:18:36.488 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-03-16T09:18:36.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:36.488 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-03-16T09:18:36.488 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-03-16T09:18:36.489 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-03-16T09:18:36.489 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T09:18:36.489 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-03-16T09:18:36.489 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T09:18:36.489 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-03-16T09:18:36.489 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T09:18:36.489 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T09:18:36.489 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T09:18:36.489 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T09:18:36.489 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T09:18:36.489 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-03-16T09:18:36.489 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T09:18:36.489 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T09:18:36.489 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-03-16T09:18:36.490 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-03-16T09:18:36.490 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-03-16T09:18:36.490 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-03-16T09:18:36.490 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T09:18:36.490 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T09:18:36.490 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T09:18:36.490 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T09:18:36.490 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T09:18:36.490 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-03-16T09:18:36.490 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-03-16T09:18:36.490 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-03-16T09:18:36.490 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T09:18:36.490 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T09:18:36.490 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-03-16T09:18:36.491 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-03-16T09:18:36.491 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-03-16T09:18:36.491 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-03-16T09:18:36.491 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T09:18:36.491 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-03-16T09:18:36.491 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T09:18:36.491 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-03-16T09:18:36.491 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 148823, 2024-03-16T09:18:36.491 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 27, 2024-03-16T09:18:36.491 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-03-16T09:18:36.491 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-03-16T09:18:36.491 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-03-16T09:18:36.491 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-03-16T09:18:36.491 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-03-16T09:18:36.491 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-03-16T09:18:36.492 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-03-16T09:18:36.492 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-03-16T09:18:36.492 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-03-16T09:18:36.492 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-03-16T09:18:36.492 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T09:18:36.492 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-03-16T09:18:36.492 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-03-16T09:18:36.492 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-03-16T09:18:36.492 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-03-16T09:18:36.492 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-03-16T09:18:36.492 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-03-16T09:18:36.492 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-03-16T09:18:36.492 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-03-16T09:18:36.492 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-03-16T09:18:36.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:36.493 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T09:18:36.493 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-03-16T09:18:36.493 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-03-16T09:18:36.493 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-03-16T09:18:36.493 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 93, 2024-03-16T09:18:36.493 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 701430, 2024-03-16T09:18:36.493 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:36.493 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 93, 2024-03-16T09:18:36.493 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-03-16T09:18:36.493 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-03-16T09:18:36.493 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 93, 2024-03-16T09:18:36.493 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 701430, 2024-03-16T09:18:36.493 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-03-16T09:18:36.493 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-03-16T09:18:36.494 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-03-16T09:18:36.494 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-03-16T09:18:36.494 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:36.494 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:36.494 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:18:36.494 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:36 smithi159 ceph-mon[36739]: pgmap v440: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 559 KiB/s wr, 207 op/s 2024-03-16T09:18:37.177 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-16T09:18:37.177 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-16T09:18:37.177 DEBUG:tasks.check_counter:Getting stats from j 2024-03-16T09:18:37.177 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-16T09:18:37.419 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.a/config 2024-03-16T09:18:38.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:38 smithi112 ceph-mon[32307]: pgmap v441: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 234 KiB/s wr, 111 op/s 2024-03-16T09:18:38.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:38 smithi194 ceph-mon[36729]: pgmap v441: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 234 KiB/s wr, 111 op/s 2024-03-16T09:18:38.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:38 smithi159 ceph-mon[36739]: pgmap v441: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 234 KiB/s wr, 111 op/s 2024-03-16T09:18:40.021 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-03-16T09:18:40.022 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T09:18:40.022 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 0, 2024-03-16T09:18:40.022 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2, 2024-03-16T09:18:40.022 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 0, 2024-03-16T09:18:40.022 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 0, 2024-03-16T09:18:40.022 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-03-16T09:18:40.022 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-03-16T09:18:40.022 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.001047216, 2024-03-16T09:18:40.022 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000000000, 2024-03-16T09:18:40.023 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000226201, 2024-03-16T09:18:40.023 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T09:18:40.023 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:40.023 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-03-16T09:18:40.023 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-03-16T09:18:40.023 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-03-16T09:18:40.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:40.023 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:40.023 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-03-16T09:18:40.023 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-03-16T09:18:40.023 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-03-16T09:18:40.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:40.024 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-16T09:18:40.024 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-03-16T09:18:40.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:40.024 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T09:18:40.024 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 102, 2024-03-16T09:18:40.024 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 95, 2024-03-16T09:18:40.024 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 719008, 2024-03-16T09:18:40.024 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 56480, 2024-03-16T09:18:40.024 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-03-16T09:18:40.024 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-03-16T09:18:40.024 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.019047497, 2024-03-16T09:18:40.024 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.005431833, 2024-03-16T09:18:40.024 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.016811639, 2024-03-16T09:18:40.025 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.001535507, 2024-03-16T09:18:40.025 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:40.025 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 95, 2024-03-16T09:18:40.025 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.004982576, 2024-03-16T09:18:40.025 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000052448 2024-03-16T09:18:40.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:40.025 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:40.025 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-03-16T09:18:40.025 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-03-16T09:18:40.025 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-03-16T09:18:40.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:40.025 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 719008, 2024-03-16T09:18:40.026 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 56480 2024-03-16T09:18:40.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:40.026 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T09:18:40.026 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 1, 2024-03-16T09:18:40.026 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 70, 2024-03-16T09:18:40.026 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 160, 2024-03-16T09:18:40.026 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 90032, 2024-03-16T09:18:40.026 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 3, 2024-03-16T09:18:40.026 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-03-16T09:18:40.026 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.005964432, 2024-03-16T09:18:40.026 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.003542963, 2024-03-16T09:18:40.026 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.001743587, 2024-03-16T09:18:40.027 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T09:18:40.027 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:40.027 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 68, 2024-03-16T09:18:40.027 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.003299032, 2024-03-16T09:18:40.027 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000048515 2024-03-16T09:18:40.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:40.027 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:40.027 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-03-16T09:18:40.027 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-03-16T09:18:40.027 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-03-16T09:18:40.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:40.027 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-16T09:18:40.027 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 90032 2024-03-16T09:18:40.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:40.028 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-03-16T09:18:40.028 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-03-16T09:18:40.028 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-03-16T09:18:40.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:40.028 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-03-16T09:18:40.028 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-03-16T09:18:40.028 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-03-16T09:18:40.028 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T09:18:40.028 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-03-16T09:18:40.028 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T09:18:40.028 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-03-16T09:18:40.028 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T09:18:40.028 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T09:18:40.028 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T09:18:40.029 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T09:18:40.029 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T09:18:40.029 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-03-16T09:18:40.029 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T09:18:40.029 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T09:18:40.029 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-03-16T09:18:40.029 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-03-16T09:18:40.029 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-03-16T09:18:40.029 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-03-16T09:18:40.029 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T09:18:40.029 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T09:18:40.029 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T09:18:40.029 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T09:18:40.029 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T09:18:40.030 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-03-16T09:18:40.030 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-03-16T09:18:40.030 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-03-16T09:18:40.030 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T09:18:40.030 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T09:18:40.030 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-03-16T09:18:40.030 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-03-16T09:18:40.030 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-03-16T09:18:40.030 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-03-16T09:18:40.030 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T09:18:40.030 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-03-16T09:18:40.030 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T09:18:40.030 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-03-16T09:18:40.030 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 148840, 2024-03-16T09:18:40.031 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 28, 2024-03-16T09:18:40.031 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-03-16T09:18:40.031 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-03-16T09:18:40.031 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-03-16T09:18:40.031 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-03-16T09:18:40.031 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-03-16T09:18:40.031 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-03-16T09:18:40.031 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-03-16T09:18:40.031 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-03-16T09:18:40.031 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-03-16T09:18:40.031 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-03-16T09:18:40.031 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T09:18:40.031 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-03-16T09:18:40.031 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-03-16T09:18:40.032 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-03-16T09:18:40.032 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-03-16T09:18:40.032 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-03-16T09:18:40.032 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-03-16T09:18:40.032 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-03-16T09:18:40.032 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-03-16T09:18:40.032 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-03-16T09:18:40.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-03-16T09:18:40.032 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T09:18:40.032 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-03-16T09:18:40.032 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-03-16T09:18:40.032 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-03-16T09:18:40.032 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 99, 2024-03-16T09:18:40.032 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 702210, 2024-03-16T09:18:40.033 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:40.033 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 99, 2024-03-16T09:18:40.033 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-03-16T09:18:40.033 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-03-16T09:18:40.033 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 99, 2024-03-16T09:18:40.033 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 702210, 2024-03-16T09:18:40.033 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-03-16T09:18:40.033 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-03-16T09:18:40.033 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-03-16T09:18:40.033 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-03-16T09:18:40.033 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:40.033 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-03-16T09:18:40.033 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-03-16T09:18:40.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:40 smithi112 ceph-mon[32307]: pgmap v442: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 223 KiB/s wr, 91 op/s 2024-03-16T09:18:40.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:40 smithi194 ceph-mon[36729]: pgmap v442: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 223 KiB/s wr, 91 op/s 2024-03-16T09:18:40.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:40 smithi159 ceph-mon[36739]: pgmap v442: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 223 KiB/s wr, 91 op/s 2024-03-16T09:18:40.693 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-16T09:18:40.693 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-16T09:18:40.693 DEBUG:tasks.check_counter:Getting stats from b 2024-03-16T09:18:40.694 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-16T09:18:40.941 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T09:18:41.404 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:41 smithi159 ceph-mon[36739]: pgmap v443: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 133 KiB/s wr, 24 op/s 2024-03-16T09:18:41.592 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:41 smithi112 ceph-mon[32307]: pgmap v443: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 133 KiB/s wr, 24 op/s 2024-03-16T09:18:41.651 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:41 smithi194 ceph-mon[36729]: pgmap v443: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 133 KiB/s wr, 24 op/s 2024-03-16T09:18:42.789 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-03-16T09:18:42.789 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T09:18:42.789 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 12883, 2024-03-16T09:18:42.789 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 12318, 2024-03-16T09:18:42.789 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 4107865, 2024-03-16T09:18:42.789 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 58356564, 2024-03-16T09:18:42.789 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 35, 2024-03-16T09:18:42.789 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2024-03-16T09:18:42.789 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 1.237902536, 2024-03-16T09:18:42.789 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.549912125, 2024-03-16T09:18:42.789 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.396651103, 2024-03-16T09:18:42.789 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.339915819, 2024-03-16T09:18:42.790 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:42.790 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 12319, 2024-03-16T09:18:42.790 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 6.872447540, 2024-03-16T09:18:42.790 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000557873 2024-03-16T09:18:42.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.790 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:42.790 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 9015, 2024-03-16T09:18:42.790 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.001372051, 2024-03-16T09:18:42.790 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000152 2024-03-16T09:18:42.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.790 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 40240, 2024-03-16T09:18:42.790 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 1456 2024-03-16T09:18:42.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.791 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T09:18:42.791 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 10744, 2024-03-16T09:18:42.791 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 11328, 2024-03-16T09:18:42.791 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 4262908, 2024-03-16T09:18:42.791 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 22489491, 2024-03-16T09:18:42.791 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 39, 2024-03-16T09:18:42.791 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 5, 2024-03-16T09:18:42.791 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.876147389, 2024-03-16T09:18:42.791 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.291481932, 2024-03-16T09:18:42.791 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.245596933, 2024-03-16T09:18:42.791 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.296569284, 2024-03-16T09:18:42.791 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:42.792 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 11329, 2024-03-16T09:18:42.792 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 10.807156685, 2024-03-16T09:18:42.792 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000953937 2024-03-16T09:18:42.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.792 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:42.792 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3632, 2024-03-16T09:18:42.792 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000628989, 2024-03-16T09:18:42.792 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000173 2024-03-16T09:18:42.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.792 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-16T09:18:42.792 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 198464 2024-03-16T09:18:42.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.792 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T09:18:42.793 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 37216, 2024-03-16T09:18:42.793 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 75209, 2024-03-16T09:18:42.793 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 12590871, 2024-03-16T09:18:42.793 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 49481305, 2024-03-16T09:18:42.793 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 14, 2024-03-16T09:18:42.793 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 8, 2024-03-16T09:18:42.793 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 2.431957908, 2024-03-16T09:18:42.793 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 1.218454330, 2024-03-16T09:18:42.794 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 2.202736981, 2024-03-16T09:18:42.794 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.269100360, 2024-03-16T09:18:42.794 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:42.794 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 75203, 2024-03-16T09:18:42.794 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 6.693582812, 2024-03-16T09:18:42.794 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000089006 2024-03-16T09:18:42.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.794 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:42.794 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 60394, 2024-03-16T09:18:42.794 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.010794681, 2024-03-16T09:18:42.794 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000178 2024-03-16T09:18:42.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.795 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 842368, 2024-03-16T09:18:42.795 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 96000 2024-03-16T09:18:42.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.795 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-03-16T09:18:42.795 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-03-16T09:18:42.795 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-03-16T09:18:42.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.795 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-03-16T09:18:42.795 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-16T09:18:42.795 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-16T09:18:42.795 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3768, 2024-03-16T09:18:42.795 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 26.800918745, 2024-03-16T09:18:42.795 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.007112770 2024-03-16T09:18:42.796 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:42.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.796 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-03-16T09:18:42.796 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-16T09:18:42.796 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-16T09:18:42.796 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 10800, 2024-03-16T09:18:42.796 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3.970139449, 2024-03-16T09:18:42.796 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000367605 2024-03-16T09:18:42.796 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:42.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.796 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-03-16T09:18:42.796 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 19391, 2024-03-16T09:18:42.796 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 19366, 2024-03-16T09:18:42.796 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-03-16T09:18:42.797 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 19366, 2024-03-16T09:18:42.797 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 49.541898243, 2024-03-16T09:18:42.797 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002558189 2024-03-16T09:18:42.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.797 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-03-16T09:18:42.797 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 25, 2024-03-16T09:18:42.797 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 5, 2024-03-16T09:18:42.797 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 2, 2024-03-16T09:18:42.797 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 1059, 2024-03-16T09:18:42.797 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 13, 2024-03-16T09:18:42.797 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 4, 2024-03-16T09:18:42.797 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 366, 2024-03-16T09:18:42.797 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 11, 2024-03-16T09:18:42.797 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 341, 2024-03-16T09:18:42.798 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 14, 2024-03-16T09:18:42.798 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 349, 2024-03-16T09:18:42.798 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 2098, 2024-03-16T09:18:42.798 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 3, 2024-03-16T09:18:42.798 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 3, 2024-03-16T09:18:42.798 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 4, 2024-03-16T09:18:42.798 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 19913, 2024-03-16T09:18:42.798 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 19695, 2024-03-16T09:18:42.799 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2024-03-16T09:18:42.799 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 0, 2024-03-16T09:18:42.799 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-03-16T09:18:42.799 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-03-16T09:18:42.799 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2024-03-16T09:18:42.799 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-03-16T09:18:42.799 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-03-16T09:18:42.799 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 120, 2024-03-16T09:18:42.799 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 1990, 2024-03-16T09:18:42.799 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 25, 2024-03-16T09:18:42.799 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 5, 2024-03-16T09:18:42.799 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-03-16T09:18:42.799 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 2320, 2024-03-16T09:18:42.799 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-03-16T09:18:42.801 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 28, 2024-03-16T09:18:42.801 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 233485, 2024-03-16T09:18:42.801 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2024-03-16T09:18:42.801 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 2320, 2024-03-16T09:18:42.801 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-03-16T09:18:42.801 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T09:18:42.801 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-03-16T09:18:42.801 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 201, 2024-03-16T09:18:42.801 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 4, 2024-03-16T09:18:42.801 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 197, 2024-03-16T09:18:42.802 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 2119, 2024-03-16T09:18:42.802 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 50, 2024-03-16T09:18:42.802 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 1415, 2024-03-16T09:18:42.802 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 18929, 2024-03-16T09:18:42.802 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T09:18:42.802 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T09:18:42.802 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T09:18:42.802 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 10803, 2024-03-16T09:18:42.802 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 8463, 2024-03-16T09:18:42.802 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 92, 2024-03-16T09:18:42.802 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 10007 2024-03-16T09:18:42.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.802 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-03-16T09:18:42.803 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2024-03-16T09:18:42.803 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2024-03-16T09:18:42.803 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2024-03-16T09:18:42.803 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 0, 2024-03-16T09:18:42.803 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 126, 2024-03-16T09:18:42.803 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 6, 2024-03-16T09:18:42.804 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-03-16T09:18:42.804 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-03-16T09:18:42.804 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 8158, 2024-03-16T09:18:42.804 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 8152, 2024-03-16T09:18:42.804 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-03-16T09:18:42.804 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-03-16T09:18:42.804 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-03-16T09:18:42.804 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-03-16T09:18:42.804 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-03-16T09:18:42.804 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-03-16T09:18:42.804 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-03-16T09:18:42.804 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_path": 0, 2024-03-16T09:18:42.804 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_inode": 0, 2024-03-16T09:18:42.805 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 4, 2024-03-16T09:18:42.805 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 120, 2024-03-16T09:18:42.805 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-03-16T09:18:42.805 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 17, 2024-03-16T09:18:42.805 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-03-16T09:18:42.805 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-03-16T09:18:42.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.805 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-03-16T09:18:42.805 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-03-16T09:18:42.805 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 40811, 2024-03-16T09:18:42.806 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 39708, 2024-03-16T09:18:42.806 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 39651, 2024-03-16T09:18:42.806 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 1160, 2024-03-16T09:18:42.806 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-03-16T09:18:42.806 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 10081, 2024-03-16T09:18:42.806 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 62, 2024-03-16T09:18:42.806 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 60, 2024-03-16T09:18:42.806 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 59, 2024-03-16T09:18:42.806 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 3, 2024-03-16T09:18:42.806 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 1, 2024-03-16T09:18:42.806 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-03-16T09:18:42.806 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 14, 2024-03-16T09:18:42.807 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 83679489, 2024-03-16T09:18:42.807 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 85674899, 2024-03-16T09:18:42.807 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2024-03-16T09:18:42.807 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-03-16T09:18:42.807 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1442, 2024-03-16T09:18:42.807 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 77.810341722, 2024-03-16T09:18:42.807 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.053960015 2024-03-16T09:18:42.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.807 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2024-03-16T09:18:42.807 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.807 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-03-16T09:18:42.807 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 36, 2024-03-16T09:18:42.807 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 10191, 2024-03-16T09:18:42.807 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 10155, 2024-03-16T09:18:42.807 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 38, 2024-03-16T09:18:42.808 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 767, 2024-03-16T09:18:42.808 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 729, 2024-03-16T09:18:42.808 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 366, 2024-03-16T09:18:42.808 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 16342, 2024-03-16T09:18:42.808 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 15976, 2024-03-16T09:18:42.808 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 3, 2024-03-16T09:18:42.808 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 10169, 2024-03-16T09:18:42.808 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 10166, 2024-03-16T09:18:42.808 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 93804, 2024-03-16T09:18:42.808 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 231804 2024-03-16T09:18:42.808 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.808 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-03-16T09:18:42.808 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 21858, 2024-03-16T09:18:42.808 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 541, 2024-03-16T09:18:42.808 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 19391, 2024-03-16T09:18:42.809 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 127, 2024-03-16T09:18:42.809 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 1258, 2024-03-16T09:18:42.809 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-03-16T09:18:42.809 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 9535, 2024-03-16T09:18:42.809 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 20.599757809, 2024-03-16T09:18:42.809 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002160436 2024-03-16T09:18:42.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.809 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-03-16T09:18:42.809 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 43, 2024-03-16T09:18:42.809 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.024546517, 2024-03-16T09:18:42.809 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000570849 2024-03-16T09:18:42.809 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.809 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-03-16T09:18:42.809 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.810 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.810 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.810 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-03-16T09:18:42.810 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.810 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.810 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.810 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-03-16T09:18:42.810 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 254, 2024-03-16T09:18:42.810 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1.284024402, 2024-03-16T09:18:42.810 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.005055214 2024-03-16T09:18:42.810 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.811 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-03-16T09:18:42.811 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.811 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.811 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.811 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-03-16T09:18:42.811 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2, 2024-03-16T09:18:42.811 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000241181, 2024-03-16T09:18:42.811 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000120590 2024-03-16T09:18:42.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.811 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-03-16T09:18:42.811 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.811 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.811 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.812 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-03-16T09:18:42.812 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.812 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.812 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.812 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-03-16T09:18:42.812 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.812 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.812 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.813 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.813 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-03-16T09:18:42.813 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.813 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.813 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.814 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-03-16T09:18:42.814 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 623, 2024-03-16T09:18:42.814 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.745860829, 2024-03-16T09:18:42.814 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001197208 2024-03-16T09:18:42.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.814 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-03-16T09:18:42.814 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.814 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.814 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.814 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.814 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-03-16T09:18:42.814 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.814 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-03-16T09:18:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-03-16T09:18:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000248952, 2024-03-16T09:18:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000248952 2024-03-16T09:18:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-03-16T09:18:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 721, 2024-03-16T09:18:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1.058459304, 2024-03-16T09:18:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001468043 2024-03-16T09:18:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.815 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-03-16T09:18:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2, 2024-03-16T09:18:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.002788273, 2024-03-16T09:18:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001394136 2024-03-16T09:18:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-03-16T09:18:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-03-16T09:18:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.816 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-03-16T09:18:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 469, 2024-03-16T09:18:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 9.060288668, 2024-03-16T09:18:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.019318312 2024-03-16T09:18:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-03-16T09:18:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-03-16T09:18:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-03-16T09:18:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 19, 2024-03-16T09:18:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 2.837321860, 2024-03-16T09:18:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.149332729 2024-03-16T09:18:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-03-16T09:18:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-03-16T09:18:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.818 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-03-16T09:18:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-03-16T09:18:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 8, 2024-03-16T09:18:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.017786175, 2024-03-16T09:18:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002223271 2024-03-16T09:18:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-03-16T09:18:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.819 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-03-16T09:18:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 7689, 2024-03-16T09:18:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 13.910574273, 2024-03-16T09:18:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001809152 2024-03-16T09:18:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-03-16T09:18:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-03-16T09:18:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-03-16T09:18:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.820 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-03-16T09:18:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2024-03-16T09:18:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 3, 2024-03-16T09:18:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 2, 2024-03-16T09:18:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 1, 2024-03-16T09:18:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-03-16T09:18:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 12623, 2024-03-16T09:18:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 12623, 2024-03-16T09:18:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 187, 2024-03-16T09:18:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-03-16T09:18:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-03-16T09:18:42.821 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 240, 2024-03-16T09:18:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 240, 2024-03-16T09:18:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T09:18:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-03-16T09:18:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T09:18:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-03-16T09:18:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T09:18:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T09:18:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T09:18:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T09:18:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T09:18:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-03-16T09:18:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T09:18:42.822 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T09:18:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-03-16T09:18:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-03-16T09:18:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-03-16T09:18:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-03-16T09:18:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T09:18:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T09:18:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T09:18:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T09:18:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T09:18:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-03-16T09:18:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-03-16T09:18:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-03-16T09:18:42.823 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T09:18:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T09:18:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-03-16T09:18:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-03-16T09:18:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-03-16T09:18:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-03-16T09:18:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T09:18:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-03-16T09:18:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T09:18:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-03-16T09:18:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 291079, 2024-03-16T09:18:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 131, 2024-03-16T09:18:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 1320, 2024-03-16T09:18:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 15, 2024-03-16T09:18:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-03-16T09:18:42.824 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-03-16T09:18:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-03-16T09:18:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-03-16T09:18:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-03-16T09:18:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-03-16T09:18:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 7528, 2024-03-16T09:18:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 158, 2024-03-16T09:18:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T09:18:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-03-16T09:18:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-03-16T09:18:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-03-16T09:18:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 406290, 2024-03-16T09:18:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 4311, 2024-03-16T09:18:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-03-16T09:18:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-03-16T09:18:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-03-16T09:18:42.825 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-03-16T09:18:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-03-16T09:18:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-03-16T09:18:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-03-16T09:18:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 23395, 2024-03-16T09:18:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 96030135, 2024-03-16T09:18:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 36, 2024-03-16T09:18:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 23395, 2024-03-16T09:18:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-03-16T09:18:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 23395, 2024-03-16T09:18:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 2220.615622724, 2024-03-16T09:18:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.094918385 2024-03-16T09:18:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-03-16T09:18:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-03-16T09:18:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 23395, 2024-03-16T09:18:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 43503 2024-03-16T09:18:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 23395, 2024-03-16T09:18:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 2459, 2024-03-16T09:18:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 20936, 2024-03-16T09:18:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-03-16T09:18:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-03-16T09:18:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 472, 2024-03-16T09:18:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 9218, 2024-03-16T09:18:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 128, 2024-03-16T09:18:42.827 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 1566, 2024-03-16T09:18:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 117, 2024-03-16T09:18:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-03-16T09:18:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-03-16T09:18:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-03-16T09:18:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 13, 2024-03-16T09:18:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 8857, 2024-03-16T09:18:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-03-16T09:18:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-03-16T09:18:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-03-16T09:18:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 2331, 2024-03-16T09:18:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 18436, 2024-03-16T09:18:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-03-16T09:18:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-03-16T09:18:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-03-16T09:18:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-03-16T09:18:42.828 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-03-16T09:18:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-03-16T09:18:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T09:18:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-03-16T09:18:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-03-16T09:18:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-03-16T09:18:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-03-16T09:18:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-03-16T09:18:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-03-16T09:18:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-03-16T09:18:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-03-16T09:18:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-03-16T09:18:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-03-16T09:18:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-03-16T09:18:42.829 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-03-16T09:18:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-03-16T09:18:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-03-16T09:18:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-03-16T09:18:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-03-16T09:18:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-03-16T09:18:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-03-16T09:18:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-03-16T09:18:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 88, 2024-03-16T09:18:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-03-16T09:18:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 4, 2024-03-16T09:18:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 12, 2024-03-16T09:18:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 12, 2024-03-16T09:18:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2024-03-16T09:18:42.830 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-03-16T09:18:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 2234, 2024-03-16T09:18:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 14, 2024-03-16T09:18:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 117 2024-03-16T09:18:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-03-16T09:18:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2024-03-16T09:18:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-03-16T09:18:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-03-16T09:18:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-03-16T09:18:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-03-16T09:18:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2024-03-16T09:18:42.831 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-03-16T09:18:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-03-16T09:18:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 40, 2024-03-16T09:18:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-03-16T09:18:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 10, 2024-03-16T09:18:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 8284, 2024-03-16T09:18:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 8152, 2024-03-16T09:18:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-03-16T09:18:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T09:18:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-16T09:18:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-16T09:18:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-16T09:18:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 60839, 2024-03-16T09:18:42.832 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 15956039, 2024-03-16T09:18:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 60839, 2024-03-16T09:18:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-16T09:18:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-16T09:18:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 60839, 2024-03-16T09:18:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 15956039, 2024-03-16T09:18:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-16T09:18:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-03-16T09:18:42.833 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-16T09:18:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-16T09:18:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-16T09:18:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-16T09:18:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-16T09:18:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-16T09:18:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 23395, 2024-03-16T09:18:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 96853503, 2024-03-16T09:18:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 23395, 2024-03-16T09:18:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 96853503, 2024-03-16T09:18:42.834 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-16T09:18:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-03-16T09:18:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-16T09:18:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-03-16T09:18:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-16T09:18:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-16T09:18:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-16T09:18:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-16T09:18:42.835 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 23395, 2024-03-16T09:18:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 23395, 2024-03-16T09:18:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 23395, 2024-03-16T09:18:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 23395, 2024-03-16T09:18:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-16T09:18:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-03-16T09:18:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-16T09:18:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-16T09:18:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-16T09:18:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 8152, 2024-03-16T09:18:42.836 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 823456, 2024-03-16T09:18:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 8152, 2024-03-16T09:18:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-16T09:18:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-16T09:18:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 124, 2024-03-16T09:18:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 823456, 2024-03-16T09:18:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-16T09:18:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x5632c2dc6500": { 2024-03-16T09:18:42.837 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-16T09:18:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-16T09:18:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-16T09:18:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 40811, 2024-03-16T09:18:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 81480595, 2024-03-16T09:18:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 40811, 2024-03-16T09:18:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-16T09:18:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-16T09:18:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1442, 2024-03-16T09:18:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 81480595, 2024-03-16T09:18:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-16T09:18:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:42.838 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:42.839 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:42.839 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:42.839 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-03-16T09:18:43.564 INFO:tasks.check_counter:Daemon mds.b mds.exported=120 2024-03-16T09:18:43.564 INFO:tasks.check_counter:Daemon mds.b mds.imported=25 2024-03-16T09:18:43.564 DEBUG:tasks.check_counter:Getting stats from e 2024-03-16T09:18:43.565 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-16T09:18:43.889 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T09:18:44.157 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:44 smithi159 ceph-mon[36739]: pgmap v444: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-03-16T09:18:44.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:44 smithi112 ceph-mon[32307]: pgmap v444: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-03-16T09:18:44.353 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:44 smithi194 ceph-mon[36729]: pgmap v444: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-03-16T09:18:45.664 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-03-16T09:18:45.664 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T09:18:45.664 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 351, 2024-03-16T09:18:45.664 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 353, 2024-03-16T09:18:45.664 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 1420444, 2024-03-16T09:18:45.664 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 111682, 2024-03-16T09:18:45.664 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 6, 2024-03-16T09:18:45.664 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 2, 2024-03-16T09:18:45.664 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.024304549, 2024-03-16T09:18:45.664 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.008167511, 2024-03-16T09:18:45.665 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.006796210, 2024-03-16T09:18:45.665 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.006644842, 2024-03-16T09:18:45.665 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:45.665 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 351, 2024-03-16T09:18:45.665 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.008963190, 2024-03-16T09:18:45.665 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000025536 2024-03-16T09:18:45.665 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.665 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:45.665 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.665 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.665 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.665 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.666 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-16T09:18:45.666 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 0 2024-03-16T09:18:45.666 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.666 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T09:18:45.666 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 196, 2024-03-16T09:18:45.666 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 263, 2024-03-16T09:18:45.666 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 50192, 2024-03-16T09:18:45.666 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 259079, 2024-03-16T09:18:45.666 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 5, 2024-03-16T09:18:45.666 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2024-03-16T09:18:45.666 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.016508598, 2024-03-16T09:18:45.666 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.005432527, 2024-03-16T09:18:45.666 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.004377915, 2024-03-16T09:18:45.667 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.005235110, 2024-03-16T09:18:45.667 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:45.667 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 262, 2024-03-16T09:18:45.667 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.007883822, 2024-03-16T09:18:45.667 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000030090 2024-03-16T09:18:45.667 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.667 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:45.667 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.667 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.667 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.667 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.667 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-03-16T09:18:45.667 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 198336 2024-03-16T09:18:45.668 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.668 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T09:18:45.668 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 420, 2024-03-16T09:18:45.668 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 406, 2024-03-16T09:18:45.668 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 871568, 2024-03-16T09:18:45.668 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 154382, 2024-03-16T09:18:45.668 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 3, 2024-03-16T09:18:45.668 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 2, 2024-03-16T09:18:45.668 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.037955537, 2024-03-16T09:18:45.668 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.010290372, 2024-03-16T09:18:45.668 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.022726575, 2024-03-16T09:18:45.668 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.009103562, 2024-03-16T09:18:45.669 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:45.669 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 405, 2024-03-16T09:18:45.669 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.011763163, 2024-03-16T09:18:45.669 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000029044 2024-03-16T09:18:45.669 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.669 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:45.669 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.669 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.669 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.669 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.669 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 769088, 2024-03-16T09:18:45.669 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 57392 2024-03-16T09:18:45.669 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.670 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-03-16T09:18:45.670 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-03-16T09:18:45.670 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-03-16T09:18:45.670 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.670 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-03-16T09:18:45.670 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-16T09:18:45.670 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-16T09:18:45.670 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1484, 2024-03-16T09:18:45.670 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.030149256, 2024-03-16T09:18:45.670 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000020316 2024-03-16T09:18:45.670 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:45.670 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.671 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-03-16T09:18:45.671 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-16T09:18:45.671 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-16T09:18:45.671 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.671 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.671 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.671 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:45.671 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.671 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-03-16T09:18:45.671 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 0, 2024-03-16T09:18:45.671 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 0, 2024-03-16T09:18:45.671 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-03-16T09:18:45.672 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.672 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.672 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.672 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.672 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-03-16T09:18:45.672 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 0, 2024-03-16T09:18:45.672 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-03-16T09:18:45.672 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-03-16T09:18:45.672 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 0, 2024-03-16T09:18:45.672 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2024-03-16T09:18:45.672 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2024-03-16T09:18:45.672 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 177, 2024-03-16T09:18:45.672 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 0, 2024-03-16T09:18:45.672 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 177, 2024-03-16T09:18:45.673 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 0, 2024-03-16T09:18:45.673 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 177, 2024-03-16T09:18:45.673 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 0, 2024-03-16T09:18:45.673 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 0, 2024-03-16T09:18:45.673 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 0, 2024-03-16T09:18:45.673 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 1, 2024-03-16T09:18:45.673 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 0, 2024-03-16T09:18:45.673 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 0, 2024-03-16T09:18:45.673 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2024-03-16T09:18:45.673 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 0, 2024-03-16T09:18:45.673 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-03-16T09:18:45.673 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-03-16T09:18:45.673 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2024-03-16T09:18:45.673 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-03-16T09:18:45.674 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-03-16T09:18:45.674 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2024-03-16T09:18:45.674 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2024-03-16T09:18:45.674 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2024-03-16T09:18:45.674 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2024-03-16T09:18:45.674 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-03-16T09:18:45.674 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2024-03-16T09:18:45.674 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-03-16T09:18:45.674 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 0, 2024-03-16T09:18:45.674 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 0, 2024-03-16T09:18:45.674 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2024-03-16T09:18:45.674 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2024-03-16T09:18:45.674 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-03-16T09:18:45.675 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T09:18:45.675 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-03-16T09:18:45.675 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2024-03-16T09:18:45.675 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T09:18:45.675 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2024-03-16T09:18:45.675 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2024-03-16T09:18:45.675 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2024-03-16T09:18:45.675 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 0, 2024-03-16T09:18:45.675 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 0, 2024-03-16T09:18:45.675 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T09:18:45.675 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T09:18:45.675 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T09:18:45.675 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 0, 2024-03-16T09:18:45.676 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 0, 2024-03-16T09:18:45.676 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 0, 2024-03-16T09:18:45.676 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 0 2024-03-16T09:18:45.676 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.676 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-03-16T09:18:45.676 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2024-03-16T09:18:45.676 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2024-03-16T09:18:45.676 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2024-03-16T09:18:45.676 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 0, 2024-03-16T09:18:45.676 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 0, 2024-03-16T09:18:45.676 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-03-16T09:18:45.676 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-03-16T09:18:45.676 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-03-16T09:18:45.676 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 0, 2024-03-16T09:18:45.676 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 0, 2024-03-16T09:18:45.677 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-03-16T09:18:45.677 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-03-16T09:18:45.677 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-03-16T09:18:45.677 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-03-16T09:18:45.677 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-03-16T09:18:45.677 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-03-16T09:18:45.677 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-03-16T09:18:45.677 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_path": 0, 2024-03-16T09:18:45.677 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_inode": 0, 2024-03-16T09:18:45.677 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T09:18:45.677 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2024-03-16T09:18:45.677 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-03-16T09:18:45.677 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2024-03-16T09:18:45.677 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-03-16T09:18:45.678 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-03-16T09:18:45.678 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.678 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-03-16T09:18:45.678 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-03-16T09:18:45.678 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 0, 2024-03-16T09:18:45.678 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2024-03-16T09:18:45.678 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2024-03-16T09:18:45.678 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 807, 2024-03-16T09:18:45.678 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-03-16T09:18:45.678 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2024-03-16T09:18:45.678 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 0, 2024-03-16T09:18:45.678 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2024-03-16T09:18:45.678 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2024-03-16T09:18:45.678 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 2, 2024-03-16T09:18:45.679 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 2, 2024-03-16T09:18:45.679 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-03-16T09:18:45.679 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2024-03-16T09:18:45.679 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2024-03-16T09:18:45.679 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 5522835, 2024-03-16T09:18:45.679 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 5521697, 2024-03-16T09:18:45.679 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-03-16T09:18:45.679 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.679 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.679 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.679 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.679 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 807 2024-03-16T09:18:45.679 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.679 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-03-16T09:18:45.680 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 4, 2024-03-16T09:18:45.680 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 243, 2024-03-16T09:18:45.680 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 239, 2024-03-16T09:18:45.680 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 2, 2024-03-16T09:18:45.680 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 75, 2024-03-16T09:18:45.680 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 73, 2024-03-16T09:18:45.680 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 177, 2024-03-16T09:18:45.680 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 357, 2024-03-16T09:18:45.680 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 180, 2024-03-16T09:18:45.680 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 0, 2024-03-16T09:18:45.680 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 0, 2024-03-16T09:18:45.680 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 0, 2024-03-16T09:18:45.680 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 50860, 2024-03-16T09:18:45.681 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 215420 2024-03-16T09:18:45.681 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.681 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-03-16T09:18:45.681 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 0, 2024-03-16T09:18:45.681 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2024-03-16T09:18:45.681 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 0, 2024-03-16T09:18:45.681 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 0, 2024-03-16T09:18:45.681 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2024-03-16T09:18:45.681 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-03-16T09:18:45.681 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.681 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.681 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.681 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.681 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-03-16T09:18:45.682 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.682 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.682 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.682 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.682 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-03-16T09:18:45.682 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.682 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.682 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.682 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.682 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-03-16T09:18:45.682 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.682 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.682 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.682 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.683 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-03-16T09:18:45.683 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.683 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.683 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.683 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.683 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-03-16T09:18:45.683 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.683 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.683 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.683 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.683 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-03-16T09:18:45.683 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.683 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.683 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.684 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.684 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-03-16T09:18:45.684 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.684 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.684 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.684 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.684 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-03-16T09:18:45.684 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.684 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.684 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.684 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.684 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-03-16T09:18:45.684 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.684 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.684 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.684 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.685 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-03-16T09:18:45.685 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.685 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.685 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.685 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.685 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-03-16T09:18:45.685 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.686 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.686 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.686 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.686 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-03-16T09:18:45.686 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.686 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.686 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.686 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.687 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-03-16T09:18:45.687 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.687 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.687 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.687 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.687 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-03-16T09:18:45.687 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.687 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.687 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.687 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.687 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-03-16T09:18:45.687 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.687 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.687 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.688 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.688 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-03-16T09:18:45.688 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.688 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.688 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.688 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.688 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-03-16T09:18:45.688 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.688 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.688 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.688 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.688 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-03-16T09:18:45.688 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.688 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.688 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.689 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.689 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-03-16T09:18:45.689 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.689 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.689 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.689 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.689 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-03-16T09:18:45.689 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.689 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.689 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.689 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.689 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-03-16T09:18:45.689 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.690 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.690 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.690 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.690 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-03-16T09:18:45.690 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.690 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.690 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.690 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.690 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-03-16T09:18:45.690 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.690 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.690 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.690 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.690 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-03-16T09:18:45.691 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.691 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.691 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.691 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.691 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-03-16T09:18:45.691 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.691 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.691 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.691 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.691 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-03-16T09:18:45.691 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.691 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.691 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.692 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.692 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-03-16T09:18:45.692 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.692 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.692 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.692 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.692 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-03-16T09:18:45.692 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.692 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.692 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.692 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.692 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-03-16T09:18:45.692 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-03-16T09:18:45.693 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-03-16T09:18:45.693 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.693 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.693 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.693 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:45.693 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.693 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-03-16T09:18:45.693 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2024-03-16T09:18:45.693 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 2, 2024-03-16T09:18:45.693 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 1, 2024-03-16T09:18:45.693 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 2, 2024-03-16T09:18:45.693 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-03-16T09:18:45.693 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 0, 2024-03-16T09:18:45.694 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 0, 2024-03-16T09:18:45.694 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 2990, 2024-03-16T09:18:45.694 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-03-16T09:18:45.694 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.694 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-03-16T09:18:45.694 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-03-16T09:18:45.694 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-03-16T09:18:45.694 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T09:18:45.694 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-03-16T09:18:45.694 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T09:18:45.694 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-03-16T09:18:45.694 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T09:18:45.695 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T09:18:45.695 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T09:18:45.695 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T09:18:45.695 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T09:18:45.695 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-03-16T09:18:45.695 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T09:18:45.695 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T09:18:45.695 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-03-16T09:18:45.695 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-03-16T09:18:45.695 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-03-16T09:18:45.695 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-03-16T09:18:45.695 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T09:18:45.695 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T09:18:45.695 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T09:18:45.696 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T09:18:45.696 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T09:18:45.696 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-03-16T09:18:45.696 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-03-16T09:18:45.696 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-03-16T09:18:45.696 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T09:18:45.696 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T09:18:45.696 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-03-16T09:18:45.696 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-03-16T09:18:45.696 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-03-16T09:18:45.696 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-03-16T09:18:45.696 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T09:18:45.696 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-03-16T09:18:45.696 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T09:18:45.697 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-03-16T09:18:45.697 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 189279, 2024-03-16T09:18:45.697 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 49, 2024-03-16T09:18:45.697 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-03-16T09:18:45.697 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-03-16T09:18:45.697 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-03-16T09:18:45.697 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-03-16T09:18:45.697 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-03-16T09:18:45.697 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-03-16T09:18:45.697 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-03-16T09:18:45.697 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-03-16T09:18:45.697 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 7528, 2024-03-16T09:18:45.697 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 158, 2024-03-16T09:18:45.697 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T09:18:45.698 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-03-16T09:18:45.698 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-03-16T09:18:45.698 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-03-16T09:18:45.698 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 136600, 2024-03-16T09:18:45.698 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 737, 2024-03-16T09:18:45.698 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-03-16T09:18:45.698 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-03-16T09:18:45.698 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-03-16T09:18:45.699 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-03-16T09:18:45.699 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.699 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-03-16T09:18:45.699 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-03-16T09:18:45.699 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-03-16T09:18:45.699 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 847, 2024-03-16T09:18:45.699 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 21, 2024-03-16T09:18:45.699 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 3, 2024-03-16T09:18:45.699 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 847, 2024-03-16T09:18:45.699 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-03-16T09:18:45.699 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 847, 2024-03-16T09:18:45.699 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1.274997480, 2024-03-16T09:18:45.699 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001505309 2024-03-16T09:18:45.700 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.700 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-03-16T09:18:45.700 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-03-16T09:18:45.700 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 847, 2024-03-16T09:18:45.700 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 848 2024-03-16T09:18:45.700 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.700 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 847, 2024-03-16T09:18:45.700 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 847, 2024-03-16T09:18:45.700 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 0, 2024-03-16T09:18:45.700 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-03-16T09:18:45.700 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-03-16T09:18:45.700 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 494, 2024-03-16T09:18:45.700 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 0, 2024-03-16T09:18:45.700 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 351, 2024-03-16T09:18:45.700 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 0, 2024-03-16T09:18:45.701 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 0, 2024-03-16T09:18:45.701 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-03-16T09:18:45.701 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-03-16T09:18:45.701 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-03-16T09:18:45.701 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2024-03-16T09:18:45.701 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 0, 2024-03-16T09:18:45.701 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-03-16T09:18:45.701 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-03-16T09:18:45.701 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-03-16T09:18:45.701 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 1, 2024-03-16T09:18:45.701 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 0, 2024-03-16T09:18:45.702 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-03-16T09:18:45.702 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-03-16T09:18:45.702 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-03-16T09:18:45.702 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-03-16T09:18:45.702 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-03-16T09:18:45.702 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-03-16T09:18:45.702 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T09:18:45.702 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-03-16T09:18:45.702 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-03-16T09:18:45.702 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-03-16T09:18:45.702 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-03-16T09:18:45.702 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-03-16T09:18:45.702 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-03-16T09:18:45.703 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-03-16T09:18:45.703 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-03-16T09:18:45.703 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-03-16T09:18:45.703 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-03-16T09:18:45.703 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-03-16T09:18:45.703 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-03-16T09:18:45.703 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-03-16T09:18:45.703 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-03-16T09:18:45.703 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-03-16T09:18:45.703 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-03-16T09:18:45.703 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-03-16T09:18:45.703 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-03-16T09:18:45.703 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-03-16T09:18:45.703 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 88, 2024-03-16T09:18:45.704 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-03-16T09:18:45.704 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 3, 2024-03-16T09:18:45.704 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 5, 2024-03-16T09:18:45.704 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 5, 2024-03-16T09:18:45.704 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2024-03-16T09:18:45.704 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-03-16T09:18:45.704 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 0, 2024-03-16T09:18:45.704 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 2, 2024-03-16T09:18:45.704 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 0 2024-03-16T09:18:45.704 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.704 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-03-16T09:18:45.704 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 0, 2024-03-16T09:18:45.704 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-03-16T09:18:45.704 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-03-16T09:18:45.704 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-03-16T09:18:45.705 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.705 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-03-16T09:18:45.705 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2024-03-16T09:18:45.705 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.705 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-03-16T09:18:45.705 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-03-16T09:18:45.705 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 0, 2024-03-16T09:18:45.705 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-03-16T09:18:45.705 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 0, 2024-03-16T09:18:45.705 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 0, 2024-03-16T09:18:45.705 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 0, 2024-03-16T09:18:45.705 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-03-16T09:18:45.705 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.705 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T09:18:45.706 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-16T09:18:45.706 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-16T09:18:45.706 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-16T09:18:45.706 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 963, 2024-03-16T09:18:45.706 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 2246688, 2024-03-16T09:18:45.706 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:45.706 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 963, 2024-03-16T09:18:45.706 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-16T09:18:45.706 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-16T09:18:45.706 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 963, 2024-03-16T09:18:45.706 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 2246688, 2024-03-16T09:18:45.706 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-16T09:18:45.706 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.706 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.707 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.707 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:45.707 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.707 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-03-16T09:18:45.707 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-16T09:18:45.707 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-16T09:18:45.707 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-16T09:18:45.707 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-16T09:18:45.707 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-16T09:18:45.707 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:45.707 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-16T09:18:45.707 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 847, 2024-03-16T09:18:45.707 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 1328536, 2024-03-16T09:18:45.707 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 847, 2024-03-16T09:18:45.708 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1328536, 2024-03-16T09:18:45.708 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-16T09:18:45.708 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.708 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.708 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.708 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:45.708 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.708 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-03-16T09:18:45.708 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-16T09:18:45.708 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-03-16T09:18:45.708 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-16T09:18:45.708 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-16T09:18:45.709 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-16T09:18:45.709 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:45.709 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-16T09:18:45.709 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 847, 2024-03-16T09:18:45.709 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 847, 2024-03-16T09:18:45.709 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 847, 2024-03-16T09:18:45.709 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 847, 2024-03-16T09:18:45.709 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-16T09:18:45.709 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.709 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.709 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.709 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:45.709 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.709 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-03-16T09:18:45.709 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-16T09:18:45.710 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-16T09:18:45.710 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-16T09:18:45.710 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-16T09:18:45.710 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-16T09:18:45.710 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:45.710 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-16T09:18:45.710 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-16T09:18:45.710 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-16T09:18:45.710 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2024-03-16T09:18:45.710 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2024-03-16T09:18:45.710 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-16T09:18:45.710 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.710 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.710 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.711 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:45.711 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:45.711 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x5610d37206e0": { 2024-03-16T09:18:45.711 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-16T09:18:45.711 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-16T09:18:45.711 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-16T09:18:45.711 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-16T09:18:45.711 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-16T09:18:45.711 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:45.711 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-16T09:18:45.711 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-16T09:18:45.711 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-16T09:18:45.712 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2024-03-16T09:18:45.712 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2024-03-16T09:18:45.712 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-16T09:18:45.712 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:45.712 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:45.712 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:45.712 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:45.712 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:45.712 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-03-16T09:18:46.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:46 smithi112 ceph-mon[32307]: pgmap v445: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-03-16T09:18:46.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:46 smithi194 ceph-mon[36729]: pgmap v445: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-03-16T09:18:46.438 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:46 smithi159 ceph-mon[36739]: pgmap v445: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-03-16T09:18:46.471 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-16T09:18:46.472 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-16T09:18:46.472 DEBUG:tasks.check_counter:Getting stats from h 2024-03-16T09:18:46.472 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-16T09:18:46.725 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T09:18:48.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:48 smithi159 ceph-mon[36739]: pgmap v446: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 6 op/s 2024-03-16T09:18:48.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:48 smithi112 ceph-mon[32307]: pgmap v446: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 6 op/s 2024-03-16T09:18:48.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:48 smithi194 ceph-mon[36729]: pgmap v446: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 6 op/s 2024-03-16T09:18:48.460 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-03-16T09:18:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T09:18:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 558, 2024-03-16T09:18:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 512, 2024-03-16T09:18:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 182929, 2024-03-16T09:18:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 1563105, 2024-03-16T09:18:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 17, 2024-03-16T09:18:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2024-03-16T09:18:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.039410614, 2024-03-16T09:18:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.014909601, 2024-03-16T09:18:48.460 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.018761862, 2024-03-16T09:18:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.005615560, 2024-03-16T09:18:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 511, 2024-03-16T09:18:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.029449506, 2024-03-16T09:18:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000057631 2024-03-16T09:18:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 623, 2024-03-16T09:18:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000082563, 2024-03-16T09:18:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000132 2024-03-16T09:18:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 5536, 2024-03-16T09:18:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 3920 2024-03-16T09:18:48.461 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T09:18:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 957, 2024-03-16T09:18:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 1501, 2024-03-16T09:18:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 1029003, 2024-03-16T09:18:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 738871, 2024-03-16T09:18:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 7, 2024-03-16T09:18:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 6, 2024-03-16T09:18:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.065578109, 2024-03-16T09:18:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.025335603, 2024-03-16T09:18:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.046019371, 2024-03-16T09:18:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.005065050, 2024-03-16T09:18:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:48.462 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1500, 2024-03-16T09:18:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.092677425, 2024-03-16T09:18:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000061784 2024-03-16T09:18:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1003, 2024-03-16T09:18:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000268233, 2024-03-16T09:18:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000267 2024-03-16T09:18:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 776000, 2024-03-16T09:18:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 74144 2024-03-16T09:18:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.463 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T09:18:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 2242, 2024-03-16T09:18:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 1789, 2024-03-16T09:18:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 853588, 2024-03-16T09:18:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 734343, 2024-03-16T09:18:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 14, 2024-03-16T09:18:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 6, 2024-03-16T09:18:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.137588542, 2024-03-16T09:18:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.060933776, 2024-03-16T09:18:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.090787646, 2024-03-16T09:18:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.003940667, 2024-03-16T09:18:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1787, 2024-03-16T09:18:48.464 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.072320222, 2024-03-16T09:18:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000040470 2024-03-16T09:18:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3607, 2024-03-16T09:18:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000592129, 2024-03-16T09:18:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000164 2024-03-16T09:18:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-16T09:18:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 197776 2024-03-16T09:18:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-03-16T09:18:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-03-16T09:18:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-03-16T09:18:48.465 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-03-16T09:18:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-16T09:18:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-16T09:18:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 376, 2024-03-16T09:18:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.306507077, 2024-03-16T09:18:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000815178 2024-03-16T09:18:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-03-16T09:18:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-16T09:18:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-16T09:18:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 337, 2024-03-16T09:18:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.018717827, 2024-03-16T09:18:48.466 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000055542 2024-03-16T09:18:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-03-16T09:18:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 360, 2024-03-16T09:18:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 357, 2024-03-16T09:18:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-03-16T09:18:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 357, 2024-03-16T09:18:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.686153300, 2024-03-16T09:18:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001921998 2024-03-16T09:18:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-03-16T09:18:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 3, 2024-03-16T09:18:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-03-16T09:18:48.467 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-03-16T09:18:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 11, 2024-03-16T09:18:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2024-03-16T09:18:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2024-03-16T09:18:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 22, 2024-03-16T09:18:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 6, 2024-03-16T09:18:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 6, 2024-03-16T09:18:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 10, 2024-03-16T09:18:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 13, 2024-03-16T09:18:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 67, 2024-03-16T09:18:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 0, 2024-03-16T09:18:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 0, 2024-03-16T09:18:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 3, 2024-03-16T09:18:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 775, 2024-03-16T09:18:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 465, 2024-03-16T09:18:48.468 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2024-03-16T09:18:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 23, 2024-03-16T09:18:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-03-16T09:18:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-03-16T09:18:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2024-03-16T09:18:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-03-16T09:18:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-03-16T09:18:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 15, 2024-03-16T09:18:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 2, 2024-03-16T09:18:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 15, 2024-03-16T09:18:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 111, 2024-03-16T09:18:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-03-16T09:18:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 65, 2024-03-16T09:18:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 1, 2024-03-16T09:18:48.469 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 620, 2024-03-16T09:18:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 3951007, 2024-03-16T09:18:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2024-03-16T09:18:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 65, 2024-03-16T09:18:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-03-16T09:18:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T09:18:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-03-16T09:18:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2024-03-16T09:18:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T09:18:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2024-03-16T09:18:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 65, 2024-03-16T09:18:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2024-03-16T09:18:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 34, 2024-03-16T09:18:48.470 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 266, 2024-03-16T09:18:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T09:18:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T09:18:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T09:18:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 217, 2024-03-16T09:18:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 138, 2024-03-16T09:18:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 15, 2024-03-16T09:18:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 354 2024-03-16T09:18:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-03-16T09:18:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2024-03-16T09:18:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2024-03-16T09:18:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2024-03-16T09:18:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 18, 2024-03-16T09:18:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 3, 2024-03-16T09:18:48.471 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-03-16T09:18:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-03-16T09:18:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-03-16T09:18:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 176, 2024-03-16T09:18:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 176, 2024-03-16T09:18:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-03-16T09:18:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-03-16T09:18:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-03-16T09:18:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-03-16T09:18:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-03-16T09:18:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-03-16T09:18:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-03-16T09:18:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_path": 0, 2024-03-16T09:18:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_inode": 0, 2024-03-16T09:18:48.472 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T09:18:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 15, 2024-03-16T09:18:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-03-16T09:18:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2024-03-16T09:18:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-03-16T09:18:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-03-16T09:18:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-03-16T09:18:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-03-16T09:18:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 807, 2024-03-16T09:18:48.473 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2024-03-16T09:18:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2024-03-16T09:18:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 807, 2024-03-16T09:18:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-03-16T09:18:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2024-03-16T09:18:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 2, 2024-03-16T09:18:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2024-03-16T09:18:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2024-03-16T09:18:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 2, 2024-03-16T09:18:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 2, 2024-03-16T09:18:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-03-16T09:18:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2024-03-16T09:18:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2024-03-16T09:18:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 5522835, 2024-03-16T09:18:48.474 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2024-03-16T09:18:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-03-16T09:18:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 113, 2024-03-16T09:18:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.257693667, 2024-03-16T09:18:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002280474 2024-03-16T09:18:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2024-03-16T09:18:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-03-16T09:18:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 27, 2024-03-16T09:18:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 237, 2024-03-16T09:18:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 210, 2024-03-16T09:18:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 25, 2024-03-16T09:18:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 71, 2024-03-16T09:18:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 46, 2024-03-16T09:18:48.475 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 22, 2024-03-16T09:18:48.476 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 374, 2024-03-16T09:18:48.476 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 352, 2024-03-16T09:18:48.476 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 0, 2024-03-16T09:18:48.476 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 194, 2024-03-16T09:18:48.476 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 194, 2024-03-16T09:18:48.476 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 54912, 2024-03-16T09:18:48.476 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 223612 2024-03-16T09:18:48.476 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.476 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-03-16T09:18:48.476 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 664, 2024-03-16T09:18:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 16, 2024-03-16T09:18:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 360, 2024-03-16T09:18:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 7, 2024-03-16T09:18:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 282, 2024-03-16T09:18:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-03-16T09:18:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 65, 2024-03-16T09:18:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.108330764, 2024-03-16T09:18:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001666627 2024-03-16T09:18:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-03-16T09:18:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-03-16T09:18:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.001305522, 2024-03-16T09:18:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001305522 2024-03-16T09:18:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.477 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-03-16T09:18:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-03-16T09:18:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-03-16T09:18:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 91, 2024-03-16T09:18:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.036791910, 2024-03-16T09:18:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000404306 2024-03-16T09:18:48.478 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-03-16T09:18:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-03-16T09:18:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-03-16T09:18:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.479 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-03-16T09:18:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-03-16T09:18:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-03-16T09:18:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.480 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-03-16T09:18:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.481 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-03-16T09:18:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-03-16T09:18:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-03-16T09:18:48.482 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-03-16T09:18:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 24, 2024-03-16T09:18:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.013013831, 2024-03-16T09:18:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000542242 2024-03-16T09:18:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-03-16T09:18:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.483 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-03-16T09:18:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-03-16T09:18:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-03-16T09:18:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-03-16T09:18:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.027695574, 2024-03-16T09:18:48.484 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.027695574 2024-03-16T09:18:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-03-16T09:18:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-03-16T09:18:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-03-16T09:18:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.485 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-03-16T09:18:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-03-16T09:18:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.486 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-03-16T09:18:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-03-16T09:18:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-03-16T09:18:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.487 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-03-16T09:18:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 175, 2024-03-16T09:18:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.499015699, 2024-03-16T09:18:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002851518 2024-03-16T09:18:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-03-16T09:18:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-03-16T09:18:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-03-16T09:18:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-03-16T09:18:48.488 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2024-03-16T09:18:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 2, 2024-03-16T09:18:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 1, 2024-03-16T09:18:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 1, 2024-03-16T09:18:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-03-16T09:18:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 230, 2024-03-16T09:18:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 230, 2024-03-16T09:18:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 182, 2024-03-16T09:18:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-03-16T09:18:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-03-16T09:18:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-03-16T09:18:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-03-16T09:18:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T09:18:48.489 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-03-16T09:18:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T09:18:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-03-16T09:18:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T09:18:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T09:18:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T09:18:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T09:18:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T09:18:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-03-16T09:18:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T09:18:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T09:18:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-03-16T09:18:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-03-16T09:18:48.490 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-03-16T09:18:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-03-16T09:18:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T09:18:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T09:18:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T09:18:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T09:18:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T09:18:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-03-16T09:18:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-03-16T09:18:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-03-16T09:18:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T09:18:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T09:18:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-03-16T09:18:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-03-16T09:18:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-03-16T09:18:48.491 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-03-16T09:18:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T09:18:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-03-16T09:18:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T09:18:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-03-16T09:18:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 290841, 2024-03-16T09:18:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 120, 2024-03-16T09:18:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 704, 2024-03-16T09:18:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 8, 2024-03-16T09:18:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-03-16T09:18:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-03-16T09:18:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-03-16T09:18:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-03-16T09:18:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-03-16T09:18:48.492 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-03-16T09:18:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 7528, 2024-03-16T09:18:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 158, 2024-03-16T09:18:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T09:18:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-03-16T09:18:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-03-16T09:18:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-03-16T09:18:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 108093, 2024-03-16T09:18:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 431, 2024-03-16T09:18:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-03-16T09:18:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-03-16T09:18:48.493 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-03-16T09:18:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-03-16T09:18:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-03-16T09:18:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-03-16T09:18:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-03-16T09:18:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 506, 2024-03-16T09:18:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 1388323, 2024-03-16T09:18:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 28, 2024-03-16T09:18:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 506, 2024-03-16T09:18:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-03-16T09:18:48.494 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 506, 2024-03-16T09:18:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1.419997195, 2024-03-16T09:18:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002806318 2024-03-16T09:18:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-03-16T09:18:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-03-16T09:18:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 506, 2024-03-16T09:18:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 574 2024-03-16T09:18:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 506, 2024-03-16T09:18:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 91, 2024-03-16T09:18:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 415, 2024-03-16T09:18:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-03-16T09:18:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-03-16T09:18:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 0, 2024-03-16T09:18:48.495 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 10, 2024-03-16T09:18:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 26, 2024-03-16T09:18:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 139, 2024-03-16T09:18:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 27, 2024-03-16T09:18:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-03-16T09:18:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-03-16T09:18:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-03-16T09:18:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2024-03-16T09:18:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 188, 2024-03-16T09:18:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-03-16T09:18:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-03-16T09:18:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-03-16T09:18:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 65, 2024-03-16T09:18:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 20, 2024-03-16T09:18:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-03-16T09:18:48.496 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-03-16T09:18:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-03-16T09:18:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-03-16T09:18:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-03-16T09:18:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-03-16T09:18:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T09:18:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-03-16T09:18:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-03-16T09:18:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-03-16T09:18:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-03-16T09:18:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-03-16T09:18:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-03-16T09:18:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-03-16T09:18:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-03-16T09:18:48.497 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-03-16T09:18:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-03-16T09:18:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-03-16T09:18:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-03-16T09:18:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-03-16T09:18:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-03-16T09:18:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-03-16T09:18:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-03-16T09:18:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-03-16T09:18:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-03-16T09:18:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-03-16T09:18:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 88, 2024-03-16T09:18:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-03-16T09:18:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 3, 2024-03-16T09:18:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 12, 2024-03-16T09:18:48.498 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 12, 2024-03-16T09:18:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2024-03-16T09:18:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-03-16T09:18:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 76, 2024-03-16T09:18:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 0, 2024-03-16T09:18:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 23 2024-03-16T09:18:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-03-16T09:18:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2024-03-16T09:18:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-03-16T09:18:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-03-16T09:18:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-03-16T09:18:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-03-16T09:18:48.499 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2024-03-16T09:18:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-03-16T09:18:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-03-16T09:18:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 10, 2024-03-16T09:18:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-03-16T09:18:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 10, 2024-03-16T09:18:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 176, 2024-03-16T09:18:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 176, 2024-03-16T09:18:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-03-16T09:18:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T09:18:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-16T09:18:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-16T09:18:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-16T09:18:48.500 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 3753, 2024-03-16T09:18:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 1727693, 2024-03-16T09:18:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 3753, 2024-03-16T09:18:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-16T09:18:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-16T09:18:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 3753, 2024-03-16T09:18:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1727693, 2024-03-16T09:18:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-16T09:18:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-03-16T09:18:48.501 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-16T09:18:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-16T09:18:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-16T09:18:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-16T09:18:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-16T09:18:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-16T09:18:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 506, 2024-03-16T09:18:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 1406099, 2024-03-16T09:18:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 506, 2024-03-16T09:18:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1406099, 2024-03-16T09:18:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-16T09:18:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.502 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-03-16T09:18:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-16T09:18:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-03-16T09:18:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-16T09:18:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-16T09:18:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-16T09:18:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-16T09:18:48.503 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 506, 2024-03-16T09:18:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 506, 2024-03-16T09:18:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 506, 2024-03-16T09:18:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 506, 2024-03-16T09:18:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-16T09:18:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-03-16T09:18:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-16T09:18:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-16T09:18:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-16T09:18:48.504 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 176, 2024-03-16T09:18:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 17776, 2024-03-16T09:18:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 176, 2024-03-16T09:18:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-16T09:18:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-16T09:18:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 26, 2024-03-16T09:18:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 17776, 2024-03-16T09:18:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-16T09:18:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x55db56464500": { 2024-03-16T09:18:48.505 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-16T09:18:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-16T09:18:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-16T09:18:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 807, 2024-03-16T09:18:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 1328531, 2024-03-16T09:18:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 807, 2024-03-16T09:18:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-16T09:18:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-16T09:18:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 113, 2024-03-16T09:18:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1328531, 2024-03-16T09:18:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-16T09:18:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:48.506 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:48.507 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:48.507 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:48.507 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-03-16T09:18:49.258 INFO:tasks.check_counter:Daemon mds.h mds.exported=15 2024-03-16T09:18:49.258 INFO:tasks.check_counter:Daemon mds.h mds.imported=15 2024-03-16T09:18:49.258 DEBUG:tasks.check_counter:Getting stats from k 2024-03-16T09:18:49.259 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-16T09:18:49.520 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.b/config 2024-03-16T09:18:50.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:50 smithi159 ceph-mon[36739]: pgmap v447: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-03-16T09:18:50.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:50 smithi112 ceph-mon[32307]: pgmap v447: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-03-16T09:18:50.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:50 smithi194 ceph-mon[36729]: pgmap v447: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s 2024-03-16T09:18:51.257 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-03-16T09:18:51.257 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T09:18:51.258 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 102, 2024-03-16T09:18:51.258 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 97, 2024-03-16T09:18:51.258 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 717072, 2024-03-16T09:18:51.258 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 58592, 2024-03-16T09:18:51.258 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 4, 2024-03-16T09:18:51.258 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 0, 2024-03-16T09:18:51.258 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.016769899, 2024-03-16T09:18:51.258 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.003936290, 2024-03-16T09:18:51.258 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.015361075, 2024-03-16T09:18:51.258 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.001699356, 2024-03-16T09:18:51.258 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:51.258 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 96, 2024-03-16T09:18:51.259 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.002979532, 2024-03-16T09:18:51.259 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000031036 2024-03-16T09:18:51.259 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:51.259 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:51.259 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:51.259 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:51.259 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:51.259 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:51.259 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 717072, 2024-03-16T09:18:51.259 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 58592 2024-03-16T09:18:51.259 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:51.259 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T09:18:51.259 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1, 2024-03-16T09:18:51.259 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 73, 2024-03-16T09:18:51.259 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 160, 2024-03-16T09:18:51.260 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 90560, 2024-03-16T09:18:51.260 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 3, 2024-03-16T09:18:51.260 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 2, 2024-03-16T09:18:51.260 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.003669978, 2024-03-16T09:18:51.260 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.002204942, 2024-03-16T09:18:51.260 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.000887991, 2024-03-16T09:18:51.260 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T09:18:51.260 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:51.260 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 71, 2024-03-16T09:18:51.260 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.002460912, 2024-03-16T09:18:51.260 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000034660 2024-03-16T09:18:51.260 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:51.260 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:51.260 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:51.260 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:51.260 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:51.261 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:51.261 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-16T09:18:51.261 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 90560 2024-03-16T09:18:51.261 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:51.261 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T09:18:51.261 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 4, 2024-03-16T09:18:51.261 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 4, 2024-03-16T09:18:51.261 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 3024, 2024-03-16T09:18:51.261 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 448, 2024-03-16T09:18:51.261 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 2, 2024-03-16T09:18:51.261 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 1, 2024-03-16T09:18:51.261 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.000744526, 2024-03-16T09:18:51.261 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.000035569, 2024-03-16T09:18:51.261 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.000622216, 2024-03-16T09:18:51.261 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T09:18:51.262 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:51.262 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3, 2024-03-16T09:18:51.262 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000743926, 2024-03-16T09:18:51.262 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000247975 2024-03-16T09:18:51.262 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:51.262 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:51.262 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:51.262 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:51.262 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:51.262 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:51.262 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-16T09:18:51.262 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 448 2024-03-16T09:18:51.262 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:51.262 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-03-16T09:18:51.262 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 0, 2024-03-16T09:18:51.263 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-03-16T09:18:51.263 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:51.263 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-03-16T09:18:51.263 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-03-16T09:18:51.263 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-03-16T09:18:51.263 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T09:18:51.263 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-03-16T09:18:51.263 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T09:18:51.263 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-03-16T09:18:51.263 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T09:18:51.263 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T09:18:51.263 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T09:18:51.263 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T09:18:51.263 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T09:18:51.264 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-03-16T09:18:51.264 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T09:18:51.264 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T09:18:51.264 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-03-16T09:18:51.264 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-03-16T09:18:51.264 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-03-16T09:18:51.264 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-03-16T09:18:51.264 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T09:18:51.264 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T09:18:51.264 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T09:18:51.264 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T09:18:51.264 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T09:18:51.264 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-03-16T09:18:51.264 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-03-16T09:18:51.265 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-03-16T09:18:51.265 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T09:18:51.265 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T09:18:51.265 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-03-16T09:18:51.265 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-03-16T09:18:51.265 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-03-16T09:18:51.265 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-03-16T09:18:51.265 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T09:18:51.265 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-03-16T09:18:51.265 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T09:18:51.265 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-03-16T09:18:51.265 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 148823, 2024-03-16T09:18:51.265 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 27, 2024-03-16T09:18:51.265 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-03-16T09:18:51.265 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-03-16T09:18:51.265 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-03-16T09:18:51.266 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-03-16T09:18:51.266 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-03-16T09:18:51.266 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-03-16T09:18:51.267 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-03-16T09:18:51.267 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-03-16T09:18:51.267 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 0, 2024-03-16T09:18:51.267 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 0, 2024-03-16T09:18:51.267 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T09:18:51.267 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-03-16T09:18:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-03-16T09:18:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-03-16T09:18:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 928, 2024-03-16T09:18:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 2, 2024-03-16T09:18:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-03-16T09:18:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-03-16T09:18:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-03-16T09:18:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-03-16T09:18:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-16T09:18:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T09:18:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-16T09:18:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-16T09:18:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-16T09:18:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 103, 2024-03-16T09:18:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 702730, 2024-03-16T09:18:51.268 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:51.269 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 103, 2024-03-16T09:18:51.269 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-16T09:18:51.269 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-16T09:18:51.269 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 103, 2024-03-16T09:18:51.269 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 702730, 2024-03-16T09:18:51.269 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-16T09:18:51.269 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-16T09:18:51.269 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-16T09:18:51.269 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-16T09:18:51.269 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:51.269 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-16T09:18:51.269 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-03-16T09:18:51.972 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-03-16T09:18:51.972 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-03-16T09:18:51.972 DEBUG:tasks.check_counter:Getting stats from c 2024-03-16T09:18:51.972 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-16T09:18:52.221 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:18:52.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:52 smithi159 ceph-mon[36739]: pgmap v448: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-16T09:18:52.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:52 smithi112 ceph-mon[32307]: pgmap v448: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-16T09:18:52.345 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:52 smithi194 ceph-mon[36729]: pgmap v448: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-16T09:18:54.091 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-03-16T09:18:54.091 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T09:18:54.091 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 184, 2024-03-16T09:18:54.091 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 186, 2024-03-16T09:18:54.091 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 25220581, 2024-03-16T09:18:54.092 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 58522, 2024-03-16T09:18:54.092 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 8, 2024-03-16T09:18:54.092 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 4, 2024-03-16T09:18:54.092 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.026909557, 2024-03-16T09:18:54.092 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.003741971, 2024-03-16T09:18:54.092 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.016289201, 2024-03-16T09:18:54.092 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.004651637, 2024-03-16T09:18:54.092 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:54.092 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 184, 2024-03-16T09:18:54.092 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.006853516, 2024-03-16T09:18:54.092 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000037247 2024-03-16T09:18:54.092 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.092 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:54.093 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.093 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.093 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.093 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.093 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-16T09:18:54.093 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 0 2024-03-16T09:18:54.093 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.093 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T09:18:54.093 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 134, 2024-03-16T09:18:54.093 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 154, 2024-03-16T09:18:54.093 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 21895316, 2024-03-16T09:18:54.093 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 160520, 2024-03-16T09:18:54.093 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 6, 2024-03-16T09:18:54.093 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 5, 2024-03-16T09:18:54.093 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.022131126, 2024-03-16T09:18:54.094 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.003024384, 2024-03-16T09:18:54.094 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.015406945, 2024-03-16T09:18:54.094 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.002767508, 2024-03-16T09:18:54.094 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:54.094 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 152, 2024-03-16T09:18:54.094 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.006090175, 2024-03-16T09:18:54.094 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000040066 2024-03-16T09:18:54.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.094 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:54.094 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.094 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.094 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.094 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.095 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-16T09:18:54.095 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 118160 2024-03-16T09:18:54.095 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.095 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T09:18:54.095 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 93, 2024-03-16T09:18:54.095 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 87, 2024-03-16T09:18:54.095 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 21161971, 2024-03-16T09:18:54.095 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 33197, 2024-03-16T09:18:54.095 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 5, 2024-03-16T09:18:54.095 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 4, 2024-03-16T09:18:54.095 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.016064287, 2024-03-16T09:18:54.095 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.002038513, 2024-03-16T09:18:54.095 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.013363931, 2024-03-16T09:18:54.095 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.001438359, 2024-03-16T09:18:54.096 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:54.096 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 87, 2024-03-16T09:18:54.096 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.004806769, 2024-03-16T09:18:54.096 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000055250 2024-03-16T09:18:54.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.096 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:54.096 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.096 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.096 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.096 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 177552, 2024-03-16T09:18:54.096 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 17296 2024-03-16T09:18:54.096 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.096 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-03-16T09:18:54.097 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2024-03-16T09:18:54.097 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-03-16T09:18:54.097 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.097 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2024-03-16T09:18:54.097 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-03-16T09:18:54.097 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-03-16T09:18:54.097 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 571, 2024-03-16T09:18:54.097 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.063089534, 2024-03-16T09:18:54.097 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000110489 2024-03-16T09:18:54.097 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:54.097 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.097 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2024-03-16T09:18:54.097 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-03-16T09:18:54.097 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-03-16T09:18:54.098 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.098 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.098 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.098 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:54.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.098 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2024-03-16T09:18:54.098 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 0, 2024-03-16T09:18:54.098 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 0, 2024-03-16T09:18:54.098 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2024-03-16T09:18:54.098 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.098 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.098 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.098 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.098 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2024-03-16T09:18:54.099 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 0, 2024-03-16T09:18:54.099 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 0, 2024-03-16T09:18:54.099 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2024-03-16T09:18:54.099 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 0, 2024-03-16T09:18:54.099 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 0, 2024-03-16T09:18:54.099 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 0, 2024-03-16T09:18:54.099 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 8234, 2024-03-16T09:18:54.099 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 5227, 2024-03-16T09:18:54.099 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 3007, 2024-03-16T09:18:54.099 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 0, 2024-03-16T09:18:54.099 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 766, 2024-03-16T09:18:54.099 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 0, 2024-03-16T09:18:54.099 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 0, 2024-03-16T09:18:54.099 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 0, 2024-03-16T09:18:54.099 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 2, 2024-03-16T09:18:54.100 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 0, 2024-03-16T09:18:54.100 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 0, 2024-03-16T09:18:54.100 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 0, 2024-03-16T09:18:54.100 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 0, 2024-03-16T09:18:54.100 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2024-03-16T09:18:54.100 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2024-03-16T09:18:54.100 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 0, 2024-03-16T09:18:54.100 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 0, 2024-03-16T09:18:54.100 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2024-03-16T09:18:54.100 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 0, 2024-03-16T09:18:54.100 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 0, 2024-03-16T09:18:54.100 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 0, 2024-03-16T09:18:54.100 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 0, 2024-03-16T09:18:54.100 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2024-03-16T09:18:54.100 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 0, 2024-03-16T09:18:54.101 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2024-03-16T09:18:54.101 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 4, 2024-03-16T09:18:54.101 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 234, 2024-03-16T09:18:54.101 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 0, 2024-03-16T09:18:54.101 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 0, 2024-03-16T09:18:54.101 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2024-03-16T09:18:54.101 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T09:18:54.101 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2024-03-16T09:18:54.101 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 0, 2024-03-16T09:18:54.101 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T09:18:54.101 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 0, 2024-03-16T09:18:54.101 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 0, 2024-03-16T09:18:54.101 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 0, 2024-03-16T09:18:54.101 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 0, 2024-03-16T09:18:54.102 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 0, 2024-03-16T09:18:54.102 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T09:18:54.102 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T09:18:54.102 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T09:18:54.102 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 0, 2024-03-16T09:18:54.102 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 0, 2024-03-16T09:18:54.102 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 0, 2024-03-16T09:18:54.102 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 0 2024-03-16T09:18:54.102 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.102 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2024-03-16T09:18:54.102 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 0, 2024-03-16T09:18:54.102 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 0, 2024-03-16T09:18:54.102 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 0, 2024-03-16T09:18:54.102 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 0, 2024-03-16T09:18:54.102 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 0, 2024-03-16T09:18:54.103 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 0, 2024-03-16T09:18:54.103 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2024-03-16T09:18:54.103 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2024-03-16T09:18:54.103 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 0, 2024-03-16T09:18:54.103 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 0, 2024-03-16T09:18:54.103 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2024-03-16T09:18:54.103 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2024-03-16T09:18:54.103 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2024-03-16T09:18:54.103 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2024-03-16T09:18:54.103 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2024-03-16T09:18:54.103 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2024-03-16T09:18:54.103 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2024-03-16T09:18:54.103 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_path": 0, 2024-03-16T09:18:54.103 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_inode": 0, 2024-03-16T09:18:54.104 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T09:18:54.104 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 0, 2024-03-16T09:18:54.104 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2024-03-16T09:18:54.104 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 0, 2024-03-16T09:18:54.104 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2024-03-16T09:18:54.104 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2024-03-16T09:18:54.104 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.104 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2024-03-16T09:18:54.104 INFO:teuthology.orchestra.run.smithi194.stdout: "evlrg": 0, 2024-03-16T09:18:54.104 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 0, 2024-03-16T09:18:54.104 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 0, 2024-03-16T09:18:54.104 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 0, 2024-03-16T09:18:54.104 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 35108, 2024-03-16T09:18:54.104 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2024-03-16T09:18:54.104 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 0, 2024-03-16T09:18:54.105 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 0, 2024-03-16T09:18:54.105 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 0, 2024-03-16T09:18:54.105 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 0, 2024-03-16T09:18:54.105 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 3, 2024-03-16T09:18:54.105 INFO:teuthology.orchestra.run.smithi194.stdout: "segmjr": 5, 2024-03-16T09:18:54.105 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2024-03-16T09:18:54.105 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 0, 2024-03-16T09:18:54.105 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 83679489, 2024-03-16T09:18:54.105 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 85679383, 2024-03-16T09:18:54.105 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 85677396, 2024-03-16T09:18:54.105 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2024-03-16T09:18:54.105 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.105 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.105 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.106 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 35108 2024-03-16T09:18:54.106 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.106 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2024-03-16T09:18:54.106 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 23, 2024-03-16T09:18:54.106 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 8429, 2024-03-16T09:18:54.106 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 8406, 2024-03-16T09:18:54.106 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 22, 2024-03-16T09:18:54.106 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 613, 2024-03-16T09:18:54.106 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 591, 2024-03-16T09:18:54.106 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 8234, 2024-03-16T09:18:54.106 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 14488, 2024-03-16T09:18:54.106 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 6254, 2024-03-16T09:18:54.106 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 0, 2024-03-16T09:18:54.107 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 0, 2024-03-16T09:18:54.107 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 0, 2024-03-16T09:18:54.107 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 59608, 2024-03-16T09:18:54.107 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 215396 2024-03-16T09:18:54.107 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.107 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2024-03-16T09:18:54.107 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 0, 2024-03-16T09:18:54.107 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 0, 2024-03-16T09:18:54.107 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 0, 2024-03-16T09:18:54.107 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 0, 2024-03-16T09:18:54.107 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 0, 2024-03-16T09:18:54.107 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2024-03-16T09:18:54.107 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.107 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.107 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.107 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.108 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2024-03-16T09:18:54.108 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.108 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.108 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.108 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2024-03-16T09:18:54.108 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.108 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.108 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.108 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.108 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2024-03-16T09:18:54.108 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.108 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.108 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.109 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.109 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2024-03-16T09:18:54.109 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.109 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.109 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.109 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.109 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2024-03-16T09:18:54.109 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.109 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.109 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.109 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.109 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2024-03-16T09:18:54.109 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.109 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.110 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.110 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2024-03-16T09:18:54.110 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.110 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.110 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.110 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2024-03-16T09:18:54.110 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.110 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.110 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.110 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.110 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2024-03-16T09:18:54.110 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.111 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.111 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.111 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2024-03-16T09:18:54.111 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.111 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.111 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.111 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.111 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2024-03-16T09:18:54.112 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.112 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.112 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.112 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2024-03-16T09:18:54.112 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.112 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.112 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.112 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.112 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2024-03-16T09:18:54.112 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.112 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.113 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.113 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2024-03-16T09:18:54.113 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.113 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.113 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.113 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2024-03-16T09:18:54.113 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.113 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.113 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.113 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.113 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2024-03-16T09:18:54.113 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.114 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.114 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.114 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2024-03-16T09:18:54.114 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.114 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.114 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.114 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.114 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2024-03-16T09:18:54.114 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.114 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.114 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.115 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2024-03-16T09:18:54.115 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.115 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.115 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.115 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2024-03-16T09:18:54.115 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.115 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.115 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.115 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.115 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2024-03-16T09:18:54.115 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.115 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.115 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.116 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2024-03-16T09:18:54.116 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.116 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.116 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.116 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2024-03-16T09:18:54.116 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.116 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.116 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.116 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.116 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2024-03-16T09:18:54.116 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.116 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.116 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.117 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2024-03-16T09:18:54.117 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.117 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.117 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.117 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2024-03-16T09:18:54.117 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.117 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.117 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.117 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.117 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2024-03-16T09:18:54.117 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.117 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.117 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.118 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2024-03-16T09:18:54.118 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.118 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.118 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.118 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2024-03-16T09:18:54.118 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2024-03-16T09:18:54.118 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2024-03-16T09:18:54.118 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.118 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.118 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.118 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:54.118 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.119 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2024-03-16T09:18:54.119 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 1, 2024-03-16T09:18:54.119 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 1, 2024-03-16T09:18:54.119 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 0, 2024-03-16T09:18:54.119 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 0, 2024-03-16T09:18:54.119 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2024-03-16T09:18:54.119 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 0, 2024-03-16T09:18:54.119 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 0, 2024-03-16T09:18:54.119 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 1611, 2024-03-16T09:18:54.119 INFO:teuthology.orchestra.run.smithi194.stdout: "mdthresh_evicted": 0 2024-03-16T09:18:54.119 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.119 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-03-16T09:18:54.119 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-03-16T09:18:54.119 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-03-16T09:18:54.119 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T09:18:54.120 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-03-16T09:18:54.120 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T09:18:54.120 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-03-16T09:18:54.120 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T09:18:54.120 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T09:18:54.120 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T09:18:54.120 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T09:18:54.120 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T09:18:54.120 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-03-16T09:18:54.120 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T09:18:54.120 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T09:18:54.120 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-03-16T09:18:54.120 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-03-16T09:18:54.120 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-03-16T09:18:54.121 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-03-16T09:18:54.121 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T09:18:54.121 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T09:18:54.121 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T09:18:54.121 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T09:18:54.121 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T09:18:54.121 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-03-16T09:18:54.121 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-03-16T09:18:54.121 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-03-16T09:18:54.121 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T09:18:54.121 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T09:18:54.121 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-03-16T09:18:54.121 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-03-16T09:18:54.121 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-03-16T09:18:54.122 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-03-16T09:18:54.122 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T09:18:54.122 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-03-16T09:18:54.122 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T09:18:54.122 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-03-16T09:18:54.122 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 229095, 2024-03-16T09:18:54.122 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 73, 2024-03-16T09:18:54.122 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 0, 2024-03-16T09:18:54.122 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 0, 2024-03-16T09:18:54.122 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-03-16T09:18:54.122 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-03-16T09:18:54.122 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-03-16T09:18:54.122 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-03-16T09:18:54.122 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-03-16T09:18:54.122 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-03-16T09:18:54.123 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 7528, 2024-03-16T09:18:54.123 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 158, 2024-03-16T09:18:54.123 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T09:18:54.123 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-03-16T09:18:54.123 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-03-16T09:18:54.123 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-03-16T09:18:54.123 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 5878508, 2024-03-16T09:18:54.123 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 34166, 2024-03-16T09:18:54.123 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-03-16T09:18:54.123 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-03-16T09:18:54.123 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-03-16T09:18:54.123 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-03-16T09:18:54.123 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.124 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2024-03-16T09:18:54.124 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2024-03-16T09:18:54.124 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2024-03-16T09:18:54.124 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 367, 2024-03-16T09:18:54.124 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 26, 2024-03-16T09:18:54.124 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 4, 2024-03-16T09:18:54.124 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 367, 2024-03-16T09:18:54.124 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2024-03-16T09:18:54.124 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 367, 2024-03-16T09:18:54.124 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 2.994936304, 2024-03-16T09:18:54.124 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.008160589 2024-03-16T09:18:54.124 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.124 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2024-03-16T09:18:54.124 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2024-03-16T09:18:54.125 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 367, 2024-03-16T09:18:54.125 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 368 2024-03-16T09:18:54.125 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.125 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 367, 2024-03-16T09:18:54.125 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 367, 2024-03-16T09:18:54.125 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 0, 2024-03-16T09:18:54.125 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2024-03-16T09:18:54.125 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2024-03-16T09:18:54.125 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 209, 2024-03-16T09:18:54.125 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 0, 2024-03-16T09:18:54.125 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 155, 2024-03-16T09:18:54.125 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 0, 2024-03-16T09:18:54.125 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 0, 2024-03-16T09:18:54.125 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2024-03-16T09:18:54.126 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2024-03-16T09:18:54.126 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2024-03-16T09:18:54.126 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 0, 2024-03-16T09:18:54.126 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 0, 2024-03-16T09:18:54.126 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2024-03-16T09:18:54.126 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2024-03-16T09:18:54.126 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2024-03-16T09:18:54.126 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 2, 2024-03-16T09:18:54.126 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 0, 2024-03-16T09:18:54.126 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2024-03-16T09:18:54.126 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2024-03-16T09:18:54.126 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2024-03-16T09:18:54.126 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2024-03-16T09:18:54.127 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2024-03-16T09:18:54.127 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2024-03-16T09:18:54.127 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T09:18:54.127 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2024-03-16T09:18:54.127 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2024-03-16T09:18:54.127 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2024-03-16T09:18:54.127 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2024-03-16T09:18:54.127 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2024-03-16T09:18:54.127 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2024-03-16T09:18:54.127 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2024-03-16T09:18:54.127 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2024-03-16T09:18:54.127 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2024-03-16T09:18:54.127 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2024-03-16T09:18:54.127 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2024-03-16T09:18:54.127 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2024-03-16T09:18:54.128 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2024-03-16T09:18:54.128 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2024-03-16T09:18:54.128 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2024-03-16T09:18:54.128 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2024-03-16T09:18:54.128 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2024-03-16T09:18:54.128 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2024-03-16T09:18:54.128 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2024-03-16T09:18:54.128 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 0, 2024-03-16T09:18:54.128 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2024-03-16T09:18:54.128 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 0, 2024-03-16T09:18:54.128 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 10, 2024-03-16T09:18:54.128 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 10, 2024-03-16T09:18:54.128 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 0, 2024-03-16T09:18:54.128 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2024-03-16T09:18:54.129 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 0, 2024-03-16T09:18:54.129 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 2, 2024-03-16T09:18:54.129 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 0 2024-03-16T09:18:54.129 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.129 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2024-03-16T09:18:54.129 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 0, 2024-03-16T09:18:54.129 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2024-03-16T09:18:54.129 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2024-03-16T09:18:54.129 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2024-03-16T09:18:54.129 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.129 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2024-03-16T09:18:54.129 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 0 2024-03-16T09:18:54.129 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.130 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2024-03-16T09:18:54.130 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2024-03-16T09:18:54.130 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 0, 2024-03-16T09:18:54.130 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2024-03-16T09:18:54.130 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 0, 2024-03-16T09:18:54.130 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed_ops": 0, 2024-03-16T09:18:54.130 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 0, 2024-03-16T09:18:54.130 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2024-03-16T09:18:54.130 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.130 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T09:18:54.130 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T09:18:54.130 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-03-16T09:18:54.130 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T09:18:54.130 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 407, 2024-03-16T09:18:54.131 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 68236248, 2024-03-16T09:18:54.131 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:54.131 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 407, 2024-03-16T09:18:54.131 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T09:18:54.131 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T09:18:54.131 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 407, 2024-03-16T09:18:54.131 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 68236248, 2024-03-16T09:18:54.131 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T09:18:54.131 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.131 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.131 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.131 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:54.131 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.132 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2024-03-16T09:18:54.132 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T09:18:54.132 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-03-16T09:18:54.132 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T09:18:54.132 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T09:18:54.132 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T09:18:54.132 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:54.132 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T09:18:54.132 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 367, 2024-03-16T09:18:54.132 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 67996802, 2024-03-16T09:18:54.132 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 367, 2024-03-16T09:18:54.132 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 67996802, 2024-03-16T09:18:54.132 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T09:18:54.132 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.132 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.133 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.133 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:54.133 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.133 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2024-03-16T09:18:54.133 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T09:18:54.133 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2024-03-16T09:18:54.133 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T09:18:54.133 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T09:18:54.133 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T09:18:54.133 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:54.133 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T09:18:54.133 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 367, 2024-03-16T09:18:54.133 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 367, 2024-03-16T09:18:54.133 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 367, 2024-03-16T09:18:54.133 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 367, 2024-03-16T09:18:54.134 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T09:18:54.134 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.134 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.134 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.134 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:54.134 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.134 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2024-03-16T09:18:54.134 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T09:18:54.134 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-03-16T09:18:54.134 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T09:18:54.134 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T09:18:54.134 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T09:18:54.134 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:54.134 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T09:18:54.135 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T09:18:54.135 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T09:18:54.135 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-03-16T09:18:54.135 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-03-16T09:18:54.135 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T09:18:54.135 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.135 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.135 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.135 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:54.135 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:54.135 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x555f1dc966e0": { 2024-03-16T09:18:54.135 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T09:18:54.135 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-03-16T09:18:54.135 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T09:18:54.135 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T09:18:54.136 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T09:18:54.136 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:54.136 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T09:18:54.136 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T09:18:54.136 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T09:18:54.136 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-03-16T09:18:54.136 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-03-16T09:18:54.136 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T09:18:54.136 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:54.136 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:54.136 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:54.136 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:54.136 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:54.136 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-03-16T09:18:54.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:54 smithi112 ceph-mon[32307]: pgmap v449: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-16T09:18:54.387 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:54 smithi194 ceph-mon[36729]: pgmap v449: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-16T09:18:54.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:54 smithi159 ceph-mon[36739]: pgmap v449: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-16T09:18:54.882 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-16T09:18:54.882 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-16T09:18:54.882 DEBUG:tasks.check_counter:Getting stats from f 2024-03-16T09:18:54.883 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-16T09:18:55.193 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:18:56.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:56 smithi112 ceph-mon[32307]: pgmap v450: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-16T09:18:56.352 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:56 smithi194 ceph-mon[36729]: pgmap v450: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-16T09:18:56.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:56 smithi159 ceph-mon[36739]: pgmap v450: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-16T09:18:56.959 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-03-16T09:18:56.959 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T09:18:56.959 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 741, 2024-03-16T09:18:56.959 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 743, 2024-03-16T09:18:56.959 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 19410114, 2024-03-16T09:18:56.959 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 235645, 2024-03-16T09:18:56.959 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 7, 2024-03-16T09:18:56.959 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 3, 2024-03-16T09:18:56.959 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.055125099, 2024-03-16T09:18:56.959 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.013875404, 2024-03-16T09:18:56.960 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.021425115, 2024-03-16T09:18:56.960 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.015249038, 2024-03-16T09:18:56.960 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:56.960 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 741, 2024-03-16T09:18:56.960 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.018450733, 2024-03-16T09:18:56.960 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000024899 2024-03-16T09:18:56.960 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.960 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:56.960 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.960 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.960 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.960 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.960 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-16T09:18:56.961 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 0 2024-03-16T09:18:56.961 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.961 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T09:18:56.961 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 52, 2024-03-16T09:18:56.961 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 121, 2024-03-16T09:18:56.961 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 8403946, 2024-03-16T09:18:56.961 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 216928, 2024-03-16T09:18:56.961 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 5, 2024-03-16T09:18:56.961 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 4, 2024-03-16T09:18:56.961 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.010861639, 2024-03-16T09:18:56.961 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.003012757, 2024-03-16T09:18:56.961 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.006114736, 2024-03-16T09:18:56.961 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.000960168, 2024-03-16T09:18:56.962 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:56.962 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 120, 2024-03-16T09:18:56.962 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.010304797, 2024-03-16T09:18:56.962 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000085873 2024-03-16T09:18:56.962 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.962 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:56.962 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.962 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.962 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.962 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.962 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-03-16T09:18:56.962 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 201920 2024-03-16T09:18:56.963 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.963 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T09:18:56.963 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 150, 2024-03-16T09:18:56.963 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 136, 2024-03-16T09:18:56.963 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 4972460, 2024-03-16T09:18:56.963 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 69166, 2024-03-16T09:18:56.963 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 4, 2024-03-16T09:18:56.963 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 3, 2024-03-16T09:18:56.963 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.021946614, 2024-03-16T09:18:56.963 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.004779480, 2024-03-16T09:18:56.963 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.019208536, 2024-03-16T09:18:56.963 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.002256711, 2024-03-16T09:18:56.963 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:56.963 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 135, 2024-03-16T09:18:56.964 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.007752276, 2024-03-16T09:18:56.964 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000057424 2024-03-16T09:18:56.964 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.964 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:56.964 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.964 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.964 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.964 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.964 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 769632, 2024-03-16T09:18:56.964 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 58672 2024-03-16T09:18:56.964 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.964 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-03-16T09:18:56.964 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2024-03-16T09:18:56.964 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-03-16T09:18:56.965 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.965 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2024-03-16T09:18:56.965 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-03-16T09:18:56.965 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-03-16T09:18:56.965 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1493, 2024-03-16T09:18:56.965 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.028320394, 2024-03-16T09:18:56.965 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000018968 2024-03-16T09:18:56.965 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:56.965 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.965 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2024-03-16T09:18:56.965 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-03-16T09:18:56.965 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-03-16T09:18:56.965 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.965 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.966 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.966 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:56.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.966 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2024-03-16T09:18:56.966 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 0, 2024-03-16T09:18:56.966 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 0, 2024-03-16T09:18:56.966 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2024-03-16T09:18:56.966 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.966 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.966 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.966 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.966 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2024-03-16T09:18:56.966 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 0, 2024-03-16T09:18:56.967 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 0, 2024-03-16T09:18:56.967 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2024-03-16T09:18:56.967 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 0, 2024-03-16T09:18:56.967 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 0, 2024-03-16T09:18:56.967 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 0, 2024-03-16T09:18:56.967 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 3435, 2024-03-16T09:18:56.967 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 0, 2024-03-16T09:18:56.967 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 3435, 2024-03-16T09:18:56.967 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 0, 2024-03-16T09:18:56.967 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 3435, 2024-03-16T09:18:56.967 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 0, 2024-03-16T09:18:56.967 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 0, 2024-03-16T09:18:56.967 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 0, 2024-03-16T09:18:56.967 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 1, 2024-03-16T09:18:56.968 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 0, 2024-03-16T09:18:56.968 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 0, 2024-03-16T09:18:56.968 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 0, 2024-03-16T09:18:56.968 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 0, 2024-03-16T09:18:56.968 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2024-03-16T09:18:56.968 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2024-03-16T09:18:56.968 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 0, 2024-03-16T09:18:56.968 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 0, 2024-03-16T09:18:56.968 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2024-03-16T09:18:56.968 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 0, 2024-03-16T09:18:56.968 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 0, 2024-03-16T09:18:56.968 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 0, 2024-03-16T09:18:56.968 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 0, 2024-03-16T09:18:56.969 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2024-03-16T09:18:56.969 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 0, 2024-03-16T09:18:56.969 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2024-03-16T09:18:56.969 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 0, 2024-03-16T09:18:56.969 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 0, 2024-03-16T09:18:56.969 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 0, 2024-03-16T09:18:56.969 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 0, 2024-03-16T09:18:56.969 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2024-03-16T09:18:56.969 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T09:18:56.969 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2024-03-16T09:18:56.969 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 0, 2024-03-16T09:18:56.969 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T09:18:56.969 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 0, 2024-03-16T09:18:56.970 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 0, 2024-03-16T09:18:56.970 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 0, 2024-03-16T09:18:56.970 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 0, 2024-03-16T09:18:56.970 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 0, 2024-03-16T09:18:56.970 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T09:18:56.970 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T09:18:56.970 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T09:18:56.970 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 0, 2024-03-16T09:18:56.970 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 0, 2024-03-16T09:18:56.970 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 0, 2024-03-16T09:18:56.970 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 0 2024-03-16T09:18:56.970 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.970 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2024-03-16T09:18:56.971 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 0, 2024-03-16T09:18:56.971 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 0, 2024-03-16T09:18:56.971 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 0, 2024-03-16T09:18:56.971 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 0, 2024-03-16T09:18:56.971 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 0, 2024-03-16T09:18:56.971 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 0, 2024-03-16T09:18:56.971 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2024-03-16T09:18:56.971 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2024-03-16T09:18:56.971 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 0, 2024-03-16T09:18:56.971 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 0, 2024-03-16T09:18:56.971 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2024-03-16T09:18:56.971 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2024-03-16T09:18:56.971 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2024-03-16T09:18:56.971 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2024-03-16T09:18:56.972 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2024-03-16T09:18:56.972 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2024-03-16T09:18:56.972 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2024-03-16T09:18:56.972 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_path": 0, 2024-03-16T09:18:56.972 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_inode": 0, 2024-03-16T09:18:56.972 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T09:18:56.972 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 0, 2024-03-16T09:18:56.972 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2024-03-16T09:18:56.972 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 0, 2024-03-16T09:18:56.972 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2024-03-16T09:18:56.972 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2024-03-16T09:18:56.972 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.972 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2024-03-16T09:18:56.973 INFO:teuthology.orchestra.run.smithi194.stdout: "evlrg": 0, 2024-03-16T09:18:56.973 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 0, 2024-03-16T09:18:56.973 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 0, 2024-03-16T09:18:56.973 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 0, 2024-03-16T09:18:56.973 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 15583, 2024-03-16T09:18:56.973 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2024-03-16T09:18:56.973 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 0, 2024-03-16T09:18:56.973 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 0, 2024-03-16T09:18:56.973 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 0, 2024-03-16T09:18:56.973 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 0, 2024-03-16T09:18:56.973 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 19, 2024-03-16T09:18:56.973 INFO:teuthology.orchestra.run.smithi194.stdout: "segmjr": 3, 2024-03-16T09:18:56.973 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2024-03-16T09:18:56.973 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 0, 2024-03-16T09:18:56.974 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 4194304, 2024-03-16T09:18:56.974 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 35968021, 2024-03-16T09:18:56.974 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 35966883, 2024-03-16T09:18:56.974 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2024-03-16T09:18:56.974 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.974 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.974 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.974 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.974 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 15583 2024-03-16T09:18:56.974 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.974 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2024-03-16T09:18:56.974 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 11, 2024-03-16T09:18:56.974 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 3504, 2024-03-16T09:18:56.974 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 3493, 2024-03-16T09:18:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 16, 2024-03-16T09:18:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 262, 2024-03-16T09:18:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 246, 2024-03-16T09:18:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 3435, 2024-03-16T09:18:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 5381, 2024-03-16T09:18:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 1946, 2024-03-16T09:18:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 0, 2024-03-16T09:18:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 0, 2024-03-16T09:18:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 0, 2024-03-16T09:18:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 60512, 2024-03-16T09:18:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 215420 2024-03-16T09:18:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2024-03-16T09:18:56.975 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 0, 2024-03-16T09:18:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 0, 2024-03-16T09:18:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 0, 2024-03-16T09:18:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 0, 2024-03-16T09:18:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 0, 2024-03-16T09:18:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2024-03-16T09:18:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2024-03-16T09:18:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.976 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2024-03-16T09:18:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2024-03-16T09:18:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2024-03-16T09:18:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.977 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2024-03-16T09:18:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2024-03-16T09:18:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2024-03-16T09:18:56.978 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.979 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.979 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.979 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2024-03-16T09:18:56.979 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.979 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.979 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.979 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2024-03-16T09:18:56.979 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.979 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.979 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.979 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.979 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2024-03-16T09:18:56.980 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.980 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.980 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.980 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.980 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2024-03-16T09:18:56.980 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.980 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.981 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.981 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.981 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2024-03-16T09:18:56.981 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.981 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.981 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.981 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.981 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2024-03-16T09:18:56.981 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.981 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.981 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.981 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.981 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2024-03-16T09:18:56.982 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.982 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.982 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.982 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.982 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2024-03-16T09:18:56.982 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.982 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.982 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.982 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.982 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2024-03-16T09:18:56.982 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.982 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.982 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.983 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.983 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2024-03-16T09:18:56.983 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.983 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.983 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.983 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.983 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2024-03-16T09:18:56.983 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.983 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.983 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.983 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.983 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2024-03-16T09:18:56.983 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.984 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.984 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.984 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2024-03-16T09:18:56.984 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.984 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.984 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.984 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2024-03-16T09:18:56.984 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.984 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.984 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.984 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.984 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2024-03-16T09:18:56.985 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.985 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.985 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.985 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2024-03-16T09:18:56.985 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.985 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.985 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.985 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.985 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2024-03-16T09:18:56.985 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.985 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.985 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.986 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2024-03-16T09:18:56.986 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.986 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.986 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.986 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2024-03-16T09:18:56.986 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.986 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.986 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.986 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.986 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2024-03-16T09:18:56.986 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.986 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.987 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.987 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2024-03-16T09:18:56.987 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.987 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.987 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.987 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.987 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2024-03-16T09:18:56.987 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2024-03-16T09:18:56.987 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2024-03-16T09:18:56.987 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:56.987 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:56.987 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:56.988 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:56.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.988 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2024-03-16T09:18:56.988 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 1, 2024-03-16T09:18:56.988 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 1, 2024-03-16T09:18:56.988 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 0, 2024-03-16T09:18:56.988 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 1, 2024-03-16T09:18:56.988 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2024-03-16T09:18:56.988 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 0, 2024-03-16T09:18:56.988 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 0, 2024-03-16T09:18:56.988 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 1612, 2024-03-16T09:18:56.988 INFO:teuthology.orchestra.run.smithi194.stdout: "mdthresh_evicted": 0 2024-03-16T09:18:56.988 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.988 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-03-16T09:18:56.989 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-03-16T09:18:56.989 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-03-16T09:18:56.989 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T09:18:56.989 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-03-16T09:18:56.989 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T09:18:56.989 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-03-16T09:18:56.989 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T09:18:56.989 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T09:18:56.989 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T09:18:56.989 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T09:18:56.989 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T09:18:56.989 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-03-16T09:18:56.989 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T09:18:56.989 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T09:18:56.989 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-03-16T09:18:56.990 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-03-16T09:18:56.990 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-03-16T09:18:56.990 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-03-16T09:18:56.990 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T09:18:56.990 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T09:18:56.990 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T09:18:56.990 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T09:18:56.990 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T09:18:56.990 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-03-16T09:18:56.990 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-03-16T09:18:56.990 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-03-16T09:18:56.990 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T09:18:56.990 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T09:18:56.991 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-03-16T09:18:56.991 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-03-16T09:18:56.991 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-03-16T09:18:56.991 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-03-16T09:18:56.991 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T09:18:56.991 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-03-16T09:18:56.991 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T09:18:56.991 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-03-16T09:18:56.991 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 205399, 2024-03-16T09:18:56.991 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 56, 2024-03-16T09:18:56.991 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 0, 2024-03-16T09:18:56.991 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 0, 2024-03-16T09:18:56.991 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-03-16T09:18:56.991 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-03-16T09:18:56.992 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-03-16T09:18:56.992 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-03-16T09:18:56.992 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-03-16T09:18:56.992 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-03-16T09:18:56.992 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 7528, 2024-03-16T09:18:56.992 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 158, 2024-03-16T09:18:56.992 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T09:18:56.992 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-03-16T09:18:56.992 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-03-16T09:18:56.992 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-03-16T09:18:56.992 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 2464560, 2024-03-16T09:18:56.992 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 13881, 2024-03-16T09:18:56.992 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-03-16T09:18:56.993 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-03-16T09:18:56.993 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-03-16T09:18:56.993 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-03-16T09:18:56.993 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.993 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2024-03-16T09:18:56.993 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2024-03-16T09:18:56.993 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2024-03-16T09:18:56.993 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 821, 2024-03-16T09:18:56.993 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 21, 2024-03-16T09:18:56.993 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 3, 2024-03-16T09:18:56.993 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 821, 2024-03-16T09:18:56.993 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2024-03-16T09:18:56.993 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 821, 2024-03-16T09:18:56.994 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.880981211, 2024-03-16T09:18:56.994 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.001073058 2024-03-16T09:18:56.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.994 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2024-03-16T09:18:56.994 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2024-03-16T09:18:56.994 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 821, 2024-03-16T09:18:56.994 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 822 2024-03-16T09:18:56.994 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.994 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 821, 2024-03-16T09:18:56.994 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 821, 2024-03-16T09:18:56.994 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 0, 2024-03-16T09:18:56.994 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2024-03-16T09:18:56.994 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2024-03-16T09:18:56.994 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 445, 2024-03-16T09:18:56.995 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 0, 2024-03-16T09:18:56.995 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 374, 2024-03-16T09:18:56.995 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 0, 2024-03-16T09:18:56.995 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 0, 2024-03-16T09:18:56.995 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2024-03-16T09:18:56.995 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2024-03-16T09:18:56.995 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2024-03-16T09:18:56.995 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 0, 2024-03-16T09:18:56.995 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 0, 2024-03-16T09:18:56.995 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2024-03-16T09:18:56.995 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2024-03-16T09:18:56.995 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2024-03-16T09:18:56.995 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 1, 2024-03-16T09:18:56.995 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 0, 2024-03-16T09:18:56.995 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2024-03-16T09:18:56.996 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2024-03-16T09:18:56.996 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2024-03-16T09:18:56.996 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2024-03-16T09:18:56.996 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2024-03-16T09:18:56.996 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2024-03-16T09:18:56.996 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T09:18:56.996 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2024-03-16T09:18:56.996 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2024-03-16T09:18:56.996 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2024-03-16T09:18:56.996 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2024-03-16T09:18:56.996 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2024-03-16T09:18:56.996 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2024-03-16T09:18:56.996 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2024-03-16T09:18:56.996 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2024-03-16T09:18:56.997 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2024-03-16T09:18:56.997 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2024-03-16T09:18:56.997 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2024-03-16T09:18:56.997 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2024-03-16T09:18:56.997 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2024-03-16T09:18:56.997 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2024-03-16T09:18:56.997 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2024-03-16T09:18:56.997 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2024-03-16T09:18:56.997 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2024-03-16T09:18:56.997 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2024-03-16T09:18:56.997 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2024-03-16T09:18:56.997 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 88, 2024-03-16T09:18:56.997 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2024-03-16T09:18:56.997 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 3, 2024-03-16T09:18:56.997 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 7, 2024-03-16T09:18:56.998 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 7, 2024-03-16T09:18:56.998 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 0, 2024-03-16T09:18:56.998 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2024-03-16T09:18:56.998 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 0, 2024-03-16T09:18:56.998 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 2, 2024-03-16T09:18:56.998 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 0 2024-03-16T09:18:56.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.998 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2024-03-16T09:18:56.998 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 0, 2024-03-16T09:18:56.998 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2024-03-16T09:18:56.998 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2024-03-16T09:18:56.998 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2024-03-16T09:18:56.998 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.998 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2024-03-16T09:18:56.999 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 0 2024-03-16T09:18:56.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.999 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2024-03-16T09:18:56.999 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2024-03-16T09:18:56.999 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 0, 2024-03-16T09:18:56.999 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2024-03-16T09:18:56.999 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 0, 2024-03-16T09:18:56.999 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed_ops": 0, 2024-03-16T09:18:56.999 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 0, 2024-03-16T09:18:56.999 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2024-03-16T09:18:56.999 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:56.999 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T09:18:56.999 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T09:18:56.999 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-03-16T09:18:56.999 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T09:18:57.000 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 939, 2024-03-16T09:18:57.000 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 32693060, 2024-03-16T09:18:57.000 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:57.000 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 939, 2024-03-16T09:18:57.000 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T09:18:57.000 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T09:18:57.000 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 939, 2024-03-16T09:18:57.000 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 32693060, 2024-03-16T09:18:57.000 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T09:18:57.000 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:57.000 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:57.000 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:57.000 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:57.000 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:57.001 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2024-03-16T09:18:57.001 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T09:18:57.001 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-03-16T09:18:57.001 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T09:18:57.001 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T09:18:57.001 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T09:18:57.001 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:57.001 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T09:18:57.001 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 821, 2024-03-16T09:18:57.001 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 31776748, 2024-03-16T09:18:57.001 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 821, 2024-03-16T09:18:57.001 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 31776748, 2024-03-16T09:18:57.001 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T09:18:57.001 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:57.001 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:57.002 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:57.002 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:57.002 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:57.002 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2024-03-16T09:18:57.002 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T09:18:57.002 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2024-03-16T09:18:57.002 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T09:18:57.002 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T09:18:57.002 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T09:18:57.002 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:57.002 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T09:18:57.002 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 821, 2024-03-16T09:18:57.002 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 821, 2024-03-16T09:18:57.002 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 821, 2024-03-16T09:18:57.003 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 821, 2024-03-16T09:18:57.003 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T09:18:57.003 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:57.003 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:57.003 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:57.003 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:57.003 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:57.003 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2024-03-16T09:18:57.003 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T09:18:57.003 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-03-16T09:18:57.003 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T09:18:57.003 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T09:18:57.003 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T09:18:57.004 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:57.004 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T09:18:57.004 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T09:18:57.004 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T09:18:57.004 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-03-16T09:18:57.004 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-03-16T09:18:57.004 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T09:18:57.004 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:57.004 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:57.004 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:57.004 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:57.004 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:57.004 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x55ff1931a6e0": { 2024-03-16T09:18:57.004 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T09:18:57.004 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-03-16T09:18:57.005 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T09:18:57.005 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T09:18:57.005 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T09:18:57.005 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:57.005 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T09:18:57.005 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T09:18:57.005 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T09:18:57.005 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-03-16T09:18:57.005 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-03-16T09:18:57.005 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T09:18:57.005 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:57.005 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:57.005 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:57.005 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:57.006 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:57.006 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-03-16T09:18:57.797 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-16T09:18:57.797 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-16T09:18:57.797 DEBUG:tasks.check_counter:Getting stats from i 2024-03-16T09:18:57.797 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-16T09:18:58.053 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:18:58.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:18:58 smithi194 ceph-mon[36729]: pgmap v451: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-03-16T09:18:58.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:18:58 smithi112 ceph-mon[32307]: pgmap v451: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-03-16T09:18:58.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:18:58 smithi159 ceph-mon[36739]: pgmap v451: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-03-16T09:18:59.793 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-03-16T09:18:59.793 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T09:18:59.793 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 13206, 2024-03-16T09:18:59.793 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 24205, 2024-03-16T09:18:59.793 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 4768225, 2024-03-16T09:18:59.793 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 27664378, 2024-03-16T09:18:59.793 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 21, 2024-03-16T09:18:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 3, 2024-03-16T09:18:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.752450373, 2024-03-16T09:18:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.379673964, 2024-03-16T09:18:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.638164435, 2024-03-16T09:18:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.058071714, 2024-03-16T09:18:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 24205, 2024-03-16T09:18:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.586893494, 2024-03-16T09:18:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000024246 2024-03-16T09:18:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 18640, 2024-03-16T09:18:59.794 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.003162553, 2024-03-16T09:18:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000169 2024-03-16T09:18:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 784496, 2024-03-16T09:18:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 65760 2024-03-16T09:18:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T09:18:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 6487, 2024-03-16T09:18:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 7038, 2024-03-16T09:18:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 3823401, 2024-03-16T09:18:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 6263331, 2024-03-16T09:18:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 8, 2024-03-16T09:18:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 6, 2024-03-16T09:18:59.795 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.383456604, 2024-03-16T09:18:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.151805149, 2024-03-16T09:18:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.192949436, 2024-03-16T09:18:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.054274277, 2024-03-16T09:18:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 7036, 2024-03-16T09:18:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.240947727, 2024-03-16T09:18:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000034244 2024-03-16T09:18:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 8850, 2024-03-16T09:18:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.001273008, 2024-03-16T09:18:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000143 2024-03-16T09:18:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.796 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-16T09:18:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 203008 2024-03-16T09:18:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T09:18:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 2239, 2024-03-16T09:18:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 2287, 2024-03-16T09:18:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 928633, 2024-03-16T09:18:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 12373013, 2024-03-16T09:18:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 8, 2024-03-16T09:18:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 6, 2024-03-16T09:18:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.196849046, 2024-03-16T09:18:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.081634311, 2024-03-16T09:18:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.049805729, 2024-03-16T09:18:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.052047750, 2024-03-16T09:18:59.797 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:18:59.798 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2285, 2024-03-16T09:18:59.798 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.138200876, 2024-03-16T09:18:59.798 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000060481 2024-03-16T09:18:59.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.798 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:18:59.798 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 604, 2024-03-16T09:18:59.798 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000078944, 2024-03-16T09:18:59.798 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000130 2024-03-16T09:18:59.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.798 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-16T09:18:59.798 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 0 2024-03-16T09:18:59.798 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-03-16T09:18:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2024-03-16T09:18:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-03-16T09:18:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2024-03-16T09:18:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-03-16T09:18:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-03-16T09:18:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1688, 2024-03-16T09:18:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 8.392180140, 2024-03-16T09:18:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.004971670 2024-03-16T09:18:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2024-03-16T09:18:59.799 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-03-16T09:18:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-03-16T09:18:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 5065, 2024-03-16T09:18:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 1.282999759, 2024-03-16T09:18:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000253306 2024-03-16T09:18:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2024-03-16T09:18:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 5542, 2024-03-16T09:18:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 5533, 2024-03-16T09:18:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2024-03-16T09:18:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 5533, 2024-03-16T09:18:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 12.650193178, 2024-03-16T09:18:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.002286317 2024-03-16T09:18:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.800 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2024-03-16T09:18:59.801 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 9, 2024-03-16T09:18:59.801 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 0, 2024-03-16T09:18:59.801 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2024-03-16T09:18:59.801 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 37, 2024-03-16T09:18:59.801 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 3, 2024-03-16T09:18:59.801 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 1, 2024-03-16T09:18:59.801 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 286, 2024-03-16T09:18:59.801 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 177, 2024-03-16T09:18:59.801 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 82, 2024-03-16T09:18:59.801 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 27, 2024-03-16T09:18:59.801 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 30, 2024-03-16T09:18:59.801 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 67, 2024-03-16T09:18:59.801 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 0, 2024-03-16T09:18:59.802 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 0, 2024-03-16T09:18:59.802 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 3, 2024-03-16T09:18:59.802 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 6386, 2024-03-16T09:18:59.802 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 5812, 2024-03-16T09:18:59.802 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 0, 2024-03-16T09:18:59.802 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 103, 2024-03-16T09:18:59.802 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2024-03-16T09:18:59.802 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2024-03-16T09:18:59.802 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 0, 2024-03-16T09:18:59.802 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 0, 2024-03-16T09:18:59.802 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2024-03-16T09:18:59.802 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 13, 2024-03-16T09:18:59.802 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 3, 2024-03-16T09:18:59.803 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 108, 2024-03-16T09:18:59.803 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 1879, 2024-03-16T09:18:59.803 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2024-03-16T09:18:59.803 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 1688, 2024-03-16T09:18:59.803 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2024-03-16T09:18:59.803 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 620, 2024-03-16T09:18:59.803 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 3951007, 2024-03-16T09:18:59.803 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 0, 2024-03-16T09:18:59.803 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 1688, 2024-03-16T09:18:59.803 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2024-03-16T09:18:59.803 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2024-03-16T09:18:59.803 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2024-03-16T09:18:59.803 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 140, 2024-03-16T09:18:59.803 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2024-03-16T09:18:59.804 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 140, 2024-03-16T09:18:59.804 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 1548, 2024-03-16T09:18:59.804 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 18, 2024-03-16T09:18:59.804 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 706, 2024-03-16T09:18:59.804 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 5576, 2024-03-16T09:18:59.804 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2024-03-16T09:18:59.804 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-16T09:18:59.804 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-16T09:18:59.804 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 5576, 2024-03-16T09:18:59.804 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 3466, 2024-03-16T09:18:59.804 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 104, 2024-03-16T09:18:59.804 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 4032 2024-03-16T09:18:59.804 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.805 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2024-03-16T09:18:59.805 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 0, 2024-03-16T09:18:59.805 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 0, 2024-03-16T09:18:59.805 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 0, 2024-03-16T09:18:59.805 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 111, 2024-03-16T09:18:59.805 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 0, 2024-03-16T09:18:59.805 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 0, 2024-03-16T09:18:59.805 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2024-03-16T09:18:59.805 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2024-03-16T09:18:59.805 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 3427, 2024-03-16T09:18:59.805 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 3427, 2024-03-16T09:18:59.805 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2024-03-16T09:18:59.805 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2024-03-16T09:18:59.805 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2024-03-16T09:18:59.806 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2024-03-16T09:18:59.806 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2024-03-16T09:18:59.806 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2024-03-16T09:18:59.806 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2024-03-16T09:18:59.806 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_path": 0, 2024-03-16T09:18:59.806 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_inode": 0, 2024-03-16T09:18:59.806 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2024-03-16T09:18:59.806 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 13, 2024-03-16T09:18:59.806 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2024-03-16T09:18:59.806 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 4, 2024-03-16T09:18:59.806 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2024-03-16T09:18:59.806 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2024-03-16T09:18:59.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.806 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2024-03-16T09:18:59.807 INFO:teuthology.orchestra.run.smithi194.stdout: "evlrg": 0, 2024-03-16T09:18:59.807 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 15583, 2024-03-16T09:18:59.807 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 0, 2024-03-16T09:18:59.807 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 0, 2024-03-16T09:18:59.807 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 15583, 2024-03-16T09:18:59.807 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2024-03-16T09:18:59.807 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 0, 2024-03-16T09:18:59.807 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 19, 2024-03-16T09:18:59.807 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 0, 2024-03-16T09:18:59.807 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 0, 2024-03-16T09:18:59.807 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 19, 2024-03-16T09:18:59.807 INFO:teuthology.orchestra.run.smithi194.stdout: "segmjr": 3, 2024-03-16T09:18:59.807 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2024-03-16T09:18:59.808 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 0, 2024-03-16T09:18:59.808 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 4194304, 2024-03-16T09:18:59.808 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 35968021, 2024-03-16T09:18:59.808 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 0, 2024-03-16T09:18:59.808 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2024-03-16T09:18:59.808 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 509, 2024-03-16T09:18:59.808 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 6.278354308, 2024-03-16T09:18:59.808 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.012334684 2024-03-16T09:18:59.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.808 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 0 2024-03-16T09:18:59.808 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.808 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2024-03-16T09:18:59.808 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 25, 2024-03-16T09:18:59.808 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 3483, 2024-03-16T09:18:59.808 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 3458, 2024-03-16T09:18:59.809 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 31, 2024-03-16T09:18:59.809 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 241, 2024-03-16T09:18:59.809 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 210, 2024-03-16T09:18:59.809 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 286, 2024-03-16T09:18:59.809 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 5388, 2024-03-16T09:18:59.809 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 5102, 2024-03-16T09:18:59.809 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 0, 2024-03-16T09:18:59.809 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 3440, 2024-03-16T09:18:59.809 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 3440, 2024-03-16T09:18:59.809 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 79132, 2024-03-16T09:18:59.809 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 223612 2024-03-16T09:18:59.809 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.809 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2024-03-16T09:18:59.809 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 6971, 2024-03-16T09:18:59.810 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 202, 2024-03-16T09:18:59.810 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 5542, 2024-03-16T09:18:59.810 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 2, 2024-03-16T09:18:59.810 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 771, 2024-03-16T09:18:59.810 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2024-03-16T09:18:59.810 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1548, 2024-03-16T09:18:59.810 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 2.047328252, 2024-03-16T09:18:59.810 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.001322563 2024-03-16T09:18:59.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.810 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2024-03-16T09:18:59.810 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 7, 2024-03-16T09:18:59.810 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.004601838, 2024-03-16T09:18:59.810 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000657405 2024-03-16T09:18:59.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.811 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2024-03-16T09:18:59.811 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.811 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.811 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.811 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2024-03-16T09:18:59.811 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.811 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.811 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.811 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2024-03-16T09:18:59.811 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 250, 2024-03-16T09:18:59.811 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.096817880, 2024-03-16T09:18:59.811 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000387271 2024-03-16T09:18:59.811 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.812 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2024-03-16T09:18:59.812 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.812 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.812 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.812 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2024-03-16T09:18:59.812 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.812 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.812 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.812 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.812 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2024-03-16T09:18:59.812 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.812 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.812 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.813 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2024-03-16T09:18:59.813 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.813 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.813 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.813 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2024-03-16T09:18:59.813 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.813 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.813 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.813 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.813 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2024-03-16T09:18:59.813 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.814 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.814 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.814 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.814 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2024-03-16T09:18:59.814 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.815 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.815 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.815 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2024-03-16T09:18:59.815 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.815 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.815 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.815 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2024-03-16T09:18:59.815 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.815 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.815 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.815 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.816 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2024-03-16T09:18:59.816 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.816 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.816 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.816 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2024-03-16T09:18:59.816 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 289, 2024-03-16T09:18:59.816 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.420896782, 2024-03-16T09:18:59.816 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.001456390 2024-03-16T09:18:59.816 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.816 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2024-03-16T09:18:59.816 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.816 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.817 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.817 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2024-03-16T09:18:59.817 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.817 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.817 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.817 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2024-03-16T09:18:59.817 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.817 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.817 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.817 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.817 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2024-03-16T09:18:59.818 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 147, 2024-03-16T09:18:59.818 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 2.757668090, 2024-03-16T09:18:59.818 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.018759646 2024-03-16T09:18:59.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.818 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2024-03-16T09:18:59.818 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.818 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.818 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.818 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.818 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2024-03-16T09:18:59.818 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.818 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.818 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.819 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2024-03-16T09:18:59.819 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 12, 2024-03-16T09:18:59.819 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 1.565628454, 2024-03-16T09:18:59.819 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.130469037 2024-03-16T09:18:59.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.819 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2024-03-16T09:18:59.819 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.819 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.819 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.819 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.819 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2024-03-16T09:18:59.819 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.820 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.820 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.820 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2024-03-16T09:18:59.820 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.820 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.820 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.820 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2024-03-16T09:18:59.820 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.820 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.820 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.820 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.820 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2024-03-16T09:18:59.821 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.821 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.821 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.821 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2024-03-16T09:18:59.821 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3280, 2024-03-16T09:18:59.821 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 5.757251882, 2024-03-16T09:18:59.821 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.001755259 2024-03-16T09:18:59.821 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.821 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2024-03-16T09:18:59.821 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2024-03-16T09:18:59.821 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2024-03-16T09:18:59.821 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.821 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.822 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.822 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:59.822 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.822 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2024-03-16T09:18:59.822 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 1, 2024-03-16T09:18:59.822 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 1, 2024-03-16T09:18:59.822 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 0, 2024-03-16T09:18:59.822 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 1, 2024-03-16T09:18:59.822 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2024-03-16T09:18:59.822 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 3275, 2024-03-16T09:18:59.822 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 3275, 2024-03-16T09:18:59.822 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 1615, 2024-03-16T09:18:59.822 INFO:teuthology.orchestra.run.smithi194.stdout: "mdthresh_evicted": 0 2024-03-16T09:18:59.823 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.823 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-03-16T09:18:59.823 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-03-16T09:18:59.823 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-03-16T09:18:59.823 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T09:18:59.823 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-03-16T09:18:59.823 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T09:18:59.823 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-03-16T09:18:59.823 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T09:18:59.823 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T09:18:59.823 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T09:18:59.823 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T09:18:59.823 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T09:18:59.823 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-03-16T09:18:59.824 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T09:18:59.824 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T09:18:59.824 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-03-16T09:18:59.824 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-03-16T09:18:59.824 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-03-16T09:18:59.824 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-03-16T09:18:59.824 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T09:18:59.824 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T09:18:59.824 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T09:18:59.824 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T09:18:59.824 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T09:18:59.824 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-03-16T09:18:59.824 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-03-16T09:18:59.825 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-03-16T09:18:59.825 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T09:18:59.825 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T09:18:59.825 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-03-16T09:18:59.825 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-03-16T09:18:59.825 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-03-16T09:18:59.825 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-03-16T09:18:59.825 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T09:18:59.825 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-03-16T09:18:59.825 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T09:18:59.825 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-03-16T09:18:59.825 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 297811, 2024-03-16T09:18:59.825 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 116, 2024-03-16T09:18:59.825 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 616, 2024-03-16T09:18:59.826 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 7, 2024-03-16T09:18:59.826 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-03-16T09:18:59.826 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-03-16T09:18:59.826 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-03-16T09:18:59.826 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-03-16T09:18:59.826 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-03-16T09:18:59.826 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-03-16T09:18:59.826 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 7528, 2024-03-16T09:18:59.826 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 158, 2024-03-16T09:18:59.826 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T09:18:59.826 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-03-16T09:18:59.826 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-03-16T09:18:59.826 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-03-16T09:18:59.827 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 314813, 2024-03-16T09:18:59.827 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 3477, 2024-03-16T09:18:59.827 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-03-16T09:18:59.827 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-03-16T09:18:59.827 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-03-16T09:18:59.827 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-03-16T09:18:59.827 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.827 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2024-03-16T09:18:59.827 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2024-03-16T09:18:59.827 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2024-03-16T09:18:59.827 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 5950, 2024-03-16T09:18:59.827 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 32934202, 2024-03-16T09:18:59.827 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 33, 2024-03-16T09:18:59.827 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 5950, 2024-03-16T09:18:59.828 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2024-03-16T09:18:59.828 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 5950, 2024-03-16T09:18:59.828 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 13.130720413, 2024-03-16T09:18:59.828 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.002206843 2024-03-16T09:18:59.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.828 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2024-03-16T09:18:59.828 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2024-03-16T09:18:59.828 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 5950, 2024-03-16T09:18:59.828 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 6057 2024-03-16T09:18:59.828 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.828 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 5950, 2024-03-16T09:18:59.828 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 1764, 2024-03-16T09:18:59.828 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 4186, 2024-03-16T09:18:59.828 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2024-03-16T09:18:59.829 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2024-03-16T09:18:59.829 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 0, 2024-03-16T09:18:59.829 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 10, 2024-03-16T09:18:59.829 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 76, 2024-03-16T09:18:59.829 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 585, 2024-03-16T09:18:59.829 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 44, 2024-03-16T09:18:59.829 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2024-03-16T09:18:59.829 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2024-03-16T09:18:59.829 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2024-03-16T09:18:59.829 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 3, 2024-03-16T09:18:59.829 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 3462, 2024-03-16T09:18:59.829 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2024-03-16T09:18:59.829 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2024-03-16T09:18:59.829 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2024-03-16T09:18:59.829 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 1688, 2024-03-16T09:18:59.830 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 20, 2024-03-16T09:18:59.830 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2024-03-16T09:18:59.830 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2024-03-16T09:18:59.830 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2024-03-16T09:18:59.830 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2024-03-16T09:18:59.830 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2024-03-16T09:18:59.830 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2024-03-16T09:18:59.830 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2024-03-16T09:18:59.830 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2024-03-16T09:18:59.830 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2024-03-16T09:18:59.830 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2024-03-16T09:18:59.830 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2024-03-16T09:18:59.830 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2024-03-16T09:18:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2024-03-16T09:18:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2024-03-16T09:18:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2024-03-16T09:18:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2024-03-16T09:18:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2024-03-16T09:18:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2024-03-16T09:18:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2024-03-16T09:18:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2024-03-16T09:18:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2024-03-16T09:18:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2024-03-16T09:18:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2024-03-16T09:18:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2024-03-16T09:18:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2024-03-16T09:18:59.831 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2024-03-16T09:18:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 88, 2024-03-16T09:18:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2024-03-16T09:18:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 3, 2024-03-16T09:18:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 12, 2024-03-16T09:18:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 12, 2024-03-16T09:18:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 0, 2024-03-16T09:18:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2024-03-16T09:18:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 138, 2024-03-16T09:18:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 0, 2024-03-16T09:18:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 31 2024-03-16T09:18:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2024-03-16T09:18:59.832 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 1, 2024-03-16T09:18:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2024-03-16T09:18:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2024-03-16T09:18:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2024-03-16T09:18:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2024-03-16T09:18:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 0 2024-03-16T09:18:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2024-03-16T09:18:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2024-03-16T09:18:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 18, 2024-03-16T09:18:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2024-03-16T09:18:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 10, 2024-03-16T09:18:59.833 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed_ops": 3435, 2024-03-16T09:18:59.834 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 3427, 2024-03-16T09:18:59.834 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2024-03-16T09:18:59.834 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.834 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T09:18:59.834 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T09:18:59.834 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-03-16T09:18:59.834 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T09:18:59.834 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 21928, 2024-03-16T09:18:59.834 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 7715989, 2024-03-16T09:18:59.834 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:59.834 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 21928, 2024-03-16T09:18:59.834 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T09:18:59.835 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T09:18:59.835 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 21928, 2024-03-16T09:18:59.835 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 7715989, 2024-03-16T09:18:59.835 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T09:18:59.835 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.835 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.835 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.835 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:59.835 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.835 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2024-03-16T09:18:59.835 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T09:18:59.835 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-03-16T09:18:59.835 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T09:18:59.836 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T09:18:59.836 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T09:18:59.836 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:59.836 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T09:18:59.836 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 5950, 2024-03-16T09:18:59.836 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 33280337, 2024-03-16T09:18:59.836 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 5950, 2024-03-16T09:18:59.836 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 33280337, 2024-03-16T09:18:59.836 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T09:18:59.836 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.836 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.836 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.836 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:59.836 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.837 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2024-03-16T09:18:59.837 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T09:18:59.837 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2024-03-16T09:18:59.837 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T09:18:59.837 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-03-16T09:18:59.837 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-03-16T09:18:59.837 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:59.837 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-03-16T09:18:59.837 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 5950, 2024-03-16T09:18:59.837 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 5950, 2024-03-16T09:18:59.837 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 5950, 2024-03-16T09:18:59.837 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 5950, 2024-03-16T09:18:59.837 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T09:18:59.838 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.838 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.838 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.838 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:59.838 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.838 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2024-03-16T09:18:59.838 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T09:18:59.838 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-03-16T09:18:59.838 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T09:18:59.838 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 3427, 2024-03-16T09:18:59.838 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 346135, 2024-03-16T09:18:59.838 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:59.838 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 3427, 2024-03-16T09:18:59.839 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T09:18:59.839 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T09:18:59.839 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 76, 2024-03-16T09:18:59.839 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 346135, 2024-03-16T09:18:59.839 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T09:18:59.839 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.839 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.839 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.839 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:59.839 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:18:59.839 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x555d38e40500": { 2024-03-16T09:18:59.839 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T09:18:59.840 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-03-16T09:18:59.840 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T09:18:59.840 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 15583, 2024-03-16T09:18:59.840 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 31773717, 2024-03-16T09:18:59.840 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T09:18:59.840 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 15583, 2024-03-16T09:18:59.840 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T09:18:59.840 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T09:18:59.840 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 509, 2024-03-16T09:18:59.840 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 31773717, 2024-03-16T09:18:59.840 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T09:18:59.840 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:18:59.840 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:18:59.840 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:18:59.841 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:59.841 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:18:59.841 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-03-16T09:19:00.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:00 smithi194 ceph-mon[36729]: pgmap v452: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T09:19:00.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:00 smithi112 ceph-mon[32307]: pgmap v452: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T09:19:00.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:00 smithi159 ceph-mon[36739]: pgmap v452: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T09:19:00.625 INFO:tasks.check_counter:Daemon mds.i mds.exported=13 2024-03-16T09:19:00.625 INFO:tasks.check_counter:Daemon mds.i mds.imported=108 2024-03-16T09:19:00.625 DEBUG:tasks.check_counter:Getting stats from l 2024-03-16T09:19:00.625 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 273be2a4-e374-11ee-95c9-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-16T09:19:00.888 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/mon.c/config 2024-03-16T09:19:01.284 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:19:01.284 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:19:01.284 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:19:01.284 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:01 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:19:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:19:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:19:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:19:01.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:01 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:19:01.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-16T09:19:01.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-16T09:19:01.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-16T09:19:01.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:01 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' 2024-03-16T09:19:02.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:02 smithi112 ceph-mon[32307]: pgmap v453: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-16T09:19:02.388 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:02 smithi194 ceph-mon[36729]: pgmap v453: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-16T09:19:02.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:02 smithi159 ceph-mon[36739]: pgmap v453: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-16T09:19:02.680 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-03-16T09:19:02.680 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-03-16T09:19:02.680 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 0, 2024-03-16T09:19:02.680 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 2, 2024-03-16T09:19:02.681 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 0, 2024-03-16T09:19:02.681 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 0, 2024-03-16T09:19:02.681 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 4, 2024-03-16T09:19:02.681 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 0, 2024-03-16T09:19:02.681 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.001106880, 2024-03-16T09:19:02.681 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.000000000, 2024-03-16T09:19:02.681 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.000315983, 2024-03-16T09:19:02.681 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T09:19:02.681 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:19:02.681 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:19:02.681 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:19:02.681 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:19:02.681 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:19:02.681 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:19:02.682 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:19:02.682 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:19:02.682 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:19:02.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:19:02.682 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-16T09:19:02.682 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 0 2024-03-16T09:19:02.682 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:19:02.682 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-03-16T09:19:02.682 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 110, 2024-03-16T09:19:02.682 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 103, 2024-03-16T09:19:02.682 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 721808, 2024-03-16T09:19:02.682 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 61296, 2024-03-16T09:19:02.682 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 2, 2024-03-16T09:19:02.683 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 1, 2024-03-16T09:19:02.683 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.016774471, 2024-03-16T09:19:02.683 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.004061395, 2024-03-16T09:19:02.683 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.016420717, 2024-03-16T09:19:02.683 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.001675906, 2024-03-16T09:19:02.683 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:19:02.683 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 103, 2024-03-16T09:19:02.683 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.004554625, 2024-03-16T09:19:02.683 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000044219 2024-03-16T09:19:02.683 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:19:02.683 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:19:02.683 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:19:02.683 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:19:02.683 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:19:02.684 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:19:02.684 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 721808, 2024-03-16T09:19:02.684 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 61296 2024-03-16T09:19:02.684 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:19:02.684 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-03-16T09:19:02.684 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 1, 2024-03-16T09:19:02.684 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 76, 2024-03-16T09:19:02.684 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 160, 2024-03-16T09:19:02.684 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 91088, 2024-03-16T09:19:02.684 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 3, 2024-03-16T09:19:02.684 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 2, 2024-03-16T09:19:02.684 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.003543063, 2024-03-16T09:19:02.684 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.002198029, 2024-03-16T09:19:02.684 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.000781322, 2024-03-16T09:19:02.684 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-16T09:19:02.685 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-03-16T09:19:02.685 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 74, 2024-03-16T09:19:02.685 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.003116267, 2024-03-16T09:19:02.685 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000042111 2024-03-16T09:19:02.685 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:19:02.685 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-03-16T09:19:02.685 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:19:02.685 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:19:02.685 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:19:02.685 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:19:02.685 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-16T09:19:02.685 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 91088 2024-03-16T09:19:02.685 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:19:02.685 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-03-16T09:19:02.685 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 0, 2024-03-16T09:19:02.686 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-03-16T09:19:02.686 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:19:02.686 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-03-16T09:19:02.686 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-03-16T09:19:02.686 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-03-16T09:19:02.686 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-03-16T09:19:02.686 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-03-16T09:19:02.686 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-03-16T09:19:02.686 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-03-16T09:19:02.686 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-16T09:19:02.686 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-03-16T09:19:02.686 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-16T09:19:02.686 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-03-16T09:19:02.686 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-03-16T09:19:02.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-03-16T09:19:02.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-16T09:19:02.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-03-16T09:19:02.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-03-16T09:19:02.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-03-16T09:19:02.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-03-16T09:19:02.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-03-16T09:19:02.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-16T09:19:02.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-03-16T09:19:02.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-16T09:19:02.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-03-16T09:19:02.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-03-16T09:19:02.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-03-16T09:19:02.687 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-03-16T09:19:02.688 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-03-16T09:19:02.688 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-16T09:19:02.688 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-03-16T09:19:02.688 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-03-16T09:19:02.688 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-03-16T09:19:02.688 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-03-16T09:19:02.688 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-03-16T09:19:02.688 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-03-16T09:19:02.688 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-03-16T09:19:02.688 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-03-16T09:19:02.688 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-03-16T09:19:02.688 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 148903, 2024-03-16T09:19:02.688 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 29, 2024-03-16T09:19:02.688 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 0, 2024-03-16T09:19:02.688 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 0, 2024-03-16T09:19:02.689 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-03-16T09:19:02.689 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-03-16T09:19:02.689 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-03-16T09:19:02.689 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-03-16T09:19:02.689 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-03-16T09:19:02.689 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-03-16T09:19:02.689 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 0, 2024-03-16T09:19:02.689 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 0, 2024-03-16T09:19:02.689 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-03-16T09:19:02.689 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-03-16T09:19:02.689 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-03-16T09:19:02.689 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-03-16T09:19:02.689 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 928, 2024-03-16T09:19:02.689 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 2, 2024-03-16T09:19:02.689 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-03-16T09:19:02.690 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-03-16T09:19:02.690 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-03-16T09:19:02.690 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-03-16T09:19:02.690 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-03-16T09:19:02.690 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-16T09:19:02.690 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-03-16T09:19:02.690 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-03-16T09:19:02.690 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-03-16T09:19:02.690 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 107, 2024-03-16T09:19:02.690 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 703887, 2024-03-16T09:19:02.690 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-03-16T09:19:02.690 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 107, 2024-03-16T09:19:02.690 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-03-16T09:19:02.690 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-03-16T09:19:02.691 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 107, 2024-03-16T09:19:02.691 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 703887, 2024-03-16T09:19:02.691 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-03-16T09:19:02.691 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-03-16T09:19:02.691 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-03-16T09:19:02.691 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-03-16T09:19:02.691 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:19:02.691 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-03-16T09:19:02.691 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-03-16T09:19:03.416 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-03-16T09:19:03.416 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-03-16T09:19:03.416 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-16T09:19:03.427 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-16T09:19:03.427 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T09:19:03.427 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T09:19:03.448 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T09:19:03.448 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T09:19:03.508 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:19:03.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.659+0000 7f79c86b2640 1 -- 172.21.15.112:0/1685651464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79c01022b0 msgr2=0x7f79c0108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:03.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.659+0000 7f79c86b2640 1 --2- 172.21.15.112:0/1685651464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79c01022b0 0x7f79c0108b40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009900 tx=0x7f79b802bc30 comp rx=0 tx=0).stop 2024-03-16T09:19:03.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.659+0000 7f79c86b2640 1 -- 172.21.15.112:0/1685651464 shutdown_connections 2024-03-16T09:19:03.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.659+0000 7f79c86b2640 1 --2- 172.21.15.112:0/1685651464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79c01022b0 0x7f79c0108b40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:03.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.659+0000 7f79c86b2640 1 --2- 172.21.15.112:0/1685651464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79c01018f0 0x7f79c0101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:03.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.659+0000 7f79c86b2640 1 --2- 172.21.15.112:0/1685651464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79c0100700 0x7f79c0100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:03.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.659+0000 7f79c86b2640 1 -- 172.21.15.112:0/1685651464 >> 172.21.15.112:0/1685651464 conn(0x7f79c00fbc70 msgr2=0x7f79c00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:03.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.659+0000 7f79c86b2640 1 -- 172.21.15.112:0/1685651464 shutdown_connections 2024-03-16T09:19:03.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.659+0000 7f79c86b2640 1 -- 172.21.15.112:0/1685651464 wait complete. 2024-03-16T09:19:03.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.660+0000 7f79c86b2640 1 Processor -- start 2024-03-16T09:19:03.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.660+0000 7f79c86b2640 1 -- start start 2024-03-16T09:19:03.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.660+0000 7f79c86b2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79c0100700 0x7f79c0199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:03.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.660+0000 7f79c86b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79c01018f0 0x7f79c019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:03.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.660+0000 7f79c86b2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79c01022b0 0x7f79c01a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:03.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.660+0000 7f79c86b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f79c010c460 con 0x7f79c01018f0 2024-03-16T09:19:03.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.660+0000 7f79c86b2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f79c010c2e0 con 0x7f79c0100700 2024-03-16T09:19:03.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.660+0000 7f79c86b2640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f79c010c5e0 con 0x7f79c01022b0 2024-03-16T09:19:03.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.660+0000 7f79c5c4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79c01018f0 0x7f79c019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:03.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.660+0000 7f79c644e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79c0100700 0x7f79c0199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:03.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.660+0000 7f79c5c4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79c01018f0 0x7f79c019a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32882/0 (socket says 172.21.15.112:32882) 2024-03-16T09:19:03.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.660+0000 7f79c5c4d640 1 -- 172.21.15.112:0/590584211 learned_addr learned my addr 172.21.15.112:0/590584211 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:19:03.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.660+0000 7f79c6c4f640 1 --2- 172.21.15.112:0/590584211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79c01022b0 0x7f79c01a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:03.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.660+0000 7f79c644e640 1 -- 172.21.15.112:0/590584211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79c01022b0 msgr2=0x7f79c01a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:03.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.660+0000 7f79c644e640 1 --2- 172.21.15.112:0/590584211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79c01022b0 0x7f79c01a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:03.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.661+0000 7f79c644e640 1 -- 172.21.15.112:0/590584211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79c01018f0 msgr2=0x7f79c019a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:03.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.661+0000 7f79c644e640 1 --2- 172.21.15.112:0/590584211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79c01018f0 0x7f79c019a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:03.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.661+0000 7f79c644e640 1 -- 172.21.15.112:0/590584211 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c01a19d0 con 0x7f79c0100700 2024-03-16T09:19:03.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.661+0000 7f79c5c4d640 1 --2- 172.21.15.112:0/590584211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79c01018f0 0x7f79c019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:19:03.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.661+0000 7f79c644e640 1 --2- 172.21.15.112:0/590584211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79c0100700 0x7f79c0199dc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f79bc005fd0 tx=0x7f79bc00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:03.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.661+0000 7f79b77fe640 1 -- 172.21.15.112:0/590584211 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc003970 con 0x7f79c0100700 2024-03-16T09:19:03.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.661+0000 7f79c86b2640 1 -- 172.21.15.112:0/590584211 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c01a1cc0 con 0x7f79c0100700 2024-03-16T09:19:03.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.661+0000 7f79b77fe640 1 -- 172.21.15.112:0/590584211 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f79bc004520 con 0x7f79c0100700 2024-03-16T09:19:03.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.661+0000 7f79b77fe640 1 -- 172.21.15.112:0/590584211 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc0107c0 con 0x7f79c0100700 2024-03-16T09:19:03.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.661+0000 7f79c86b2640 1 -- 172.21.15.112:0/590584211 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c01a21c0 con 0x7f79c0100700 2024-03-16T09:19:03.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.662+0000 7f79c86b2640 1 -- 172.21.15.112:0/590584211 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79c01a23f0 con 0x7f79c0100700 2024-03-16T09:19:03.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.663+0000 7f79b77fe640 1 -- 172.21.15.112:0/590584211 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7f79bc004000 con 0x7f79c0100700 2024-03-16T09:19:03.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.663+0000 7f79b77fe640 1 --2- 172.21.15.112:0/590584211 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f79ac0a2800 0x7f79ac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:03.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.664+0000 7f79c5c4d640 1 --2- 172.21.15.112:0/590584211 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f79ac0a2800 0x7f79ac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:03.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.664+0000 7f79b77fe640 1 -- 172.21.15.112:0/590584211 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f79bc0cf320 con 0x7f79c0100700 2024-03-16T09:19:03.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.664+0000 7f79c5c4d640 1 --2- 172.21.15.112:0/590584211 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f79ac0a2800 0x7f79ac0a4cc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f79b0011550 tx=0x7f79b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:03.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.672+0000 7f79b77fe640 1 -- 172.21.15.112:0/590584211 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f79bc09c3d0 con 0x7f79c0100700 2024-03-16T09:19:03.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.849+0000 7f79c86b2640 1 -- 172.21.15.112:0/590584211 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79c01a25c0 con 0x7f79c0100700 2024-03-16T09:19:03.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.852+0000 7f79b77fe640 1 -- 172.21.15.112:0/590584211 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22725 (secure 0 0 0) 0x7f79bc09bcd0 con 0x7f79c0100700 2024-03-16T09:19:03.855 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:19:03.855 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":88,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:17:28.259672+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T09:13:35.040628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T09:13:35.689419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:19:03.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.853+0000 7f79c86b2640 1 -- 172.21.15.112:0/590584211 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f79ac0a2800 msgr2=0x7f79ac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:03.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.853+0000 7f79c86b2640 1 --2- 172.21.15.112:0/590584211 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f79ac0a2800 0x7f79ac0a4cc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f79b0011550 tx=0x7f79b0005f90 comp rx=0 tx=0).stop 2024-03-16T09:19:03.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.853+0000 7f79c86b2640 1 -- 172.21.15.112:0/590584211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79c0100700 msgr2=0x7f79c0199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:03.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.853+0000 7f79c86b2640 1 --2- 172.21.15.112:0/590584211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79c0100700 0x7f79c0199dc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f79bc005fd0 tx=0x7f79bc00ef90 comp rx=0 tx=0).stop 2024-03-16T09:19:03.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.853+0000 7f79c86b2640 1 -- 172.21.15.112:0/590584211 shutdown_connections 2024-03-16T09:19:03.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.853+0000 7f79c86b2640 1 --2- 172.21.15.112:0/590584211 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f79ac0a2800 0x7f79ac0a4cc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:03.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.853+0000 7f79c86b2640 1 --2- 172.21.15.112:0/590584211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f79c01022b0 0x7f79c01a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:03.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.853+0000 7f79c86b2640 1 --2- 172.21.15.112:0/590584211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79c01018f0 0x7f79c019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:03.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.853+0000 7f79c86b2640 1 --2- 172.21.15.112:0/590584211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79c0100700 0x7f79c0199dc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:03.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.853+0000 7f79c86b2640 1 -- 172.21.15.112:0/590584211 >> 172.21.15.112:0/590584211 conn(0x7f79c00fbc70 msgr2=0x7f79c00fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:03.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.854+0000 7f79c86b2640 1 -- 172.21.15.112:0/590584211 shutdown_connections 2024-03-16T09:19:03.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.854+0000 7f79c86b2640 1 -- 172.21.15.112:0/590584211 wait complete. 2024-03-16T09:19:03.868 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T09:08:57.135312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T09:13:35.040628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T09:13:35.689419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T09:19:03.868 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T09:19:03.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.970+0000 7fc609d76640 1 -- 172.21.15.112:0/2873650950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6041017f0 msgr2=0x7fc604101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:03.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.970+0000 7fc609d76640 1 --2- 172.21.15.112:0/2873650950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6041017f0 0x7fc604101c10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009900 tx=0x7fc5f802bc30 comp rx=0 tx=0).stop 2024-03-16T09:19:03.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.970+0000 7fc609d76640 1 -- 172.21.15.112:0/2873650950 shutdown_connections 2024-03-16T09:19:03.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.970+0000 7fc609d76640 1 --2- 172.21.15.112:0/2873650950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6041033a0 0x7fc604109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:03.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.970+0000 7fc609d76640 1 --2- 172.21.15.112:0/2873650950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6041029e0 0x7fc604102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:03.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.970+0000 7fc609d76640 1 --2- 172.21.15.112:0/2873650950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6041017f0 0x7fc604101c10 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:03.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.970+0000 7fc609d76640 1 -- 172.21.15.112:0/2873650950 >> 172.21.15.112:0/2873650950 conn(0x7fc6040fcd80 msgr2=0x7fc6040ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:03.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.970+0000 7fc609d76640 1 -- 172.21.15.112:0/2873650950 shutdown_connections 2024-03-16T09:19:03.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.970+0000 7fc609d76640 1 -- 172.21.15.112:0/2873650950 wait complete. 2024-03-16T09:19:03.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.970+0000 7fc609d76640 1 Processor -- start 2024-03-16T09:19:03.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.970+0000 7fc609d76640 1 -- start start 2024-03-16T09:19:03.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc609d76640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6041017f0 0x7fc604075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:03.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc609d76640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6041029e0 0x7fc604073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:03.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc609d76640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6041033a0 0x7fc604074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:03.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc609d76640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc60410d550 con 0x7fc6041017f0 2024-03-16T09:19:03.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc609d76640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc60410d3d0 con 0x7fc6041029e0 2024-03-16T09:19:03.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc609d76640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc60410d6d0 con 0x7fc6041033a0 2024-03-16T09:19:03.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc6037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6041017f0 0x7fc604075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:03.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc6037fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6041017f0 0x7fc604075590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32888/0 (socket says 172.21.15.112:32888) 2024-03-16T09:19:03.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc6037fe640 1 -- 172.21.15.112:0/1324425089 learned_addr learned my addr 172.21.15.112:0/1324425089 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:19:03.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc602ffd640 1 --2- 172.21.15.112:0/1324425089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6041029e0 0x7fc604073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:03.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc603fff640 1 --2- 172.21.15.112:0/1324425089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6041033a0 0x7fc604074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:03.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc6037fe640 1 -- 172.21.15.112:0/1324425089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6041033a0 msgr2=0x7fc604074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:03.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc6037fe640 1 --2- 172.21.15.112:0/1324425089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6041033a0 0x7fc604074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:03.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc6037fe640 1 -- 172.21.15.112:0/1324425089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6041029e0 msgr2=0x7fc604073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:03.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc6037fe640 1 --2- 172.21.15.112:0/1324425089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6041029e0 0x7fc604073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:03.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc6037fe640 1 -- 172.21.15.112:0/1324425089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6040749a0 con 0x7fc6041017f0 2024-03-16T09:19:03.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc6037fe640 1 --2- 172.21.15.112:0/1324425089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6041017f0 0x7fc604075590 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8000c00 tx=0x7fc5f8004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:03.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc600ff9640 1 -- 172.21.15.112:0/1324425089 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8043020 con 0x7fc6041017f0 2024-03-16T09:19:03.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc609d76640 1 -- 172.21.15.112:0/1324425089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6041a1d20 con 0x7fc6041017f0 2024-03-16T09:19:03.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc600ff9640 1 -- 172.21.15.112:0/1324425089 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc5f8004250 con 0x7fc6041017f0 2024-03-16T09:19:03.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc600ff9640 1 -- 172.21.15.112:0/1324425089 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8003b90 con 0x7fc6041017f0 2024-03-16T09:19:03.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.971+0000 7fc609d76640 1 -- 172.21.15.112:0/1324425089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6041a21b0 con 0x7fc6041017f0 2024-03-16T09:19:03.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.972+0000 7fc609d76640 1 -- 172.21.15.112:0/1324425089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc604108b80 con 0x7fc6041017f0 2024-03-16T09:19:03.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.973+0000 7fc600ff9640 1 -- 172.21.15.112:0/1324425089 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7fc5f8046020 con 0x7fc6041017f0 2024-03-16T09:19:03.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.974+0000 7fc600ff9640 1 --2- 172.21.15.112:0/1324425089 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc5ec0a2800 0x7fc5ec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:03.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.974+0000 7fc602ffd640 1 --2- 172.21.15.112:0/1324425089 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc5ec0a2800 0x7fc5ec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:03.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.974+0000 7fc600ff9640 1 -- 172.21.15.112:0/1324425089 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc5f80f0bb0 con 0x7fc6041017f0 2024-03-16T09:19:03.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.975+0000 7fc602ffd640 1 --2- 172.21.15.112:0/1324425089 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc5ec0a2800 0x7fc5ec0a4cc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc604075300 tx=0x7fc5f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:03.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:03.979+0000 7fc600ff9640 1 -- 172.21.15.112:0/1324425089 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc5f804d020 con 0x7fc6041017f0 2024-03-16T09:19:04.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.158+0000 7fc609d76640 1 -- 172.21.15.112:0/1324425089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc6040623e0 con 0x7fc6041017f0 2024-03-16T09:19:04.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.158+0000 7fc600ff9640 1 -- 172.21.15.112:0/1324425089 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fc5f80bdc60 con 0x7fc6041017f0 2024-03-16T09:19:04.162 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-03-16T09:19:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.160+0000 7fc609d76640 1 -- 172.21.15.112:0/1324425089 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc5ec0a2800 msgr2=0x7fc5ec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.160+0000 7fc609d76640 1 --2- 172.21.15.112:0/1324425089 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc5ec0a2800 0x7fc5ec0a4cc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc604075300 tx=0x7fc5f4005f90 comp rx=0 tx=0).stop 2024-03-16T09:19:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.161+0000 7fc609d76640 1 -- 172.21.15.112:0/1324425089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6041017f0 msgr2=0x7fc604075590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.161+0000 7fc609d76640 1 --2- 172.21.15.112:0/1324425089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6041017f0 0x7fc604075590 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8000c00 tx=0x7fc5f8004a30 comp rx=0 tx=0).stop 2024-03-16T09:19:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.161+0000 7fc609d76640 1 -- 172.21.15.112:0/1324425089 shutdown_connections 2024-03-16T09:19:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.161+0000 7fc609d76640 1 --2- 172.21.15.112:0/1324425089 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc5ec0a2800 0x7fc5ec0a4cc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.161+0000 7fc609d76640 1 --2- 172.21.15.112:0/1324425089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6041033a0 0x7fc604074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.161+0000 7fc609d76640 1 --2- 172.21.15.112:0/1324425089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6041029e0 0x7fc604073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.161+0000 7fc609d76640 1 --2- 172.21.15.112:0/1324425089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6041017f0 0x7fc604075590 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.161+0000 7fc609d76640 1 -- 172.21.15.112:0/1324425089 >> 172.21.15.112:0/1324425089 conn(0x7fc6040fcd80 msgr2=0x7fc604105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.161+0000 7fc609d76640 1 -- 172.21.15.112:0/1324425089 shutdown_connections 2024-03-16T09:19:04.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.161+0000 7fc609d76640 1 -- 172.21.15.112:0/1324425089 wait complete. 2024-03-16T09:19:04.174 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:04 smithi112 ceph-mon[32307]: pgmap v454: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T09:19:04.174 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:04 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/590584211' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:19:04.175 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T09:19:04.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.276+0000 7fc3da122640 1 -- 172.21.15.112:0/3277899269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d41018d0 msgr2=0x7fc3d4101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:04.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.276+0000 7fc3da122640 1 --2- 172.21.15.112:0/3277899269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d41018d0 0x7fc3d4101d50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009900 tx=0x7fc3c802bc30 comp rx=0 tx=0).stop 2024-03-16T09:19:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.276+0000 7fc3da122640 1 -- 172.21.15.112:0/3277899269 shutdown_connections 2024-03-16T09:19:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.276+0000 7fc3da122640 1 --2- 172.21.15.112:0/3277899269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3d4102290 0x7fc3d4108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.276+0000 7fc3da122640 1 --2- 172.21.15.112:0/3277899269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d41018d0 0x7fc3d4101d50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.276+0000 7fc3da122640 1 --2- 172.21.15.112:0/3277899269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3d41006e0 0x7fc3d4100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.276+0000 7fc3da122640 1 -- 172.21.15.112:0/3277899269 >> 172.21.15.112:0/3277899269 conn(0x7fc3d40fbc70 msgr2=0x7fc3d40fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.277+0000 7fc3da122640 1 -- 172.21.15.112:0/3277899269 shutdown_connections 2024-03-16T09:19:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.277+0000 7fc3da122640 1 -- 172.21.15.112:0/3277899269 wait complete. 2024-03-16T09:19:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.277+0000 7fc3da122640 1 Processor -- start 2024-03-16T09:19:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.277+0000 7fc3da122640 1 -- start start 2024-03-16T09:19:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.277+0000 7fc3da122640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3d41006e0 0x7fc3d4104600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.277+0000 7fc3da122640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d41018d0 0x7fc3d4102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.277+0000 7fc3da122640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3d4102290 0x7fc3d41031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.277+0000 7fc3da122640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d410c440 con 0x7fc3d41018d0 2024-03-16T09:19:04.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.277+0000 7fc3da122640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d410c2c0 con 0x7fc3d4102290 2024-03-16T09:19:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.277+0000 7fc3da122640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d410c5c0 con 0x7fc3d41006e0 2024-03-16T09:19:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.277+0000 7fc3d2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d41018d0 0x7fc3d4102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.278+0000 7fc3d2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d41018d0 0x7fc3d4102cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32902/0 (socket says 172.21.15.112:32902) 2024-03-16T09:19:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.278+0000 7fc3d37fe640 1 --2- 172.21.15.112:0/429624137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3d41006e0 0x7fc3d4104600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.278+0000 7fc3d2ffd640 1 -- 172.21.15.112:0/429624137 learned_addr learned my addr 172.21.15.112:0/429624137 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:19:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.278+0000 7fc3d3fff640 1 --2- 172.21.15.112:0/429624137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3d4102290 0x7fc3d41031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.278+0000 7fc3d2ffd640 1 -- 172.21.15.112:0/429624137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3d41006e0 msgr2=0x7fc3d4104600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.278+0000 7fc3d2ffd640 1 --2- 172.21.15.112:0/429624137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3d41006e0 0x7fc3d4104600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.278+0000 7fc3d2ffd640 1 -- 172.21.15.112:0/429624137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3d4102290 msgr2=0x7fc3d41031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.278+0000 7fc3d2ffd640 1 --2- 172.21.15.112:0/429624137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3d4102290 0x7fc3d41031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.278+0000 7fc3d2ffd640 1 -- 172.21.15.112:0/429624137 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d4103a10 con 0x7fc3d41018d0 2024-03-16T09:19:04.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.278+0000 7fc3d37fe640 1 --2- 172.21.15.112:0/429624137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3d41006e0 0x7fc3d4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:19:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.278+0000 7fc3d3fff640 1 --2- 172.21.15.112:0/429624137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3d4102290 0x7fc3d41031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:19:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.278+0000 7fc3d2ffd640 1 --2- 172.21.15.112:0/429624137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d41018d0 0x7fc3d4102cb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009230 tx=0x7fc3c8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.278+0000 7fc3d0ff9640 1 -- 172.21.15.112:0/429624137 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8043020 con 0x7fc3d41018d0 2024-03-16T09:19:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.278+0000 7fc3d0ff9640 1 -- 172.21.15.112:0/429624137 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc3c8004250 con 0x7fc3d41018d0 2024-03-16T09:19:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.278+0000 7fc3da122640 1 -- 172.21.15.112:0/429624137 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d41a1ca0 con 0x7fc3d41018d0 2024-03-16T09:19:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.278+0000 7fc3d0ff9640 1 -- 172.21.15.112:0/429624137 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8003b50 con 0x7fc3d41018d0 2024-03-16T09:19:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.278+0000 7fc3da122640 1 -- 172.21.15.112:0/429624137 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d41a21a0 con 0x7fc3d41018d0 2024-03-16T09:19:04.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.279+0000 7fc3da122640 1 -- 172.21.15.112:0/429624137 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d41a2370 con 0x7fc3d41018d0 2024-03-16T09:19:04.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.280+0000 7fc3d0ff9640 1 -- 172.21.15.112:0/429624137 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7fc3c8046020 con 0x7fc3d41018d0 2024-03-16T09:19:04.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.281+0000 7fc3d0ff9640 1 --2- 172.21.15.112:0/429624137 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc3bc0a2800 0x7fc3bc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:04.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.281+0000 7fc3d37fe640 1 --2- 172.21.15.112:0/429624137 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc3bc0a2800 0x7fc3bc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:04.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.281+0000 7fc3d0ff9640 1 -- 172.21.15.112:0/429624137 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc3c80f0b40 con 0x7fc3d41018d0 2024-03-16T09:19:04.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.281+0000 7fc3d37fe640 1 --2- 172.21.15.112:0/429624137 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc3bc0a2800 0x7fc3bc0a4cc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4101730 tx=0x7fc3c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:04.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.285+0000 7fc3d0ff9640 1 -- 172.21.15.112:0/429624137 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc3c804d020 con 0x7fc3d41018d0 2024-03-16T09:19:04.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:04 smithi194 ceph-mon[36729]: pgmap v454: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T09:19:04.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:04 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/590584211' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:19:04.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.463+0000 7fc3da122640 1 -- 172.21.15.112:0/429624137 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc3d41a2540 con 0x7fc3d41018d0 2024-03-16T09:19:04.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.464+0000 7fc3d0ff9640 1 -- 172.21.15.112:0/429624137 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fc3c80bdbf0 con 0x7fc3d41018d0 2024-03-16T09:19:04.467 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-03-16T09:19:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.466+0000 7fc3da122640 1 -- 172.21.15.112:0/429624137 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc3bc0a2800 msgr2=0x7fc3bc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.466+0000 7fc3da122640 1 --2- 172.21.15.112:0/429624137 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc3bc0a2800 0x7fc3bc0a4cc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4101730 tx=0x7fc3c4005f90 comp rx=0 tx=0).stop 2024-03-16T09:19:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.466+0000 7fc3da122640 1 -- 172.21.15.112:0/429624137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d41018d0 msgr2=0x7fc3d4102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.466+0000 7fc3da122640 1 --2- 172.21.15.112:0/429624137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d41018d0 0x7fc3d4102cb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009230 tx=0x7fc3c8004ab0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.466+0000 7fc3da122640 1 -- 172.21.15.112:0/429624137 shutdown_connections 2024-03-16T09:19:04.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.466+0000 7fc3da122640 1 --2- 172.21.15.112:0/429624137 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fc3bc0a2800 0x7fc3bc0a4cc0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.466+0000 7fc3da122640 1 --2- 172.21.15.112:0/429624137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3d4102290 0x7fc3d41031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.466+0000 7fc3da122640 1 --2- 172.21.15.112:0/429624137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d41018d0 0x7fc3d4102cb0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.466+0000 7fc3da122640 1 --2- 172.21.15.112:0/429624137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3d41006e0 0x7fc3d4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.466+0000 7fc3da122640 1 -- 172.21.15.112:0/429624137 >> 172.21.15.112:0/429624137 conn(0x7fc3d40fbc70 msgr2=0x7fc3d40fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.466+0000 7fc3da122640 1 -- 172.21.15.112:0/429624137 shutdown_connections 2024-03-16T09:19:04.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.466+0000 7fc3da122640 1 -- 172.21.15.112:0/429624137 wait complete. 2024-03-16T09:19:04.480 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T09:19:04.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:04 smithi159 ceph-mon[36739]: pgmap v454: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-16T09:19:04.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:04 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/590584211' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:19:04.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf355b640 1 -- 172.21.15.112:0/3871877404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaec104c60 msgr2=0x7faaec107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:04.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf355b640 1 --2- 172.21.15.112:0/3871877404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaec104c60 0x7faaec107050 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7faadc009900 tx=0x7faadc02bc30 comp rx=0 tx=0).stop 2024-03-16T09:19:04.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf355b640 1 -- 172.21.15.112:0/3871877404 shutdown_connections 2024-03-16T09:19:04.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf355b640 1 --2- 172.21.15.112:0/3871877404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faaec107590 0x7faaec109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf355b640 1 --2- 172.21.15.112:0/3871877404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaec104c60 0x7faaec107050 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf355b640 1 --2- 172.21.15.112:0/3871877404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faaec102330 0x7faaec104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf355b640 1 -- 172.21.15.112:0/3871877404 >> 172.21.15.112:0/3871877404 conn(0x7faaec0fbc80 msgr2=0x7faaec0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:04.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf355b640 1 -- 172.21.15.112:0/3871877404 shutdown_connections 2024-03-16T09:19:04.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf355b640 1 -- 172.21.15.112:0/3871877404 wait complete. 2024-03-16T09:19:04.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf355b640 1 Processor -- start 2024-03-16T09:19:04.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf355b640 1 -- start start 2024-03-16T09:19:04.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf355b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaec102330 0x7faaec197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:04.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf355b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faaec104c60 0x7faaec1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:04.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf355b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faaec107590 0x7faaec19f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:04.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf355b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faaec10d560 con 0x7faaec102330 2024-03-16T09:19:04.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf355b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faaec10d3e0 con 0x7faaec104c60 2024-03-16T09:19:04.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf355b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7faaec10d6e0 con 0x7faaec107590 2024-03-16T09:19:04.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf12f7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaec102330 0x7faaec197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:04.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf12f7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaec102330 0x7faaec197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32908/0 (socket says 172.21.15.112:32908) 2024-03-16T09:19:04.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.581+0000 7faaf12f7640 1 -- 172.21.15.112:0/653363860 learned_addr learned my addr 172.21.15.112:0/653363860 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:19:04.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.582+0000 7faaf0af6640 1 --2- 172.21.15.112:0/653363860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faaec104c60 0x7faaec1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:04.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.582+0000 7faaf1af8640 1 --2- 172.21.15.112:0/653363860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faaec107590 0x7faaec19f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:04.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.582+0000 7faaf12f7640 1 -- 172.21.15.112:0/653363860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faaec107590 msgr2=0x7faaec19f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:04.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.582+0000 7faaf12f7640 1 --2- 172.21.15.112:0/653363860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faaec107590 0x7faaec19f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.582+0000 7faaf12f7640 1 -- 172.21.15.112:0/653363860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faaec104c60 msgr2=0x7faaec1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:04.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.582+0000 7faaf12f7640 1 --2- 172.21.15.112:0/653363860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faaec104c60 0x7faaec1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.582+0000 7faaf12f7640 1 -- 172.21.15.112:0/653363860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaec19f8b0 con 0x7faaec102330 2024-03-16T09:19:04.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.582+0000 7faaf1af8640 1 --2- 172.21.15.112:0/653363860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faaec107590 0x7faaec19f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T09:19:04.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.582+0000 7faaf12f7640 1 --2- 172.21.15.112:0/653363860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaec102330 0x7faaec197c70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7faae400ebc0 tx=0x7faae400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:04.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.582+0000 7faae27fc640 1 -- 172.21.15.112:0/653363860 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4010630 con 0x7faaec102330 2024-03-16T09:19:04.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.582+0000 7faae27fc640 1 -- 172.21.15.112:0/653363860 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faae4010cd0 con 0x7faaec102330 2024-03-16T09:19:04.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.582+0000 7faaf355b640 1 -- 172.21.15.112:0/653363860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaec19fba0 con 0x7faaec102330 2024-03-16T09:19:04.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.582+0000 7faae27fc640 1 -- 172.21.15.112:0/653363860 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4018cf0 con 0x7faaec102330 2024-03-16T09:19:04.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.582+0000 7faaf355b640 1 -- 172.21.15.112:0/653363860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaec1a00c0 con 0x7faaec102330 2024-03-16T09:19:04.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.583+0000 7faaf355b640 1 -- 172.21.15.112:0/653363860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaec0fd770 con 0x7faaec102330 2024-03-16T09:19:04.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.584+0000 7faae27fc640 1 -- 172.21.15.112:0/653363860 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7faae40107d0 con 0x7faaec102330 2024-03-16T09:19:04.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.584+0000 7faae27fc640 1 --2- 172.21.15.112:0/653363860 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7faad80a2800 0x7faad80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:04.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.585+0000 7faaf0af6640 1 --2- 172.21.15.112:0/653363860 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7faad80a2800 0x7faad80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:04.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.585+0000 7faae27fc640 1 -- 172.21.15.112:0/653363860 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7faae40cf180 con 0x7faaec102330 2024-03-16T09:19:04.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.585+0000 7faaf0af6640 1 --2- 172.21.15.112:0/653363860 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7faad80a2800 0x7faad80a4cc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faadc000c00 tx=0x7faadc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:04.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.589+0000 7faae27fc640 1 -- 172.21.15.112:0/653363860 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7faae409c230 con 0x7faaec102330 2024-03-16T09:19:04.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.764+0000 7faaf355b640 1 -- 172.21.15.112:0/653363860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faaec0623e0 con 0x7faaec102330 2024-03-16T09:19:04.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.764+0000 7faae27fc640 1 -- 172.21.15.112:0/653363860 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7faae409bb30 con 0x7faaec102330 2024-03-16T09:19:04.767 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-03-16T09:19:04.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.766+0000 7faaf355b640 1 -- 172.21.15.112:0/653363860 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7faad80a2800 msgr2=0x7faad80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:04.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.766+0000 7faaf355b640 1 --2- 172.21.15.112:0/653363860 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7faad80a2800 0x7faad80a4cc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faadc000c00 tx=0x7faadc005fd0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.766+0000 7faaf355b640 1 -- 172.21.15.112:0/653363860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaec102330 msgr2=0x7faaec197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:04.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.766+0000 7faaf355b640 1 --2- 172.21.15.112:0/653363860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaec102330 0x7faaec197c70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7faae400ebc0 tx=0x7faae400c6a0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.766+0000 7faaf355b640 1 -- 172.21.15.112:0/653363860 shutdown_connections 2024-03-16T09:19:04.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.766+0000 7faaf355b640 1 --2- 172.21.15.112:0/653363860 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7faad80a2800 0x7faad80a4cc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.766+0000 7faaf355b640 1 --2- 172.21.15.112:0/653363860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faaec107590 0x7faaec19f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.766+0000 7faaf355b640 1 --2- 172.21.15.112:0/653363860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faaec104c60 0x7faaec1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.766+0000 7faaf355b640 1 --2- 172.21.15.112:0/653363860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faaec102330 0x7faaec197c70 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.766+0000 7faaf355b640 1 -- 172.21.15.112:0/653363860 >> 172.21.15.112:0/653363860 conn(0x7faaec0fbc80 msgr2=0x7faaec102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:04.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.766+0000 7faaf355b640 1 -- 172.21.15.112:0/653363860 shutdown_connections 2024-03-16T09:19:04.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.766+0000 7faaf355b640 1 -- 172.21.15.112:0/653363860 wait complete. 2024-03-16T09:19:04.781 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T09:19:04.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.880+0000 7efef0661640 1 -- 172.21.15.112:0/3500028190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee81018e0 msgr2=0x7efee8101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:04.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.880+0000 7efef0661640 1 --2- 172.21.15.112:0/3500028190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee81018e0 0x7efee8101d60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7efed8009900 tx=0x7efed802bc30 comp rx=0 tx=0).stop 2024-03-16T09:19:04.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.880+0000 7efef0661640 1 -- 172.21.15.112:0/3500028190 shutdown_connections 2024-03-16T09:19:04.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.880+0000 7efef0661640 1 --2- 172.21.15.112:0/3500028190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efee81022a0 0x7efee8108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.880+0000 7efef0661640 1 --2- 172.21.15.112:0/3500028190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee81018e0 0x7efee8101d60 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.880+0000 7efef0661640 1 --2- 172.21.15.112:0/3500028190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee81006f0 0x7efee8100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.880+0000 7efef0661640 1 -- 172.21.15.112:0/3500028190 >> 172.21.15.112:0/3500028190 conn(0x7efee80fbc80 msgr2=0x7efee80fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:04.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.880+0000 7efef0661640 1 -- 172.21.15.112:0/3500028190 shutdown_connections 2024-03-16T09:19:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.880+0000 7efef0661640 1 -- 172.21.15.112:0/3500028190 wait complete. 2024-03-16T09:19:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.881+0000 7efef0661640 1 Processor -- start 2024-03-16T09:19:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.881+0000 7efef0661640 1 -- start start 2024-03-16T09:19:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.881+0000 7efef0661640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efee81006f0 0x7efee8199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.881+0000 7efef0661640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee81018e0 0x7efee819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.881+0000 7efef0661640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee81022a0 0x7efee81a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.881+0000 7efef0661640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efee810c450 con 0x7efee81018e0 2024-03-16T09:19:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.881+0000 7efef0661640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efee810c2d0 con 0x7efee81006f0 2024-03-16T09:19:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.881+0000 7efef0661640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efee810c5d0 con 0x7efee81022a0 2024-03-16T09:19:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.881+0000 7efeedbfc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee81018e0 0x7efee819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.881+0000 7efeee3fd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efee81006f0 0x7efee8199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.881+0000 7efeee3fd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efee81006f0 0x7efee8199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:56688/0 (socket says 172.21.15.112:56688) 2024-03-16T09:19:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.881+0000 7efeedbfc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee81018e0 0x7efee819a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32920/0 (socket says 172.21.15.112:32920) 2024-03-16T09:19:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.881+0000 7efeee3fd640 1 -- 172.21.15.112:0/4096135486 learned_addr learned my addr 172.21.15.112:0/4096135486 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:19:04.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.881+0000 7efeeebfe640 1 --2- 172.21.15.112:0/4096135486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee81022a0 0x7efee81a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.881+0000 7efeee3fd640 1 -- 172.21.15.112:0/4096135486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee81022a0 msgr2=0x7efee81a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.881+0000 7efeee3fd640 1 --2- 172.21.15.112:0/4096135486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee81022a0 0x7efee81a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.881+0000 7efeee3fd640 1 -- 172.21.15.112:0/4096135486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee81018e0 msgr2=0x7efee819a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.881+0000 7efeee3fd640 1 --2- 172.21.15.112:0/4096135486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee81018e0 0x7efee819a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.882+0000 7efeee3fd640 1 -- 172.21.15.112:0/4096135486 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee81a1a40 con 0x7efee81006f0 2024-03-16T09:19:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.882+0000 7efeedbfc640 1 --2- 172.21.15.112:0/4096135486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee81018e0 0x7efee819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:19:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.882+0000 7efeee3fd640 1 --2- 172.21.15.112:0/4096135486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efee81006f0 0x7efee8199e20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efee400e9c0 tx=0x7efee400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.882+0000 7efedf7fe640 1 -- 172.21.15.112:0/4096135486 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee400ccc0 con 0x7efee81006f0 2024-03-16T09:19:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.882+0000 7efef0661640 1 -- 172.21.15.112:0/4096135486 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee81a1d30 con 0x7efee81006f0 2024-03-16T09:19:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.882+0000 7efedf7fe640 1 -- 172.21.15.112:0/4096135486 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efee40035f0 con 0x7efee81006f0 2024-03-16T09:19:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.882+0000 7efedf7fe640 1 -- 172.21.15.112:0/4096135486 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee400ce60 con 0x7efee81006f0 2024-03-16T09:19:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.882+0000 7efef0661640 1 -- 172.21.15.112:0/4096135486 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee81a21a0 con 0x7efee81006f0 2024-03-16T09:19:04.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.883+0000 7efef0661640 1 -- 172.21.15.112:0/4096135486 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee81a23d0 con 0x7efee81006f0 2024-03-16T09:19:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.884+0000 7efedf7fe640 1 -- 172.21.15.112:0/4096135486 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7efee40037a0 con 0x7efee81006f0 2024-03-16T09:19:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.884+0000 7efedf7fe640 1 --2- 172.21.15.112:0/4096135486 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efed40a2800 0x7efed40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.885+0000 7efeedbfc640 1 --2- 172.21.15.112:0/4096135486 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efed40a2800 0x7efed40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.885+0000 7efedf7fe640 1 -- 172.21.15.112:0/4096135486 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7efee40ced90 con 0x7efee81006f0 2024-03-16T09:19:04.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.885+0000 7efeedbfc640 1 --2- 172.21.15.112:0/4096135486 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efed40a2800 0x7efed40a4cc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efed802c900 tx=0x7efed8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:04.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:04.888+0000 7efedf7fe640 1 -- 172.21.15.112:0/4096135486 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efee409be40 con 0x7efee81006f0 2024-03-16T09:19:05.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.077+0000 7efef0661640 1 -- 172.21.15.112:0/4096135486 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efee81a2710 con 0x7efee81006f0 2024-03-16T09:19:05.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.078+0000 7efedf7fe640 1 -- 172.21.15.112:0/4096135486 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+602 (secure 0 0 0) 0x7efee409b740 con 0x7efee81006f0 2024-03-16T09:19:05.081 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1871711739 2024-03-17T09:06:30.792961+0000 2024-03-16T09:19:05.081 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/2629104617 2024-03-17T09:06:30.792961+0000 2024-03-16T09:19:05.081 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3938289317 2024-03-17T09:06:30.792961+0000 2024-03-16T09:19:05.081 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/1790808883 2024-03-17T09:06:04.186776+0000 2024-03-16T09:19:05.081 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/1790808883 2024-03-17T09:06:04.186776+0000 2024-03-16T09:19:05.081 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/4123520095 2024-03-17T09:06:04.186776+0000 2024-03-16T09:19:05.081 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2535377744 2024-03-17T09:06:04.186776+0000 2024-03-16T09:19:05.081 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1888627499 2024-03-17T09:06:30.792961+0000 2024-03-16T09:19:05.081 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/2629104617 2024-03-17T09:06:30.792961+0000 2024-03-16T09:19:05.081 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3006861017 2024-03-17T09:06:04.186776+0000 2024-03-16T09:19:05.082 INFO:teuthology.orchestra.run.smithi112.stderr:listed 10 entries 2024-03-16T09:19:05.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.079+0000 7efef0661640 1 -- 172.21.15.112:0/4096135486 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efed40a2800 msgr2=0x7efed40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:05.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.079+0000 7efef0661640 1 --2- 172.21.15.112:0/4096135486 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efed40a2800 0x7efed40a4cc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efed802c900 tx=0x7efed8005fd0 comp rx=0 tx=0).stop 2024-03-16T09:19:05.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.079+0000 7efef0661640 1 -- 172.21.15.112:0/4096135486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efee81006f0 msgr2=0x7efee8199e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:05.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.079+0000 7efef0661640 1 --2- 172.21.15.112:0/4096135486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efee81006f0 0x7efee8199e20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efee400e9c0 tx=0x7efee400edc0 comp rx=0 tx=0).stop 2024-03-16T09:19:05.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.080+0000 7efef0661640 1 -- 172.21.15.112:0/4096135486 shutdown_connections 2024-03-16T09:19:05.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.080+0000 7efef0661640 1 --2- 172.21.15.112:0/4096135486 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7efed40a2800 0x7efed40a4cc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:05.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.080+0000 7efef0661640 1 --2- 172.21.15.112:0/4096135486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efee81022a0 0x7efee81a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:05.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.080+0000 7efef0661640 1 --2- 172.21.15.112:0/4096135486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efee81018e0 0x7efee819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:05.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.080+0000 7efef0661640 1 --2- 172.21.15.112:0/4096135486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efee81006f0 0x7efee8199e20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:05.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.080+0000 7efef0661640 1 -- 172.21.15.112:0/4096135486 >> 172.21.15.112:0/4096135486 conn(0x7efee80fbc80 msgr2=0x7efee80fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:05.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.080+0000 7efef0661640 1 -- 172.21.15.112:0/4096135486 shutdown_connections 2024-03-16T09:19:05.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.080+0000 7efef0661640 1 -- 172.21.15.112:0/4096135486 wait complete. 2024-03-16T09:19:05.094 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-16T09:19:05.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:19:05.094 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-16T09:19:05.142 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-03-16T09:19:05.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:19:05.142 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-16T09:19:05.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:05 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1324425089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:19:05.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:05 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/429624137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:19:05.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:05 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/653363860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:19:05.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:19:05.182 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-03-16T09:19:05.247 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-16T09:19:05.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:19:05.248 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T09:19:05.313 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:05 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1324425089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:19:05.313 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:05 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/429624137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:19:05.313 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:05 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/653363860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:19:05.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:19:05.317 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-03-16T09:19:05.428 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-16T09:19:05.428 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-03-16T09:19:05.428 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T09:19:05.445 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056356,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18694 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/97829b529d36f5e898b1e9d89734866b8f26c29f688e0f0db01571ee530de86c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IK6TRGJT2TM42JHPHWU5FH5G6:/var/lib/containers/storage/overlay/l/KU5WWDBG5PA4KFIQ7XVJOPCPDI,upperdir=/var/lib/containers/storage/overlay/97829b529d36f5e898b1e9d89734866b8f26c29f688e0f0db01571ee530de86c/diff,workdir=/var/lib/containers/storage/overlay/97829b529d36f5e898b1e9d89734866b8f26c29f688e0f0db01571ee530de86c/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/588410e7b83e88f3663a0c7fec943731bcb7a4e46b2ab7a7d2839b44455f3143/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IK6TRGJT2TM42JHPHWU5FH5G6:/var/lib/containers/storage/overlay/l/KU5WWDBG5PA4KFIQ7XVJOPCPDI,upperdir=/var/lib/containers/storage/overlay/588410e7b83e88f3663a0c7fec943731bcb7a4e46b2ab7a7d2839b44455f3143/diff,workdir=/var/lib/containers/storage/overlay/588410e7b83e88f3663a0c7fec943731bcb7a4e46b2ab7a7d2839b44455f3143/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/1f9b72fa5be0a050347c3586159f09485dd9162c2fee36f386c76da0ae4250d1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IK6TRGJT2TM42JHPHWU5FH5G6:/var/lib/containers/storage/overlay/l/KU5WWDBG5PA4KFIQ7XVJOPCPDI,upperdir=/var/lib/containers/storage/overlay/1f9b72fa5be0a050347c3586159f09485dd9162c2fee36f386c76da0ae4250d1/diff,workdir=/var/lib/containers/storage/overlay/1f9b72fa5be0a050347c3586159f09485dd9162c2fee36f386c76da0ae4250d1/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6b54d65fe31bac8fd59e2fe2ac91018fc73d566502e2c57ade6a039edd6ad382/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IK6TRGJT2TM42JHPHWU5FH5G6:/var/lib/containers/storage/overlay/l/KU5WWDBG5PA4KFIQ7XVJOPCPDI,upperdir=/var/lib/containers/storage/overlay/6b54d65fe31bac8fd59e2fe2ac91018fc73d566502e2c57ade6a039edd6ad382/diff,workdir=/var/lib/containers/storage/overlay/6b54d65fe31bac8fd59e2fe2ac91018fc73d566502e2c57ade6a039edd6ad382/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/23da687d819466aabd1b9ac057a1dd314f25f944f16443fc463da5b3be552d90/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IK6TRGJT2TM42JHPHWU5FH5G6:/var/lib/containers/storage/overlay/l/KU5WWDBG5PA4KFIQ7XVJOPCPDI,upperdir=/var/lib/containers/storage/overlay/23da687d819466aabd1b9ac057a1dd314f25f944f16443fc463da5b3be552d90/diff,workdir=/var/lib/containers/storage/overlay/23da687d819466aabd1b9ac057a1dd314f25f944f16443fc463da5b3be552d90/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/89f8c9eacab037403ea3c0e7205308a0fbdab7b74db5d83aadcc5eb5ae0aef26/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IK6TRGJT2TM42JHPHWU5FH5G6:/var/lib/containers/storage/overlay/l/KU5WWDBG5PA4KFIQ7XVJOPCPDI,upperdir=/var/lib/containers/storage/overlay/89f8c9eacab037403ea3c0e7205308a0fbdab7b74db5d83aadcc5eb5ae0aef26/diff,workdir=/var/lib/containers/storage/overlay/89f8c9eacab037403ea3c0e7205308a0fbdab7b74db5d83aadcc5eb5ae0aef26/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ccc536b7ba0eada4604a87fd353241998be7efa939a71a6967f29ba98e1e308d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IK6TRGJT2TM42JHPHWU5FH5G6:/var/lib/containers/storage/overlay/l/KU5WWDBG5PA4KFIQ7XVJOPCPDI,upperdir=/var/lib/containers/storage/overlay/ccc536b7ba0eada4604a87fd353241998be7efa939a71a6967f29ba98e1e308d/diff,workdir=/var/lib/containers/storage/overlay/ccc536b7ba0eada4604a87fd353241998be7efa939a71a6967f29ba98e1e308d/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9502b6c994e3f0d70d84c0ea0efc2a8cefa30a49f75e5d5c31913df3c810e7c2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IK6TRGJT2TM42JHPHWU5FH5G6:/var/lib/containers/storage/overlay/l/KU5WWDBG5PA4KFIQ7XVJOPCPDI,upperdir=/var/lib/containers/storage/overlay/9502b6c994e3f0d70d84c0ea0efc2a8cefa30a49f75e5d5c31913df3c810e7c2/diff,workdir=/var/lib/containers/storage/overlay/9502b6c994e3f0d70d84c0ea0efc2a8cefa30a49f75e5d5c31913df3c810e7c2/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/491e2141bc16c6a06867bf4a7c762a1915aa04cd2eca41b7027b6f0f5940525c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IK6TRGJT2TM42JHPHWU5FH5G6:/var/lib/containers/storage/overlay/l/KU5WWDBG5PA4KFIQ7XVJOPCPDI,upperdir=/var/lib/containers/storage/overlay/491e2141bc16c6a06867bf4a7c762a1915aa04cd2eca41b7027b6f0f5940525c/diff,workdir=/var/lib/containers/storage/overlay/491e2141bc16c6a06867bf4a7c762a1915aa04cd2eca41b7027b6f0f5940525c/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/02dc83fd6c5bbbcc882ce18fe973a4f171ed4d2c20519880135fbc681827cc20/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/7IK6TRGJT2TM42JHPHWU5FH5G6:/var/lib/containers/storage/overlay/l/KU5WWDBG5PA4KFIQ7XVJOPCPDI,upperdir=/var/lib/containers/storage/overlay/02dc83fd6c5bbbcc882ce18fe973a4f171ed4d2c20519880135fbc681827cc20/diff,workdir=/var/lib/containers/storage/overlay/02dc83fd6c5bbbcc882ce18fe973a4f171ed4d2c20519880135fbc681827cc20/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 2024-03-16T09:19:05.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:19:05.445 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-03-16T09:19:05.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:05 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1324425089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-16T09:19:05.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:05 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/429624137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-16T09:19:05.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:05 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/653363860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-16T09:19:05.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T09:19:05.500 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-03-16T09:19:05.570 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2024-03-16T09:19:05.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:19:05.572 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-16T09:19:05.583 INFO:tasks.cephadm:Teardown begin 2024-03-16T09:19:05.583 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-16T09:19:05.583 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:19:05.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.711+0000 7ff689fcd640 1 -- 172.21.15.112:0/3818831562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff684100700 msgr2=0x7ff684100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:05.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.711+0000 7ff689fcd640 1 --2- 172.21.15.112:0/3818831562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff684100700 0x7ff684100b20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff674009900 tx=0x7ff67402bc30 comp rx=0 tx=0).stop 2024-03-16T09:19:05.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.711+0000 7ff689fcd640 1 -- 172.21.15.112:0/3818831562 shutdown_connections 2024-03-16T09:19:05.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.711+0000 7ff689fcd640 1 --2- 172.21.15.112:0/3818831562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6841022b0 0x7ff684108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:05.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.711+0000 7ff689fcd640 1 --2- 172.21.15.112:0/3818831562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6841018f0 0x7ff684101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:05.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.711+0000 7ff689fcd640 1 --2- 172.21.15.112:0/3818831562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff684100700 0x7ff684100b20 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:05.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.711+0000 7ff689fcd640 1 -- 172.21.15.112:0/3818831562 >> 172.21.15.112:0/3818831562 conn(0x7ff6840fbc70 msgr2=0x7ff6840fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:05.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.711+0000 7ff689fcd640 1 -- 172.21.15.112:0/3818831562 shutdown_connections 2024-03-16T09:19:05.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.711+0000 7ff689fcd640 1 -- 172.21.15.112:0/3818831562 wait complete. 2024-03-16T09:19:05.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.711+0000 7ff689fcd640 1 Processor -- start 2024-03-16T09:19:05.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.711+0000 7ff689fcd640 1 -- start start 2024-03-16T09:19:05.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.711+0000 7ff689fcd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff684100700 0x7ff684197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:05.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.711+0000 7ff689fcd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6841018f0 0x7ff6841981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:05.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff689fcd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6841022b0 0x7ff68419f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:05.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff689fcd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff68410d570 con 0x7ff6841022b0 2024-03-16T09:19:05.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff689fcd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff68410d3f0 con 0x7ff684100700 2024-03-16T09:19:05.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff689fcd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff68410d6f0 con 0x7ff6841018f0 2024-03-16T09:19:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff683fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6841022b0 0x7ff68419f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff683fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6841022b0 0x7ff68419f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32944/0 (socket says 172.21.15.112:32944) 2024-03-16T09:19:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff683fff640 1 -- 172.21.15.112:0/1554535189 learned_addr learned my addr 172.21.15.112:0/1554535189 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:19:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff67affd640 1 --2- 172.21.15.112:0/1554535189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6841018f0 0x7ff6841981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff6837fe640 1 --2- 172.21.15.112:0/1554535189 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff684100700 0x7ff684197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff683fff640 1 -- 172.21.15.112:0/1554535189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6841018f0 msgr2=0x7ff6841981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff683fff640 1 --2- 172.21.15.112:0/1554535189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6841018f0 0x7ff6841981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff683fff640 1 -- 172.21.15.112:0/1554535189 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff684100700 msgr2=0x7ff684197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff683fff640 1 --2- 172.21.15.112:0/1554535189 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff684100700 0x7ff684197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff683fff640 1 -- 172.21.15.112:0/1554535189 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff68419f870 con 0x7ff6841022b0 2024-03-16T09:19:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff683fff640 1 --2- 172.21.15.112:0/1554535189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6841022b0 0x7ff68419f1d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00eb10 tx=0x7ff67c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff6817fa640 1 -- 172.21.15.112:0/1554535189 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c004b20 con 0x7ff6841022b0 2024-03-16T09:19:05.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff6817fa640 1 -- 172.21.15.112:0/1554535189 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff67c004cc0 con 0x7ff6841022b0 2024-03-16T09:19:05.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff689fcd640 1 -- 172.21.15.112:0/1554535189 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff68419fb60 con 0x7ff6841022b0 2024-03-16T09:19:05.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff6817fa640 1 -- 172.21.15.112:0/1554535189 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c0107f0 con 0x7ff6841022b0 2024-03-16T09:19:05.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.712+0000 7ff689fcd640 1 -- 172.21.15.112:0/1554535189 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6841a0030 con 0x7ff6841022b0 2024-03-16T09:19:05.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.713+0000 7ff689fcd640 1 -- 172.21.15.112:0/1554535189 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6841079c0 con 0x7ff6841022b0 2024-03-16T09:19:05.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.714+0000 7ff6817fa640 1 -- 172.21.15.112:0/1554535189 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7ff67c010990 con 0x7ff6841022b0 2024-03-16T09:19:05.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.715+0000 7ff6817fa640 1 --2- 172.21.15.112:0/1554535189 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff6700a2800 0x7ff6700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:05.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.715+0000 7ff6837fe640 1 --2- 172.21.15.112:0/1554535189 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff6700a2800 0x7ff6700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:05.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.715+0000 7ff6817fa640 1 -- 172.21.15.112:0/1554535189 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7ff67c0cf2a0 con 0x7ff6841022b0 2024-03-16T09:19:05.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.715+0000 7ff6837fe640 1 --2- 172.21.15.112:0/1554535189 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff6700a2800 0x7ff6700a4cc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff674000c00 tx=0x7ff674005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:05.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.719+0000 7ff6817fa640 1 -- 172.21.15.112:0/1554535189 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff67c053740 con 0x7ff6841022b0 2024-03-16T09:19:05.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.934+0000 7ff689fcd640 1 -- 172.21.15.112:0/1554535189 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff68402d1b0 con 0x7ff6841022b0 2024-03-16T09:19:05.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.935+0000 7ff6817fa640 1 -- 172.21.15.112:0/1554535189 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11898 (secure 0 0 0) 0x7ff67c09c350 con 0x7ff6841022b0 2024-03-16T09:19:05.938 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:19:05.938 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3939748854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3939748854},{"type":"v1","addr":"172.21.15.112:6839","nonce":3939748854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3118523258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3118523258},{"type":"v1","addr":"172.21.15.112:6841","nonce":3118523258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T09:13:36.450216+0000","modified":"2024-03-16T09:17:49.810536+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":62,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.194:6839/2528943309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2528943309},{"type":"v1","addr":"172.21.15.194:6839","nonce":2528943309}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/272895965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":272895965},{"type":"v1","addr":"172.21.15.159:6839","nonce":272895965}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25756":{"gid":25756,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/2561048853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":2561048853},{"type":"v1","addr":"172.21.15.194:6837","nonce":2561048853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24452]},"id":1}]} 2024-03-16T09:19:05.938 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 29 2024-03-16T09:19:05.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.937+0000 7ff689fcd640 1 -- 172.21.15.112:0/1554535189 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff6700a2800 msgr2=0x7ff6700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:05.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.937+0000 7ff689fcd640 1 --2- 172.21.15.112:0/1554535189 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff6700a2800 0x7ff6700a4cc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff674000c00 tx=0x7ff674005fd0 comp rx=0 tx=0).stop 2024-03-16T09:19:05.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.937+0000 7ff689fcd640 1 -- 172.21.15.112:0/1554535189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6841022b0 msgr2=0x7ff68419f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:05.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.937+0000 7ff689fcd640 1 --2- 172.21.15.112:0/1554535189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6841022b0 0x7ff68419f1d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00eb10 tx=0x7ff67c00c2d0 comp rx=0 tx=0).stop 2024-03-16T09:19:05.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.937+0000 7ff689fcd640 1 -- 172.21.15.112:0/1554535189 shutdown_connections 2024-03-16T09:19:05.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.937+0000 7ff689fcd640 1 --2- 172.21.15.112:0/1554535189 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7ff6700a2800 0x7ff6700a4cc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:05.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.937+0000 7ff689fcd640 1 --2- 172.21.15.112:0/1554535189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff6841022b0 0x7ff68419f1d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:05.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.937+0000 7ff689fcd640 1 --2- 172.21.15.112:0/1554535189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6841018f0 0x7ff6841981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:05.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.937+0000 7ff689fcd640 1 --2- 172.21.15.112:0/1554535189 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff684100700 0x7ff684197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:05.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.937+0000 7ff689fcd640 1 -- 172.21.15.112:0/1554535189 >> 172.21.15.112:0/1554535189 conn(0x7ff6840fbc70 msgr2=0x7ff684104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:05.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.937+0000 7ff689fcd640 1 -- 172.21.15.112:0/1554535189 shutdown_connections 2024-03-16T09:19:05.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:05.937+0000 7ff689fcd640 1 -- 172.21.15.112:0/1554535189 wait complete. 2024-03-16T09:19:05.951 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T09:19:06.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.052+0000 7f26f6485640 1 -- 172.21.15.112:0/2626479772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26f0100930 msgr2=0x7f26f0100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:06.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.052+0000 7f26f6485640 1 --2- 172.21.15.112:0/2626479772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26f0100930 0x7f26f0100db0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009900 tx=0x7f26e402bc30 comp rx=0 tx=0).stop 2024-03-16T09:19:06.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.052+0000 7f26f6485640 1 -- 172.21.15.112:0/2626479772 shutdown_connections 2024-03-16T09:19:06.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.052+0000 7f26f6485640 1 --2- 172.21.15.112:0/2626479772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f01012f0 0x7f26f0108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.052+0000 7f26f6485640 1 --2- 172.21.15.112:0/2626479772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26f0100930 0x7f26f0100db0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.052+0000 7f26f6485640 1 --2- 172.21.15.112:0/2626479772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f0101ce0 0x7f26f0102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.052+0000 7f26f6485640 1 -- 172.21.15.112:0/2626479772 >> 172.21.15.112:0/2626479772 conn(0x7f26f00fb400 msgr2=0x7f26f00fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:06.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.052+0000 7f26f6485640 1 -- 172.21.15.112:0/2626479772 shutdown_connections 2024-03-16T09:19:06.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.052+0000 7f26f6485640 1 -- 172.21.15.112:0/2626479772 wait complete. 2024-03-16T09:19:06.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.053+0000 7f26f6485640 1 Processor -- start 2024-03-16T09:19:06.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.053+0000 7f26f6485640 1 -- start start 2024-03-16T09:19:06.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.053+0000 7f26f6485640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26f0100930 0x7f26f0100390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:06.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.053+0000 7f26f6485640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f01012f0 0x7f26f019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:06.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.053+0000 7f26f6485640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f0101ce0 0x7f26f019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:06.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.053+0000 7f26f6485640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f26f010c4a0 con 0x7f26f0101ce0 2024-03-16T09:19:06.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.053+0000 7f26f6485640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f26f010c320 con 0x7f26f01012f0 2024-03-16T09:19:06.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.053+0000 7f26f6485640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f26f010c620 con 0x7f26f0100930 2024-03-16T09:19:06.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.053+0000 7f26f4a22640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f0101ce0 0x7f26f019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:06.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.053+0000 7f26ef7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f01012f0 0x7f26f019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:06.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.053+0000 7f26ef7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f01012f0 0x7f26f019c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:56730/0 (socket says 172.21.15.112:56730) 2024-03-16T09:19:06.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.053+0000 7f26f4a22640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f0101ce0 0x7f26f019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32958/0 (socket says 172.21.15.112:32958) 2024-03-16T09:19:06.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.053+0000 7f26ef7fe640 1 -- 172.21.15.112:0/153674268 learned_addr learned my addr 172.21.15.112:0/153674268 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:19:06.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.053+0000 7f26effff640 1 --2- 172.21.15.112:0/153674268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26f0100930 0x7f26f0100390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:06.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.054+0000 7f26ef7fe640 1 -- 172.21.15.112:0/153674268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26f0100930 msgr2=0x7f26f0100390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:06.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.054+0000 7f26ef7fe640 1 --2- 172.21.15.112:0/153674268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26f0100930 0x7f26f0100390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.054+0000 7f26ef7fe640 1 -- 172.21.15.112:0/153674268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f0101ce0 msgr2=0x7f26f019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:06.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.054+0000 7f26ef7fe640 1 --2- 172.21.15.112:0/153674268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f0101ce0 0x7f26f019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.054+0000 7f26ef7fe640 1 -- 172.21.15.112:0/153674268 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f019f850 con 0x7f26f01012f0 2024-03-16T09:19:06.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.054+0000 7f26f4a22640 1 --2- 172.21.15.112:0/153674268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f0101ce0 0x7f26f019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T09:19:06.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.054+0000 7f26ef7fe640 1 --2- 172.21.15.112:0/153674268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f01012f0 0x7f26f019c920 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f26e4005ab0 tx=0x7f26e4004a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:06.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.054+0000 7f26ed7fa640 1 -- 172.21.15.112:0/153674268 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4043020 con 0x7f26f01012f0 2024-03-16T09:19:06.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.054+0000 7f26ed7fa640 1 -- 172.21.15.112:0/153674268 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f26e4004250 con 0x7f26f01012f0 2024-03-16T09:19:06.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.054+0000 7f26f6485640 1 -- 172.21.15.112:0/153674268 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f01966f0 con 0x7f26f01012f0 2024-03-16T09:19:06.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.054+0000 7f26ed7fa640 1 -- 172.21.15.112:0/153674268 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4003c00 con 0x7f26f01012f0 2024-03-16T09:19:06.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.054+0000 7f26f6485640 1 -- 172.21.15.112:0/153674268 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f0196c40 con 0x7f26f01012f0 2024-03-16T09:19:06.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.055+0000 7f26f6485640 1 -- 172.21.15.112:0/153674268 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f0107a10 con 0x7f26f01012f0 2024-03-16T09:19:06.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.056+0000 7f26ed7fa640 1 -- 172.21.15.112:0/153674268 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7f26e4046020 con 0x7f26f01012f0 2024-03-16T09:19:06.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.056+0000 7f26ed7fa640 1 --2- 172.21.15.112:0/153674268 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f26d80a2800 0x7f26d80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:06.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.057+0000 7f26effff640 1 --2- 172.21.15.112:0/153674268 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f26d80a2800 0x7f26d80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:06.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.057+0000 7f26ed7fa640 1 -- 172.21.15.112:0/153674268 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f26e40f0ae0 con 0x7f26f01012f0 2024-03-16T09:19:06.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.057+0000 7f26effff640 1 --2- 172.21.15.112:0/153674268 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f26d80a2800 0x7f26d80a4cc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f26f0069960 tx=0x7f26e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:06.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.061+0000 7f26ed7fa640 1 -- 172.21.15.112:0/153674268 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f26e404d020 con 0x7f26f01012f0 2024-03-16T09:19:06.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.274+0000 7f26f6485640 1 -- 172.21.15.112:0/153674268 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26f002d590 con 0x7f26f01012f0 2024-03-16T09:19:06.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.275+0000 7f26ed7fa640 1 -- 172.21.15.112:0/153674268 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11898 (secure 0 0 0) 0x7f26f002d590 con 0x7f26f01012f0 2024-03-16T09:19:06.278 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:19:06.278 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/1370033819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1370033819},{"type":"v1","addr":"172.21.15.112:6835","nonce":1370033819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1353236253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1353236253},{"type":"v1","addr":"172.21.15.112:6837","nonce":1353236253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/3939748854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":3939748854},{"type":"v1","addr":"172.21.15.112:6839","nonce":3939748854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3118523258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3118523258},{"type":"v1","addr":"172.21.15.112:6841","nonce":3118523258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24386,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2947166433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2947166433},{"type":"v1","addr":"172.21.15.159:6833","nonce":2947166433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1514670407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1514670407},{"type":"v1","addr":"172.21.15.194:6833","nonce":1514670407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-16T09:13:36.450216+0000","modified":"2024-03-16T09:17:49.810536+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24466,"mds_2":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24413":{"gid":24413,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/567707106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":567707106},{"type":"v1","addr":"172.21.15.159:6835","nonce":567707106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/2682964784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":2682964784},{"type":"v1","addr":"172.21.15.194:6835","nonce":2682964784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":62,"addr":"172.21.15.159:6837/4023942203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":4023942203},{"type":"v1","addr":"172.21.15.159:6837","nonce":4023942203}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.194:6839/2528943309","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":2528943309},{"type":"v1","addr":"172.21.15.194:6839","nonce":2528943309}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/272895965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":272895965},{"type":"v1","addr":"172.21.15.159:6839","nonce":272895965}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25756":{"gid":25756,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/2561048853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":2561048853},{"type":"v1","addr":"172.21.15.194:6837","nonce":2561048853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24466,24494,24452]},"id":1}]} 2024-03-16T09:19:06.278 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 29 2024-03-16T09:19:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.277+0000 7f26f6485640 1 -- 172.21.15.112:0/153674268 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f26d80a2800 msgr2=0x7f26d80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.277+0000 7f26f6485640 1 --2- 172.21.15.112:0/153674268 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f26d80a2800 0x7f26d80a4cc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f26f0069960 tx=0x7f26e0005f90 comp rx=0 tx=0).stop 2024-03-16T09:19:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.277+0000 7f26f6485640 1 -- 172.21.15.112:0/153674268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f01012f0 msgr2=0x7f26f019c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.277+0000 7f26f6485640 1 --2- 172.21.15.112:0/153674268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f01012f0 0x7f26f019c920 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f26e4005ab0 tx=0x7f26e4004a50 comp rx=0 tx=0).stop 2024-03-16T09:19:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.277+0000 7f26f6485640 1 -- 172.21.15.112:0/153674268 shutdown_connections 2024-03-16T09:19:06.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.277+0000 7f26f6485640 1 --2- 172.21.15.112:0/153674268 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f26d80a2800 0x7f26d80a4cc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.277+0000 7f26f6485640 1 --2- 172.21.15.112:0/153674268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26f0101ce0 0x7f26f019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.277+0000 7f26f6485640 1 --2- 172.21.15.112:0/153674268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26f01012f0 0x7f26f019c920 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.277+0000 7f26f6485640 1 --2- 172.21.15.112:0/153674268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26f0100930 0x7f26f0100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.277+0000 7f26f6485640 1 -- 172.21.15.112:0/153674268 >> 172.21.15.112:0/153674268 conn(0x7f26f00fb400 msgr2=0x7f26f0104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:06.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.277+0000 7f26f6485640 1 -- 172.21.15.112:0/153674268 shutdown_connections 2024-03-16T09:19:06.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.277+0000 7f26f6485640 1 -- 172.21.15.112:0/153674268 wait complete. 2024-03-16T09:19:06.290 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:06 smithi112 ceph-mon[32307]: pgmap v455: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-16T09:19:06.290 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:06 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/4096135486' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T09:19:06.290 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:06 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1554535189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:19:06.291 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T09:19:06.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.390+0000 7fd70fa4b640 1 -- 172.21.15.112:0/1822155105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd708101800 msgr2=0x7fd708101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:06.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.390+0000 7fd70fa4b640 1 --2- 172.21.15.112:0/1822155105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd708101800 0x7fd708101c20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009900 tx=0x7fd6f802bc30 comp rx=0 tx=0).stop 2024-03-16T09:19:06.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.390+0000 7fd70fa4b640 1 -- 172.21.15.112:0/1822155105 shutdown_connections 2024-03-16T09:19:06.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.390+0000 7fd70fa4b640 1 --2- 172.21.15.112:0/1822155105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7081033b0 0x7fd708109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.390+0000 7fd70fa4b640 1 --2- 172.21.15.112:0/1822155105 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7081029f0 0x7fd708102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.390+0000 7fd70fa4b640 1 --2- 172.21.15.112:0/1822155105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd708101800 0x7fd708101c20 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.390+0000 7fd70fa4b640 1 -- 172.21.15.112:0/1822155105 >> 172.21.15.112:0/1822155105 conn(0x7fd7080fcd90 msgr2=0x7fd7080ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:06.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.390+0000 7fd70fa4b640 1 -- 172.21.15.112:0/1822155105 shutdown_connections 2024-03-16T09:19:06.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.390+0000 7fd70fa4b640 1 -- 172.21.15.112:0/1822155105 wait complete. 2024-03-16T09:19:06.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.391+0000 7fd70fa4b640 1 Processor -- start 2024-03-16T09:19:06.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.391+0000 7fd70fa4b640 1 -- start start 2024-03-16T09:19:06.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.391+0000 7fd70fa4b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd708101800 0x7fd708195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:06.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.391+0000 7fd70fa4b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7081029f0 0x7fd708195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:06.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.391+0000 7fd70fa4b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7081033b0 0x7fd70819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:06.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.391+0000 7fd70fa4b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd70810b350 con 0x7fd708101800 2024-03-16T09:19:06.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.391+0000 7fd70fa4b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd70810b1d0 con 0x7fd7081033b0 2024-03-16T09:19:06.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.391+0000 7fd70fa4b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd70810b4d0 con 0x7fd7081029f0 2024-03-16T09:19:06.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.391+0000 7fd70d7e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd708101800 0x7fd708195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:06.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.391+0000 7fd70d7e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd708101800 0x7fd708195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32966/0 (socket says 172.21.15.112:32966) 2024-03-16T09:19:06.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.391+0000 7fd70d7e7640 1 -- 172.21.15.112:0/1913818231 learned_addr learned my addr 172.21.15.112:0/1913818231 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:19:06.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.391+0000 7fd70cfe6640 1 --2- 172.21.15.112:0/1913818231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7081029f0 0x7fd708195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:06.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.391+0000 7fd70dfe8640 1 --2- 172.21.15.112:0/1913818231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7081033b0 0x7fd70819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:06.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.391+0000 7fd70d7e7640 1 -- 172.21.15.112:0/1913818231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7081029f0 msgr2=0x7fd708195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:06.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.391+0000 7fd70d7e7640 1 --2- 172.21.15.112:0/1913818231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7081029f0 0x7fd708195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.391+0000 7fd70d7e7640 1 -- 172.21.15.112:0/1913818231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7081033b0 msgr2=0x7fd70819cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:06.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.391+0000 7fd70d7e7640 1 --2- 172.21.15.112:0/1913818231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7081033b0 0x7fd70819cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.391+0000 7fd70d7e7640 1 -- 172.21.15.112:0/1913818231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70819d590 con 0x7fd708101800 2024-03-16T09:19:06.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.391+0000 7fd70d7e7640 1 --2- 172.21.15.112:0/1913818231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd708101800 0x7fd708195970 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8000c00 tx=0x7fd6f8004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:06.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.392+0000 7fd6fe7fc640 1 -- 172.21.15.112:0/1913818231 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8043020 con 0x7fd708101800 2024-03-16T09:19:06.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.392+0000 7fd6fe7fc640 1 -- 172.21.15.112:0/1913818231 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd6f8004250 con 0x7fd708101800 2024-03-16T09:19:06.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.392+0000 7fd70fa4b640 1 -- 172.21.15.112:0/1913818231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd70819d820 con 0x7fd708101800 2024-03-16T09:19:06.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.392+0000 7fd6fe7fc640 1 -- 172.21.15.112:0/1913818231 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8003b90 con 0x7fd708101800 2024-03-16T09:19:06.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.392+0000 7fd70fa4b640 1 -- 172.21.15.112:0/1913818231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd70819ddc0 con 0x7fd708101800 2024-03-16T09:19:06.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.393+0000 7fd70fa4b640 1 -- 172.21.15.112:0/1913818231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd708108b20 con 0x7fd708101800 2024-03-16T09:19:06.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.394+0000 7fd6fe7fc640 1 -- 172.21.15.112:0/1913818231 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7fd6f8046020 con 0x7fd708101800 2024-03-16T09:19:06.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.394+0000 7fd6fe7fc640 1 --2- 172.21.15.112:0/1913818231 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6f40a2880 0x7fd6f40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:06.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.395+0000 7fd70cfe6640 1 --2- 172.21.15.112:0/1913818231 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6f40a2880 0x7fd6f40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:06.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.395+0000 7fd6fe7fc640 1 -- 172.21.15.112:0/1913818231 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fd6f80f0a80 con 0x7fd708101800 2024-03-16T09:19:06.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.395+0000 7fd70cfe6640 1 --2- 172.21.15.112:0/1913818231 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6f40a2880 0x7fd6f40a4d40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd708196e80 tx=0x7fd704005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:06.401 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:06 smithi194 ceph-mon[36729]: pgmap v455: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-16T09:19:06.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:06 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/4096135486' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T09:19:06.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:06 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1554535189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:19:06.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.399+0000 7fd6fe7fc640 1 -- 172.21.15.112:0/1913818231 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd6f804d020 con 0x7fd708101800 2024-03-16T09:19:06.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:06 smithi159 ceph-mon[36739]: pgmap v455: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-16T09:19:06.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:06 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/4096135486' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-16T09:19:06.486 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:06 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1554535189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:19:06.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.573+0000 7fd70fa4b640 1 -- 172.21.15.112:0/1913818231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7080623e0 con 0x7fd708101800 2024-03-16T09:19:06.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.575+0000 7fd6fe7fc640 1 -- 172.21.15.112:0/1913818231 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22725 (secure 0 0 0) 0x7fd6f80bdb30 con 0x7fd708101800 2024-03-16T09:19:06.578 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-03-16T09:19:06.578 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":88,"fsid":"273be2a4-e374-11ee-95c9-87774f69a715","created":"2024-03-16T09:05:30.615922+0000","modified":"2024-03-16T09:17:28.259672+0000","last_up_change":"2024-03-16T09:12:47.100458+0000","last_in_change":"2024-03-16T09:12:26.733891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T09:08:57.135312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T09:13:35.040628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T09:13:35.689419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cac77ab4-0b17-41b1-ac04-3ca10c14ea93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6803","nonce":1848821115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6805","nonce":1848821115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6809","nonce":1848821115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1848821115},{"type":"v1","addr":"172.21.15.112:6807","nonce":1848821115}]},"public_addr":"172.21.15.112:6803/1848821115","cluster_addr":"172.21.15.112:6805/1848821115","heartbeat_back_addr":"172.21.15.112:6809/1848821115","heartbeat_front_addr":"172.21.15.112:6807/1848821115","state":["exists","up"]},{"osd":1,"uuid":"b46a1d96-d8cc-4b30-b9d7-b656c94ac581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6801","nonce":140175088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6803","nonce":140175088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6807","nonce":140175088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":140175088},{"type":"v1","addr":"172.21.15.159:6805","nonce":140175088}]},"public_addr":"172.21.15.159:6801/140175088","cluster_addr":"172.21.15.159:6803/140175088","heartbeat_back_addr":"172.21.15.159:6807/140175088","heartbeat_front_addr":"172.21.15.159:6805/140175088","state":["exists","up"]},{"osd":2,"uuid":"4debd434-e089-4ee8-a944-c22a8405f8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6801","nonce":4037530979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6803","nonce":4037530979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6807","nonce":4037530979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":4037530979},{"type":"v1","addr":"172.21.15.194:6805","nonce":4037530979}]},"public_addr":"172.21.15.194:6801/4037530979","cluster_addr":"172.21.15.194:6803/4037530979","heartbeat_back_addr":"172.21.15.194:6807/4037530979","heartbeat_front_addr":"172.21.15.194:6805/4037530979","state":["exists","up"]},{"osd":3,"uuid":"f81eddbc-f120-4077-a6f5-2bee53c270e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6811","nonce":3090522464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6813","nonce":3090522464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6817","nonce":3090522464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":3090522464},{"type":"v1","addr":"172.21.15.112:6815","nonce":3090522464}]},"public_addr":"172.21.15.112:6811/3090522464","cluster_addr":"172.21.15.112:6813/3090522464","heartbeat_back_addr":"172.21.15.112:6817/3090522464","heartbeat_front_addr":"172.21.15.112:6815/3090522464","state":["exists","up"]},{"osd":4,"uuid":"9e27e743-08f5-4722-a903-1adac60ada69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6809","nonce":4036563014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6811","nonce":4036563014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6815","nonce":4036563014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4036563014},{"type":"v1","addr":"172.21.15.159:6813","nonce":4036563014}]},"public_addr":"172.21.15.159:6809/4036563014","cluster_addr":"172.21.15.159:6811/4036563014","heartbeat_back_addr":"172.21.15.159:6815/4036563014","heartbeat_front_addr":"172.21.15.159:6813/4036563014","state":["exists","up"]},{"osd":5,"uuid":"f7440b71-313a-42ff-9d9b-4badc07608cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6809","nonce":1459524918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6811","nonce":1459524918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6815","nonce":1459524918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1459524918},{"type":"v1","addr":"172.21.15.194:6813","nonce":1459524918}]},"public_addr":"172.21.15.194:6809/1459524918","cluster_addr":"172.21.15.194:6811/1459524918","heartbeat_back_addr":"172.21.15.194:6815/1459524918","heartbeat_front_addr":"172.21.15.194:6813/1459524918","state":["exists","up"]},{"osd":6,"uuid":"1e250eec-8d30-465d-8352-3f7db892c370","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6819","nonce":3853469923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6821","nonce":3853469923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6825","nonce":3853469923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3853469923},{"type":"v1","addr":"172.21.15.112:6823","nonce":3853469923}]},"public_addr":"172.21.15.112:6819/3853469923","cluster_addr":"172.21.15.112:6821/3853469923","heartbeat_back_addr":"172.21.15.112:6825/3853469923","heartbeat_front_addr":"172.21.15.112:6823/3853469923","state":["exists","up"]},{"osd":7,"uuid":"f2eb2220-107d-411e-9572-8fae0faf0a20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6817","nonce":2484095468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6819","nonce":2484095468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6823","nonce":2484095468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2484095468},{"type":"v1","addr":"172.21.15.159:6821","nonce":2484095468}]},"public_addr":"172.21.15.159:6817/2484095468","cluster_addr":"172.21.15.159:6819/2484095468","heartbeat_back_addr":"172.21.15.159:6823/2484095468","heartbeat_front_addr":"172.21.15.159:6821/2484095468","state":["exists","up"]},{"osd":8,"uuid":"c9ce2ec1-b2db-4ce1-b306-ea6e333b02de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6817","nonce":3334422298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6819","nonce":3334422298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6823","nonce":3334422298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":3334422298},{"type":"v1","addr":"172.21.15.194:6821","nonce":3334422298}]},"public_addr":"172.21.15.194:6817/3334422298","cluster_addr":"172.21.15.194:6819/3334422298","heartbeat_back_addr":"172.21.15.194:6823/3334422298","heartbeat_front_addr":"172.21.15.194:6821/3334422298","state":["exists","up"]},{"osd":9,"uuid":"e2b22c20-177b-41ec-b579-f18bc619d79c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6827","nonce":952596860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6829","nonce":952596860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6833","nonce":952596860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":952596860},{"type":"v1","addr":"172.21.15.112:6831","nonce":952596860}]},"public_addr":"172.21.15.112:6827/952596860","cluster_addr":"172.21.15.112:6829/952596860","heartbeat_back_addr":"172.21.15.112:6833/952596860","heartbeat_front_addr":"172.21.15.112:6831/952596860","state":["exists","up"]},{"osd":10,"uuid":"1922eaad-cff8-46cc-b837-238133613e28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6825","nonce":1906671120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6827","nonce":1906671120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6831","nonce":1906671120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1906671120},{"type":"v1","addr":"172.21.15.159:6829","nonce":1906671120}]},"public_addr":"172.21.15.159:6825/1906671120","cluster_addr":"172.21.15.159:6827/1906671120","heartbeat_back_addr":"172.21.15.159:6831/1906671120","heartbeat_front_addr":"172.21.15.159:6829/1906671120","state":["exists","up"]},{"osd":11,"uuid":"eb0e58e0-7d51-4a00-9004-03607251189f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6825","nonce":3230458513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6827","nonce":3230458513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6831","nonce":3230458513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3230458513},{"type":"v1","addr":"172.21.15.194:6829","nonce":3230458513}]},"public_addr":"172.21.15.194:6825/3230458513","cluster_addr":"172.21.15.194:6827/3230458513","heartbeat_back_addr":"172.21.15.194:6831/3230458513","heartbeat_front_addr":"172.21.15.194:6829/3230458513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:06.646057+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:30.650135+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:08:54.261982+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:19.870430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:09:44.882940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:09.993398+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:10:35.821744+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:01.766796+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:27.213005+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:11:53.040855+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:18.975344+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T09:12:45.153215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1871711739":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3938289317":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6801/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:6800/1790808883":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/4123520095":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/2535377744":"2024-03-17T09:06:04.186776+0000","172.21.15.112:0/1888627499":"2024-03-17T09:06:30.792961+0000","172.21.15.112:6800/2629104617":"2024-03-17T09:06:30.792961+0000","172.21.15.112:0/3006861017":"2024-03-17T09:06:04.186776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T09:19:06.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.577+0000 7fd70fa4b640 1 -- 172.21.15.112:0/1913818231 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6f40a2880 msgr2=0x7fd6f40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:06.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.577+0000 7fd70fa4b640 1 --2- 172.21.15.112:0/1913818231 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6f40a2880 0x7fd6f40a4d40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd708196e80 tx=0x7fd704005f90 comp rx=0 tx=0).stop 2024-03-16T09:19:06.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.577+0000 7fd70fa4b640 1 -- 172.21.15.112:0/1913818231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd708101800 msgr2=0x7fd708195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:06.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.577+0000 7fd70fa4b640 1 --2- 172.21.15.112:0/1913818231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd708101800 0x7fd708195970 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8000c00 tx=0x7fd6f8004a30 comp rx=0 tx=0).stop 2024-03-16T09:19:06.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.577+0000 7fd70fa4b640 1 -- 172.21.15.112:0/1913818231 shutdown_connections 2024-03-16T09:19:06.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.577+0000 7fd70fa4b640 1 --2- 172.21.15.112:0/1913818231 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd6f40a2880 0x7fd6f40a4d40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.577+0000 7fd70fa4b640 1 --2- 172.21.15.112:0/1913818231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7081033b0 0x7fd70819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.577+0000 7fd70fa4b640 1 --2- 172.21.15.112:0/1913818231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7081029f0 0x7fd708195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.577+0000 7fd70fa4b640 1 --2- 172.21.15.112:0/1913818231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd708101800 0x7fd708195970 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.577+0000 7fd70fa4b640 1 -- 172.21.15.112:0/1913818231 >> 172.21.15.112:0/1913818231 conn(0x7fd7080fcd90 msgr2=0x7fd708105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:06.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.577+0000 7fd70fa4b640 1 -- 172.21.15.112:0/1913818231 shutdown_connections 2024-03-16T09:19:06.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.577+0000 7fd70fa4b640 1 -- 172.21.15.112:0/1913818231 wait complete. 2024-03-16T09:19:06.592 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-16T09:19:06.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.690+0000 7f3f18236640 1 -- 172.21.15.112:0/4114603875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100ff870 msgr2=0x7f3f100ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.690+0000 7f3f18236640 1 --2- 172.21.15.112:0/4114603875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100ff870 0x7f3f100ffcf0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009900 tx=0x7f3f0402bc30 comp rx=0 tx=0).stop 2024-03-16T09:19:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.690+0000 7f3f18236640 1 -- 172.21.15.112:0/4114603875 shutdown_connections 2024-03-16T09:19:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.690+0000 7f3f18236640 1 --2- 172.21.15.112:0/4114603875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f10100230 0x7f3f10109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.690+0000 7f3f18236640 1 --2- 172.21.15.112:0/4114603875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100ff870 0x7f3f100ffcf0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.690+0000 7f3f18236640 1 --2- 172.21.15.112:0/4114603875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f10102e00 0x7f3f10103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.690+0000 7f3f18236640 1 -- 172.21.15.112:0/4114603875 >> 172.21.15.112:0/4114603875 conn(0x7f3f100fb410 msgr2=0x7f3f100fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.690+0000 7f3f18236640 1 -- 172.21.15.112:0/4114603875 shutdown_connections 2024-03-16T09:19:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.690+0000 7f3f18236640 1 -- 172.21.15.112:0/4114603875 wait complete. 2024-03-16T09:19:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f18236640 1 Processor -- start 2024-03-16T09:19:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f18236640 1 -- start start 2024-03-16T09:19:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f18236640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100ff870 0x7f3f10197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f18236640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f10100230 0x7f3f10198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:06.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f18236640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f10102e00 0x7f3f1019f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f18236640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1010d5b0 con 0x7f3f100ff870 2024-03-16T09:19:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f18236640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1010d430 con 0x7f3f10100230 2024-03-16T09:19:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f18236640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1010d730 con 0x7f3f10102e00 2024-03-16T09:19:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f15fd2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100ff870 0x7f3f10197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f15fd2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100ff870 0x7f3f10197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32982/0 (socket says 172.21.15.112:32982) 2024-03-16T09:19:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f15fd2640 1 -- 172.21.15.112:0/3499248685 learned_addr learned my addr 172.21.15.112:0/3499248685 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:19:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f157d1640 1 --2- 172.21.15.112:0/3499248685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f10100230 0x7f3f10198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f167d3640 1 --2- 172.21.15.112:0/3499248685 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f10102e00 0x7f3f1019f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f15fd2640 1 -- 172.21.15.112:0/3499248685 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f10102e00 msgr2=0x7f3f1019f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f15fd2640 1 --2- 172.21.15.112:0/3499248685 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f10102e00 0x7f3f1019f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f15fd2640 1 -- 172.21.15.112:0/3499248685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f10100230 msgr2=0x7f3f10198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f15fd2640 1 --2- 172.21.15.112:0/3499248685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f10100230 0x7f3f10198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f15fd2640 1 -- 172.21.15.112:0/3499248685 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1019f7f0 con 0x7f3f100ff870 2024-03-16T09:19:06.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f15fd2640 1 --2- 172.21.15.112:0/3499248685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100ff870 0x7f3f10197bd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800ebc0 tx=0x7f3f0800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.691+0000 7f3f0effd640 1 -- 172.21.15.112:0/3499248685 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08003b70 con 0x7f3f100ff870 2024-03-16T09:19:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.692+0000 7f3f0effd640 1 -- 172.21.15.112:0/3499248685 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3f08003d10 con 0x7f3f100ff870 2024-03-16T09:19:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.692+0000 7f3f18236640 1 -- 172.21.15.112:0/3499248685 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1019fae0 con 0x7f3f100ff870 2024-03-16T09:19:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.692+0000 7f3f0effd640 1 -- 172.21.15.112:0/3499248685 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08019980 con 0x7f3f100ff870 2024-03-16T09:19:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.692+0000 7f3f18236640 1 -- 172.21.15.112:0/3499248685 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f101a0000 con 0x7f3f100ff870 2024-03-16T09:19:06.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.693+0000 7f3f18236640 1 -- 172.21.15.112:0/3499248685 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f10108c60 con 0x7f3f100ff870 2024-03-16T09:19:06.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.693+0000 7f3f0effd640 1 -- 172.21.15.112:0/3499248685 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7f3f08019b20 con 0x7f3f100ff870 2024-03-16T09:19:06.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.694+0000 7f3f0effd640 1 --2- 172.21.15.112:0/3499248685 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3ef80a2800 0x7f3ef80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:06.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.694+0000 7f3f157d1640 1 --2- 172.21.15.112:0/3499248685 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3ef80a2800 0x7f3ef80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:06.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.694+0000 7f3f0effd640 1 -- 172.21.15.112:0/3499248685 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f3f080cf2e0 con 0x7f3f100ff870 2024-03-16T09:19:06.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.695+0000 7f3f157d1640 1 --2- 172.21.15.112:0/3499248685 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3ef80a2800 0x7f3ef80a4cc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3f0402c900 tx=0x7f3f04005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:06.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.698+0000 7f3f0effd640 1 -- 172.21.15.112:0/3499248685 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3f0809c390 con 0x7f3f100ff870 2024-03-16T09:19:06.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.911+0000 7f3f18236640 1 -- 172.21.15.112:0/3499248685 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3f10062480 con 0x7f3f100ff870 2024-03-16T09:19:06.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.913+0000 7f3f0effd640 1 -- 172.21.15.112:0/3499248685 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7f3f0809bc90 con 0x7f3f100ff870 2024-03-16T09:19:06.916 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-16T09:19:06.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.916+0000 7f3f18236640 1 -- 172.21.15.112:0/3499248685 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3ef80a2800 msgr2=0x7f3ef80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:06.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.916+0000 7f3f18236640 1 --2- 172.21.15.112:0/3499248685 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3ef80a2800 0x7f3ef80a4cc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3f0402c900 tx=0x7f3f04005fd0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.916+0000 7f3f18236640 1 -- 172.21.15.112:0/3499248685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100ff870 msgr2=0x7f3f10197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:06.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.916+0000 7f3f18236640 1 --2- 172.21.15.112:0/3499248685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100ff870 0x7f3f10197bd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800ebc0 tx=0x7f3f0800c6a0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.916+0000 7f3f18236640 1 -- 172.21.15.112:0/3499248685 shutdown_connections 2024-03-16T09:19:06.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.916+0000 7f3f18236640 1 --2- 172.21.15.112:0/3499248685 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f3ef80a2800 0x7f3ef80a4cc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.916+0000 7f3f18236640 1 --2- 172.21.15.112:0/3499248685 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f10102e00 0x7f3f1019f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.916+0000 7f3f18236640 1 --2- 172.21.15.112:0/3499248685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f10100230 0x7f3f10198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.916+0000 7f3f18236640 1 --2- 172.21.15.112:0/3499248685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f100ff870 0x7f3f10197bd0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:06.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.916+0000 7f3f18236640 1 -- 172.21.15.112:0/3499248685 >> 172.21.15.112:0/3499248685 conn(0x7f3f100fb410 msgr2=0x7f3f10105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:06.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.916+0000 7f3f18236640 1 -- 172.21.15.112:0/3499248685 shutdown_connections 2024-03-16T09:19:06.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:06.916+0000 7f3f18236640 1 -- 172.21.15.112:0/3499248685 wait complete. 2024-03-16T09:19:06.930 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-16T09:19:07.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.032+0000 7f459f7cd640 1 -- 172.21.15.112:0/3588214191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4598102290 msgr2=0x7f4598108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:07.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.032+0000 7f459f7cd640 1 --2- 172.21.15.112:0/3588214191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4598102290 0x7f4598108b20 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4590009ca0 tx=0x7f459002c0f0 comp rx=0 tx=0).stop 2024-03-16T09:19:07.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.032+0000 7f459f7cd640 1 -- 172.21.15.112:0/3588214191 shutdown_connections 2024-03-16T09:19:07.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.032+0000 7f459f7cd640 1 --2- 172.21.15.112:0/3588214191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4598102290 0x7f4598108b20 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:07.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.032+0000 7f459f7cd640 1 --2- 172.21.15.112:0/3588214191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45981018d0 0x7f4598101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:07.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.032+0000 7f459f7cd640 1 --2- 172.21.15.112:0/3588214191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45981006e0 0x7f4598100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:07.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.032+0000 7f459f7cd640 1 -- 172.21.15.112:0/3588214191 >> 172.21.15.112:0/3588214191 conn(0x7f45980fbc70 msgr2=0x7f45980fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:07.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.032+0000 7f459f7cd640 1 -- 172.21.15.112:0/3588214191 shutdown_connections 2024-03-16T09:19:07.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.032+0000 7f459f7cd640 1 -- 172.21.15.112:0/3588214191 wait complete. 2024-03-16T09:19:07.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459f7cd640 1 Processor -- start 2024-03-16T09:19:07.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459f7cd640 1 -- start start 2024-03-16T09:19:07.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459f7cd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45981006e0 0x7f4598196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:07.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459f7cd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45981018d0 0x7f4598197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:07.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459f7cd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4598102290 0x7f459819e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:07.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459f7cd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f459810c490 con 0x7f45981018d0 2024-03-16T09:19:07.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459f7cd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f459810c310 con 0x7f4598102290 2024-03-16T09:19:07.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459f7cd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f459810c610 con 0x7f45981006e0 2024-03-16T09:19:07.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459cd68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45981018d0 0x7f4598197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:07.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459cd68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45981018d0 0x7f4598197010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33004/0 (socket says 172.21.15.112:33004) 2024-03-16T09:19:07.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459cd68640 1 -- 172.21.15.112:0/1301607535 learned_addr learned my addr 172.21.15.112:0/1301607535 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:19:07.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459d569640 1 --2- 172.21.15.112:0/1301607535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45981006e0 0x7f4598196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:07.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459dd6a640 1 --2- 172.21.15.112:0/1301607535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4598102290 0x7f459819e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:07.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459cd68640 1 -- 172.21.15.112:0/1301607535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45981006e0 msgr2=0x7f4598196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:07.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459cd68640 1 --2- 172.21.15.112:0/1301607535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45981006e0 0x7f4598196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:07.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459cd68640 1 -- 172.21.15.112:0/1301607535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4598102290 msgr2=0x7f459819e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:07.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459cd68640 1 --2- 172.21.15.112:0/1301607535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4598102290 0x7f459819e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:07.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459cd68640 1 -- 172.21.15.112:0/1301607535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459819e6f0 con 0x7f45981018d0 2024-03-16T09:19:07.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459dd6a640 1 --2- 172.21.15.112:0/1301607535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4598102290 0x7f459819e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:19:07.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.033+0000 7f459d569640 1 --2- 172.21.15.112:0/1301607535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45981006e0 0x7f4598196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T09:19:07.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.034+0000 7f459cd68640 1 --2- 172.21.15.112:0/1301607535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45981018d0 0x7f4598197010 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f458800d770 tx=0x7f458800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:07.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.034+0000 7f458e7fc640 1 -- 172.21.15.112:0/1301607535 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4588017070 con 0x7f45981018d0 2024-03-16T09:19:07.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.034+0000 7f458e7fc640 1 -- 172.21.15.112:0/1301607535 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4588005520 con 0x7f45981018d0 2024-03-16T09:19:07.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.034+0000 7f458e7fc640 1 -- 172.21.15.112:0/1301607535 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4588010460 con 0x7f45981018d0 2024-03-16T09:19:07.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.034+0000 7f459f7cd640 1 -- 172.21.15.112:0/1301607535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f459819e9e0 con 0x7f45981018d0 2024-03-16T09:19:07.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.034+0000 7f459f7cd640 1 -- 172.21.15.112:0/1301607535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f459819ef30 con 0x7f45981018d0 2024-03-16T09:19:07.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.035+0000 7f459f7cd640 1 -- 172.21.15.112:0/1301607535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45981079a0 con 0x7f45981018d0 2024-03-16T09:19:07.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.036+0000 7f458e7fc640 1 -- 172.21.15.112:0/1301607535 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7f4588004ea0 con 0x7f45981018d0 2024-03-16T09:19:07.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.036+0000 7f458e7fc640 1 --2- 172.21.15.112:0/1301607535 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f45840a2880 0x7f45840a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:07.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.036+0000 7f459d569640 1 --2- 172.21.15.112:0/1301607535 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f45840a2880 0x7f45840a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:07.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.036+0000 7f458e7fc640 1 -- 172.21.15.112:0/1301607535 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f45880ceac0 con 0x7f45981018d0 2024-03-16T09:19:07.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.037+0000 7f459d569640 1 --2- 172.21.15.112:0/1301607535 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f45840a2880 0x7f45840a4d40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4598101730 tx=0x7f4594005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:07.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.040+0000 7f458e7fc640 1 -- 172.21.15.112:0/1301607535 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f458809baf0 con 0x7f45981018d0 2024-03-16T09:19:07.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:07 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/153674268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:19:07.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:07 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1913818231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:19:07.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:07 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3499248685' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-16T09:19:07.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:07 smithi159 ceph-mon[36739]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-16T09:19:07.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:07 smithi159 ceph-mon[36739]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T09:19:07.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:07 smithi159 ceph-mon[36739]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-03-16T09:19:07.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:07 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3499248685' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-16T09:19:07.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:07 smithi159 ceph-mon[36739]: osdmap e89: 12 total, 12 up, 12 in 2024-03-16T09:19:07.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:07 smithi159 ceph-mon[36739]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-16T09:19:07.238 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:19:06 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-e[65884]: ignoring --setuser ceph since I am not root 2024-03-16T09:19:07.238 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:19:06 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-e[65884]: ignoring --setgroup ceph since I am not root 2024-03-16T09:19:07.238 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:19:06 smithi159 ceph-mds[65888]: -- 172.21.15.159:0/286597016 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5569f9adba40 con 0x5569f9aaf000 2024-03-16T09:19:07.238 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:19:06 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-e[65884]: starting mds.e at 2024-03-16T09:19:07.238 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:19:06 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-b[66504]: ignoring --setuser ceph since I am not root 2024-03-16T09:19:07.238 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:19:06 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-b[66504]: ignoring --setgroup ceph since I am not root 2024-03-16T09:19:07.238 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:19:06 smithi159 ceph-mds[66508]: -- 172.21.15.159:0/1070782916 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555c274bba40 con 0x555c2748e800 2024-03-16T09:19:07.239 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:19:06 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-b[66504]: starting mds.b at 2024-03-16T09:19:07.239 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:19:06 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: ignoring --setuser ceph since I am not root 2024-03-16T09:19:07.239 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:19:06 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: ignoring --setgroup ceph since I am not root 2024-03-16T09:19:07.239 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:19:06 smithi159 ceph-mds[66201]: -- 172.21.15.159:0/514608989 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d7e7fbba40 con 0x55d7e7f8f000 2024-03-16T09:19:07.239 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:19:06 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: starting mds.h at 2024-03-16T09:19:07.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.254+0000 7f459f7cd640 1 -- 172.21.15.112:0/1301607535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f459802d420 con 0x7f45981018d0 2024-03-16T09:19:07.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:06 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-a[32303]: 2024-03-16T09:19:06.912+0000 7f7236895700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T09:19:07.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:07 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/153674268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:19:07.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:07 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1913818231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:19:07.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:07 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3499248685' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-16T09:19:07.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:07 smithi112 ceph-mon[32307]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-16T09:19:07.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:07 smithi112 ceph-mon[32307]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T09:19:07.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:07 smithi112 ceph-mon[32307]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-03-16T09:19:07.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:07 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3499248685' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-16T09:19:07.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:07 smithi112 ceph-mon[32307]: osdmap e89: 12 total, 12 up, 12 in 2024-03-16T09:19:07.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:07 smithi112 ceph-mon[32307]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-16T09:19:07.402 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 09:19:06 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-f[67573]: ignoring --setuser ceph since I am not root 2024-03-16T09:19:07.402 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 09:19:06 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-f[67573]: ignoring --setgroup ceph since I am not root 2024-03-16T09:19:07.402 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 09:19:06 smithi194 ceph-mds[67577]: -- 172.21.15.194:0/341233496 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e5f141a40 con 0x563e5f114c00 2024-03-16T09:19:07.402 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 09:19:06 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-f[67573]: starting mds.f at 2024-03-16T09:19:07.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:07 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/153674268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-16T09:19:07.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:07 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1913818231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-16T09:19:07.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:07 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3499248685' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-16T09:19:07.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:07 smithi194 ceph-mon[36729]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-16T09:19:07.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:07 smithi194 ceph-mon[36729]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T09:19:07.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:07 smithi194 ceph-mon[36729]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2024-03-16T09:19:07.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:07 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3499248685' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-16T09:19:07.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:07 smithi194 ceph-mon[36729]: osdmap e89: 12 total, 12 up, 12 in 2024-03-16T09:19:07.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:07 smithi194 ceph-mon[36729]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-16T09:19:07.404 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:19:06 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-c[67892]: ignoring --setuser ceph since I am not root 2024-03-16T09:19:07.404 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:19:06 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-c[67892]: ignoring --setgroup ceph since I am not root 2024-03-16T09:19:07.404 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:19:06 smithi194 ceph-mds[67898]: -- 172.21.15.194:0/3386391586 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557f7fcb7a40 con 0x557f7fc8a800 2024-03-16T09:19:07.404 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:19:06 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-c[67892]: starting mds.c at 2024-03-16T09:19:07.404 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 09:19:06 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-i[68207]: ignoring --setuser ceph since I am not root 2024-03-16T09:19:07.404 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 09:19:06 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-i[68207]: ignoring --setgroup ceph since I am not root 2024-03-16T09:19:07.405 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 09:19:06 smithi194 ceph-mds[68211]: -- 172.21.15.194:0/4045890613 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dce518da40 con 0x55dce5160c00 2024-03-16T09:19:07.405 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 09:19:06 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-i[68207]: starting mds.i at 2024-03-16T09:19:07.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.919+0000 7f458e7fc640 1 -- 172.21.15.112:0/1301607535 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7f458809b3f0 con 0x7f45981018d0 2024-03-16T09:19:07.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.922+0000 7f459f7cd640 1 -- 172.21.15.112:0/1301607535 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f45840a2880 msgr2=0x7f45840a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:07.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.922+0000 7f459f7cd640 1 --2- 172.21.15.112:0/1301607535 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f45840a2880 0x7f45840a4d40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4598101730 tx=0x7f4594005e70 comp rx=0 tx=0).stop 2024-03-16T09:19:07.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.922+0000 7f459f7cd640 1 -- 172.21.15.112:0/1301607535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45981018d0 msgr2=0x7f4598197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:07.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.922+0000 7f459f7cd640 1 --2- 172.21.15.112:0/1301607535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45981018d0 0x7f4598197010 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f458800d770 tx=0x7f458800dc30 comp rx=0 tx=0).stop 2024-03-16T09:19:07.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.922+0000 7f459f7cd640 1 -- 172.21.15.112:0/1301607535 shutdown_connections 2024-03-16T09:19:07.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.922+0000 7f459f7cd640 1 --2- 172.21.15.112:0/1301607535 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f45840a2880 0x7f45840a4d40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:07.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.922+0000 7f459f7cd640 1 --2- 172.21.15.112:0/1301607535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4598102290 0x7f459819e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:07.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.922+0000 7f459f7cd640 1 --2- 172.21.15.112:0/1301607535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45981018d0 0x7f4598197010 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:07.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.922+0000 7f459f7cd640 1 --2- 172.21.15.112:0/1301607535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45981006e0 0x7f4598196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:07.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.922+0000 7f459f7cd640 1 -- 172.21.15.112:0/1301607535 >> 172.21.15.112:0/1301607535 conn(0x7f45980fbc70 msgr2=0x7f4598104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:07.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.922+0000 7f459f7cd640 1 -- 172.21.15.112:0/1301607535 shutdown_connections 2024-03-16T09:19:07.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:07.922+0000 7f459f7cd640 1 -- 172.21.15.112:0/1301607535 wait complete. 2024-03-16T09:19:07.941 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-16T09:19:08.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.052+0000 7fd63a1cd640 1 -- 172.21.15.112:0/913284441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634102a60 msgr2=0x7fd634102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:08.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.052+0000 7fd63a1cd640 1 --2- 172.21.15.112:0/913284441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634102a60 0x7fd634102ee0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd628009900 tx=0x7fd62802bc30 comp rx=0 tx=0).stop 2024-03-16T09:19:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.053+0000 7fd63a1cd640 1 -- 172.21.15.112:0/913284441 shutdown_connections 2024-03-16T09:19:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.053+0000 7fd63a1cd640 1 --2- 172.21.15.112:0/913284441 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd634103420 0x7fd634109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.053+0000 7fd63a1cd640 1 --2- 172.21.15.112:0/913284441 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634102a60 0x7fd634102ee0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.053+0000 7fd63a1cd640 1 --2- 172.21.15.112:0/913284441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd634101870 0x7fd634101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.053+0000 7fd63a1cd640 1 -- 172.21.15.112:0/913284441 >> 172.21.15.112:0/913284441 conn(0x7fd6340fce00 msgr2=0x7fd6340ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.053+0000 7fd63a1cd640 1 -- 172.21.15.112:0/913284441 shutdown_connections 2024-03-16T09:19:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.053+0000 7fd63a1cd640 1 -- 172.21.15.112:0/913284441 wait complete. 2024-03-16T09:19:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.053+0000 7fd63a1cd640 1 Processor -- start 2024-03-16T09:19:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.053+0000 7fd63a1cd640 1 -- start start 2024-03-16T09:19:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.054+0000 7fd63a1cd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd634101870 0x7fd634197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.054+0000 7fd63a1cd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd634102a60 0x7fd6341980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:08.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.054+0000 7fd63a1cd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634103420 0x7fd63419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.054+0000 7fd63a1cd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd63410d560 con 0x7fd634103420 2024-03-16T09:19:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.054+0000 7fd63a1cd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd63410d3e0 con 0x7fd634102a60 2024-03-16T09:19:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.054+0000 7fd63a1cd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd63410d6e0 con 0x7fd634101870 2024-03-16T09:19:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.054+0000 7fd632ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd634102a60 0x7fd6341980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.054+0000 7fd6337fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd634101870 0x7fd634197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.054+0000 7fd632ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd634102a60 0x7fd6341980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.112:56800/0 (socket says 172.21.15.112:56800) 2024-03-16T09:19:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.054+0000 7fd632ffd640 1 -- 172.21.15.112:0/3214928879 learned_addr learned my addr 172.21.15.112:0/3214928879 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:19:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.054+0000 7fd633fff640 1 --2- 172.21.15.112:0/3214928879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634103420 0x7fd63419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.054+0000 7fd632ffd640 1 -- 172.21.15.112:0/3214928879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd634101870 msgr2=0x7fd634197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.054+0000 7fd632ffd640 1 --2- 172.21.15.112:0/3214928879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd634101870 0x7fd634197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.054+0000 7fd632ffd640 1 -- 172.21.15.112:0/3214928879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634103420 msgr2=0x7fd63419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.054+0000 7fd632ffd640 1 --2- 172.21.15.112:0/3214928879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634103420 0x7fd63419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.054+0000 7fd632ffd640 1 -- 172.21.15.112:0/3214928879 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63419f7a0 con 0x7fd634102a60 2024-03-16T09:19:08.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.055+0000 7fd632ffd640 1 --2- 172.21.15.112:0/3214928879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd634102a60 0x7fd6341980c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd628009230 tx=0x7fd62802c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.058+0000 7fd630ff9640 1 -- 172.21.15.112:0/3214928879 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628043020 con 0x7fd634102a60 2024-03-16T09:19:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.058+0000 7fd630ff9640 1 -- 172.21.15.112:0/3214928879 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd6280043d0 con 0x7fd634102a60 2024-03-16T09:19:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.058+0000 7fd63a1cd640 1 -- 172.21.15.112:0/3214928879 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63419fa30 con 0x7fd634102a60 2024-03-16T09:19:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.058+0000 7fd630ff9640 1 -- 172.21.15.112:0/3214928879 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628003c30 con 0x7fd634102a60 2024-03-16T09:19:08.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.058+0000 7fd63a1cd640 1 -- 172.21.15.112:0/3214928879 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63419ffd0 con 0x7fd634102a60 2024-03-16T09:19:08.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.059+0000 7fd63a1cd640 1 -- 172.21.15.112:0/3214928879 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd634108b90 con 0x7fd634102a60 2024-03-16T09:19:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.060+0000 7fd630ff9640 1 -- 172.21.15.112:0/3214928879 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7fd628046020 con 0x7fd634102a60 2024-03-16T09:19:08.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.060+0000 7fd630ff9640 1 --2- 172.21.15.112:0/3214928879 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd61c0a2880 0x7fd61c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:08.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.060+0000 7fd6337fe640 1 --2- 172.21.15.112:0/3214928879 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd61c0a2880 0x7fd61c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:08.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.061+0000 7fd630ff9640 1 -- 172.21.15.112:0/3214928879 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7fd6280f11d0 con 0x7fd634102a60 2024-03-16T09:19:08.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.061+0000 7fd6337fe640 1 --2- 172.21.15.112:0/3214928879 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd61c0a2880 0x7fd61c0a4d40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd6341028c0 tx=0x7fd62400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:08.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.064+0000 7fd630ff9640 1 -- 172.21.15.112:0/3214928879 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd62804d020 con 0x7fd634102a60 2024-03-16T09:19:08.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: pgmap v457: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 716 B/s wr, 6 op/s 2024-03-16T09:19:08.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1301607535' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T09:19:08.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-16T09:19:08.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-16T09:19:08.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: Cluster is now healthy 2024-03-16T09:19:08.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/1301607535' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-16T09:19:08.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.194:6838/3598811981,v1:172.21.15.194:6839/3598811981] up:boot 2024-03-16T09:19:08.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.159:6838/2361142737,v1:172.21.15.159:6839/2361142737] up:boot 2024-03-16T09:19:08.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.194:6834/4155186573,v1:172.21.15.194:6835/4155186573] up:boot 2024-03-16T09:19:08.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.159:6834/3896323893,v1:172.21.15.159:6835/3896323893] up:boot 2024-03-16T09:19:08.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.194:6836/1907759375,v1:172.21.15.194:6837/1907759375] up:boot 2024-03-16T09:19:08.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: mds.? [v2:172.21.15.159:6836/2919140678,v1:172.21.15.159:6837/2919140678] up:boot 2024-03-16T09:19:08.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: fsmap 12 up:standby 2024-03-16T09:19:08.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T09:19:08.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T09:19:08.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T09:19:08.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T09:19:08.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T09:19:08.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T09:19:08.222 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T09:19:08.222 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:08 smithi112 ceph-mon[32307]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T09:19:08.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:08.246+0000 7fd63a1cd640 1 -- 172.21.15.112:0/3214928879 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd63402d1b0 con 0x7fd634102a60 2024-03-16T09:19:08.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: pgmap v457: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 716 B/s wr, 6 op/s 2024-03-16T09:19:08.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1301607535' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T09:19:08.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-16T09:19:08.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-16T09:19:08.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: Cluster is now healthy 2024-03-16T09:19:08.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/1301607535' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-16T09:19:08.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.194:6838/3598811981,v1:172.21.15.194:6839/3598811981] up:boot 2024-03-16T09:19:08.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.159:6838/2361142737,v1:172.21.15.159:6839/2361142737] up:boot 2024-03-16T09:19:08.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.194:6834/4155186573,v1:172.21.15.194:6835/4155186573] up:boot 2024-03-16T09:19:08.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.159:6834/3896323893,v1:172.21.15.159:6835/3896323893] up:boot 2024-03-16T09:19:08.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.194:6836/1907759375,v1:172.21.15.194:6837/1907759375] up:boot 2024-03-16T09:19:08.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: mds.? [v2:172.21.15.159:6836/2919140678,v1:172.21.15.159:6837/2919140678] up:boot 2024-03-16T09:19:08.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: fsmap 12 up:standby 2024-03-16T09:19:08.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T09:19:08.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T09:19:08.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T09:19:08.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T09:19:08.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T09:19:08.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T09:19:08.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T09:19:08.404 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:08 smithi194 ceph-mon[36729]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T09:19:08.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: pgmap v457: 97 pgs: 97 active+clean; 39 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 716 B/s wr, 6 op/s 2024-03-16T09:19:08.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1301607535' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-16T09:19:08.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-16T09:19:08.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-16T09:19:08.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: Cluster is now healthy 2024-03-16T09:19:08.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/1301607535' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-16T09:19:08.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.194:6838/3598811981,v1:172.21.15.194:6839/3598811981] up:boot 2024-03-16T09:19:08.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.159:6838/2361142737,v1:172.21.15.159:6839/2361142737] up:boot 2024-03-16T09:19:08.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.194:6834/4155186573,v1:172.21.15.194:6835/4155186573] up:boot 2024-03-16T09:19:08.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.159:6834/3896323893,v1:172.21.15.159:6835/3896323893] up:boot 2024-03-16T09:19:08.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.194:6836/1907759375,v1:172.21.15.194:6837/1907759375] up:boot 2024-03-16T09:19:08.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: mds.? [v2:172.21.15.159:6836/2919140678,v1:172.21.15.159:6837/2919140678] up:boot 2024-03-16T09:19:08.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: fsmap 12 up:standby 2024-03-16T09:19:08.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-16T09:19:08.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-16T09:19:08.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-16T09:19:08.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T09:19:08.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T09:19:08.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-16T09:19:08.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-16T09:19:08.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:08 smithi159 ceph-mon[36739]: from='mgr.14152 172.21.15.112:0/939375413' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-16T09:19:09.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.064+0000 7fd630ff9640 1 -- 172.21.15.112:0/3214928879 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) ==== 181+0+0 (secure 0 0 0) 0x7fd6280be180 con 0x7fd634102a60 2024-03-16T09:19:09.067 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2024-03-16T09:19:09.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.067+0000 7fd63a1cd640 1 -- 172.21.15.112:0/3214928879 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd61c0a2880 msgr2=0x7fd61c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:09.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.067+0000 7fd63a1cd640 1 --2- 172.21.15.112:0/3214928879 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd61c0a2880 0x7fd61c0a4d40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd6341028c0 tx=0x7fd62400b390 comp rx=0 tx=0).stop 2024-03-16T09:19:09.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.067+0000 7fd63a1cd640 1 -- 172.21.15.112:0/3214928879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd634102a60 msgr2=0x7fd6341980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:09.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.067+0000 7fd63a1cd640 1 --2- 172.21.15.112:0/3214928879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd634102a60 0x7fd6341980c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd628009230 tx=0x7fd62802c820 comp rx=0 tx=0).stop 2024-03-16T09:19:09.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.067+0000 7fd63a1cd640 1 -- 172.21.15.112:0/3214928879 shutdown_connections 2024-03-16T09:19:09.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.067+0000 7fd63a1cd640 1 --2- 172.21.15.112:0/3214928879 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7fd61c0a2880 0x7fd61c0a4d40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:09.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.067+0000 7fd63a1cd640 1 --2- 172.21.15.112:0/3214928879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634103420 0x7fd63419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:09.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.067+0000 7fd63a1cd640 1 --2- 172.21.15.112:0/3214928879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd634102a60 0x7fd6341980c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:09.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.067+0000 7fd63a1cd640 1 --2- 172.21.15.112:0/3214928879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd634101870 0x7fd634197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:09.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.067+0000 7fd63a1cd640 1 -- 172.21.15.112:0/3214928879 >> 172.21.15.112:0/3214928879 conn(0x7fd6340fce00 msgr2=0x7fd634105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:09.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.067+0000 7fd63a1cd640 1 -- 172.21.15.112:0/3214928879 shutdown_connections 2024-03-16T09:19:09.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.067+0000 7fd63a1cd640 1 -- 172.21.15.112:0/3214928879 wait complete. 2024-03-16T09:19:09.081 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-16T09:19:09.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.191+0000 7f65d071b640 1 -- 172.21.15.112:0/2571579368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c8101cc0 msgr2=0x7f65c81020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:09.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.191+0000 7f65d071b640 1 --2- 172.21.15.112:0/2571579368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c8101cc0 0x7f65c81020e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009900 tx=0x7f65c402bc30 comp rx=0 tx=0).stop 2024-03-16T09:19:09.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.191+0000 7f65d071b640 1 -- 172.21.15.112:0/2571579368 shutdown_connections 2024-03-16T09:19:09.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.191+0000 7f65d071b640 1 --2- 172.21.15.112:0/2571579368 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65c81012b0 0x7f65c8108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:09.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.191+0000 7f65d071b640 1 --2- 172.21.15.112:0/2571579368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65c8100910 0x7f65c8100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:09.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.191+0000 7f65d071b640 1 --2- 172.21.15.112:0/2571579368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c8101cc0 0x7f65c81020e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:09.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.191+0000 7f65d071b640 1 -- 172.21.15.112:0/2571579368 >> 172.21.15.112:0/2571579368 conn(0x7f65c80fb3e0 msgr2=0x7f65c80fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:09.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.191+0000 7f65d071b640 1 -- 172.21.15.112:0/2571579368 shutdown_connections 2024-03-16T09:19:09.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.191+0000 7f65d071b640 1 -- 172.21.15.112:0/2571579368 wait complete. 2024-03-16T09:19:09.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.191+0000 7f65d071b640 1 Processor -- start 2024-03-16T09:19:09.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65d071b640 1 -- start start 2024-03-16T09:19:09.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65d071b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c8100910 0x7f65c8100370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:09.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65d071b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65c81012b0 0x7f65c819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:09.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65d071b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65c8101cc0 0x7f65c819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:09.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65d071b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65c810c4c0 con 0x7f65c8100910 2024-03-16T09:19:09.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65d071b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f65c810c340 con 0x7f65c81012b0 2024-03-16T09:19:09.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65d071b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f65c810c640 con 0x7f65c8101cc0 2024-03-16T09:19:09.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65ce4b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c8100910 0x7f65c8100370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:09.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65ce4b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c8100910 0x7f65c8100370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33026/0 (socket says 172.21.15.112:33026) 2024-03-16T09:19:09.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65ce4b7640 1 -- 172.21.15.112:0/2950874432 learned_addr learned my addr 172.21.15.112:0/2950874432 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-03-16T09:19:09.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65cdcb6640 1 --2- 172.21.15.112:0/2950874432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65c81012b0 0x7f65c819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:09.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65cecb8640 1 --2- 172.21.15.112:0/2950874432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65c8101cc0 0x7f65c819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:09.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65ce4b7640 1 -- 172.21.15.112:0/2950874432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65c8101cc0 msgr2=0x7f65c819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:09.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65ce4b7640 1 --2- 172.21.15.112:0/2950874432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65c8101cc0 0x7f65c819efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:09.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65ce4b7640 1 -- 172.21.15.112:0/2950874432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65c81012b0 msgr2=0x7f65c819c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:09.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65ce4b7640 1 --2- 172.21.15.112:0/2950874432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65c81012b0 0x7f65c819c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:09.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65ce4b7640 1 -- 172.21.15.112:0/2950874432 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c819f8b0 con 0x7f65c8100910 2024-03-16T09:19:09.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65ce4b7640 1 --2- 172.21.15.112:0/2950874432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c8100910 0x7f65c8100370 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009230 tx=0x7f65c402c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:09.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65bf7fe640 1 -- 172.21.15.112:0/2950874432 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c4043020 con 0x7f65c8100910 2024-03-16T09:19:09.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65d071b640 1 -- 172.21.15.112:0/2950874432 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c819fae0 con 0x7f65c8100910 2024-03-16T09:19:09.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.192+0000 7f65bf7fe640 1 -- 172.21.15.112:0/2950874432 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f65c40043d0 con 0x7f65c8100910 2024-03-16T09:19:09.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.193+0000 7f65bf7fe640 1 -- 172.21.15.112:0/2950874432 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c4003c30 con 0x7f65c8100910 2024-03-16T09:19:09.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.193+0000 7f65d071b640 1 -- 172.21.15.112:0/2950874432 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c81a0080 con 0x7f65c8100910 2024-03-16T09:19:09.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.193+0000 7f65d071b640 1 -- 172.21.15.112:0/2950874432 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c8107a50 con 0x7f65c8100910 2024-03-16T09:19:09.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.194+0000 7f65bf7fe640 1 -- 172.21.15.112:0/2950874432 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 20) ==== 137291+0+0 (secure 0 0 0) 0x7f65c4046020 con 0x7f65c8100910 2024-03-16T09:19:09.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.195+0000 7f65bf7fe640 1 --2- 172.21.15.112:0/2950874432 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f65b40a2800 0x7f65b40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T09:19:09.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.195+0000 7f65cdcb6640 1 --2- 172.21.15.112:0/2950874432 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f65b40a2800 0x7f65b40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T09:19:09.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.195+0000 7f65bf7fe640 1 -- 172.21.15.112:0/2950874432 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7915+0+0 (secure 0 0 0) 0x7f65c40f0be0 con 0x7f65c8100910 2024-03-16T09:19:09.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.195+0000 7f65cdcb6640 1 --2- 172.21.15.112:0/2950874432 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f65b40a2800 0x7f65b40a4cc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f65c81000e0 tx=0x7f65b800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T09:19:09.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.199+0000 7f65bf7fe640 1 -- 172.21.15.112:0/2950874432 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f65c404d020 con 0x7f65c8100910 2024-03-16T09:19:09.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:09 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/3214928879' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T09:19:09.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:09 smithi112 ceph-mon[32307]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T09:19:09.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:09.376+0000 7f65d071b640 1 -- 172.21.15.112:0/2950874432 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f65c80623e0 con 0x7f65c8100910 2024-03-16T09:19:09.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:09 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/3214928879' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T09:19:09.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:09 smithi194 ceph-mon[36729]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T09:19:09.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:09 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/3214928879' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T09:19:09.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:09 smithi159 ceph-mon[36739]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T09:19:10.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:10.065+0000 7f65bf7fe640 1 -- 172.21.15.112:0/2950874432 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) ==== 169+0+0 (secure 0 0 0) 0x7f65c40bddf0 con 0x7f65c8100910 2024-03-16T09:19:10.069 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2024-03-16T09:19:10.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:10.069+0000 7f65d071b640 1 -- 172.21.15.112:0/2950874432 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f65b40a2800 msgr2=0x7f65b40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:10.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:10.069+0000 7f65d071b640 1 --2- 172.21.15.112:0/2950874432 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f65b40a2800 0x7f65b40a4cc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f65c81000e0 tx=0x7f65b800b390 comp rx=0 tx=0).stop 2024-03-16T09:19:10.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:10.069+0000 7f65d071b640 1 -- 172.21.15.112:0/2950874432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c8100910 msgr2=0x7f65c8100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T09:19:10.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:10.069+0000 7f65d071b640 1 --2- 172.21.15.112:0/2950874432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c8100910 0x7f65c8100370 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009230 tx=0x7f65c402c820 comp rx=0 tx=0).stop 2024-03-16T09:19:10.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:10.069+0000 7f65d071b640 1 -- 172.21.15.112:0/2950874432 shutdown_connections 2024-03-16T09:19:10.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:10.069+0000 7f65d071b640 1 --2- 172.21.15.112:0/2950874432 >> [v2:172.21.15.112:6800/2628870229,v1:172.21.15.112:6801/2628870229] conn(0x7f65b40a2800 0x7f65b40a4cc0 secure :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f65c81000e0 tx=0x7f65b800b390 comp rx=0 tx=0).stop 2024-03-16T09:19:10.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:10.069+0000 7f65d071b640 1 --2- 172.21.15.112:0/2950874432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65c8101cc0 0x7f65c819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:10.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:10.069+0000 7f65d071b640 1 --2- 172.21.15.112:0/2950874432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65c81012b0 0x7f65c819c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T09:19:10.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:10.069+0000 7f65d071b640 1 --2- 172.21.15.112:0/2950874432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65c8100910 0x7f65c8100370 secure :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009230 tx=0x7f65c402c820 comp rx=0 tx=0).stop 2024-03-16T09:19:10.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:10.069+0000 7f65d071b640 1 -- 172.21.15.112:0/2950874432 >> 172.21.15.112:0/2950874432 conn(0x7f65c80fb3e0 msgr2=0x7f65c8104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T09:19:10.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:10.070+0000 7f65d071b640 1 -- 172.21.15.112:0/2950874432 shutdown_connections 2024-03-16T09:19:10.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-03-16T09:19:10.070+0000 7f65d071b640 1 -- 172.21.15.112:0/2950874432 wait complete. 2024-03-16T09:19:10.089 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T09:19:10.119 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T09:19:10.152 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T09:19:10.186 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-16T09:19:10.186 DEBUG:teuthology.orchestra.run.smithi112:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-16T09:19:10.201 DEBUG:teuthology.orchestra.run.smithi159:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-16T09:19:10.217 DEBUG:teuthology.orchestra.run.smithi194:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-16T09:19:10.243 INFO:tasks.cephadm:Stopping all daemons... 2024-03-16T09:19:10.243 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-16T09:19:10.244 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@mon.a 2024-03-16T09:19:10.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:10 smithi112 ceph-mon[32307]: pgmap v458: 97 pgs: 97 active+clean; 39 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 7 op/s 2024-03-16T09:19:10.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:10 smithi112 ceph-mon[32307]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-16T09:19:10.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:10 smithi112 ceph-mon[32307]: osdmap e90: 12 total, 12 up, 12 in 2024-03-16T09:19:10.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:10 smithi112 ceph-mon[32307]: from='client.? 172.21.15.112:0/2950874432' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T09:19:10.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:10 smithi112 systemd[1]: Stopping Ceph mon.a for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:19:10.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:10 smithi194 ceph-mon[36729]: pgmap v458: 97 pgs: 97 active+clean; 39 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 7 op/s 2024-03-16T09:19:10.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:10 smithi194 ceph-mon[36729]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-16T09:19:10.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:10 smithi194 ceph-mon[36729]: osdmap e90: 12 total, 12 up, 12 in 2024-03-16T09:19:10.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:10 smithi194 ceph-mon[36729]: from='client.? 172.21.15.112:0/2950874432' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T09:19:10.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:10 smithi159 ceph-mon[36739]: pgmap v458: 97 pgs: 97 active+clean; 39 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 7 op/s 2024-03-16T09:19:10.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:10 smithi159 ceph-mon[36739]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-16T09:19:10.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:10 smithi159 ceph-mon[36739]: osdmap e90: 12 total, 12 up, 12 in 2024-03-16T09:19:10.487 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:10 smithi159 ceph-mon[36739]: from='client.? 172.21.15.112:0/2950874432' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-16T09:19:10.843 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:10 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-a[32303]: 2024-03-16T09:19:10.552+0000 7f723c0a0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:19:10.843 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:10 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-a[32303]: 2024-03-16T09:19:10.552+0000 7f723c0a0700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-16T09:19:10.843 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:10 smithi112 podman[90255]: 2024-03-16 09:19:10.636453135 +0000 UTC m=+0.210079902 container died c7d8c6135c176c34c9778c50ff86ef9507cad5b05156eb2e6c3f4aef93534801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-16T09:19:11.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:11 smithi112 podman[90255]: 2024-03-16 09:19:11.163927477 +0000 UTC m=+0.737554286 container cleanup c7d8c6135c176c34c9778c50ff86ef9507cad5b05156eb2e6c3f4aef93534801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-16T09:19:11.342 INFO:journalctl@ceph.mon.a.smithi112.stdout:Mar 16 09:19:11 smithi112 bash[90255]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-a 2024-03-16T09:19:11.576 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mon.a.service' 2024-03-16T09:19:11.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:19:11.649 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-16T09:19:11.650 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-16T09:19:11.650 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@mon.b 2024-03-16T09:19:11.964 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:11 smithi159 systemd[1]: Stopping Ceph mon.b for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:19:12.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:11 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-b[36734]: 2024-03-16T09:19:11.961+0000 7f0c634fc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:19:12.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:11 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-b[36734]: 2024-03-16T09:19:11.961+0000 7f0c634fc700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-16T09:19:12.237 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:12 smithi159 podman[69523]: 2024-03-16 09:19:12.139030999 +0000 UTC m=+0.291319523 container died 0bfb0dba10badc8831570b04a8f0138804bb26a05aa20369b7ce5853e313d74b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-16T09:19:12.874 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:12 smithi159 podman[69523]: 2024-03-16 09:19:12.621499804 +0000 UTC m=+0.773788337 container cleanup 0bfb0dba10badc8831570b04a8f0138804bb26a05aa20369b7ce5853e313d74b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-b, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:19:12.874 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 16 09:19:12 smithi159 bash[69523]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-b 2024-03-16T09:19:13.037 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mon.b.service' 2024-03-16T09:19:13.091 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:19:13.092 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-16T09:19:13.092 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-16T09:19:13.092 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@mon.c 2024-03-16T09:19:13.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:13 smithi194 systemd[1]: Stopping Ceph mon.c for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:19:13.828 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:13 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-c[36723]: 2024-03-16T09:19:13.416+0000 7fbe4f98b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:19:13.828 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:13 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-c[36723]: 2024-03-16T09:19:13.416+0000 7fbe4f98b700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-16T09:19:13.828 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:13 smithi194 podman[71384]: 2024-03-16 09:19:13.51895513 +0000 UTC m=+0.233733200 container died e32ced22c7fb3f081990414768cb7a03dc34841b1386b5e7ef7eff5ee39b65b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-16T09:19:14.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:13 smithi194 podman[71384]: 2024-03-16 09:19:13.934094795 +0000 UTC m=+0.648872884 container cleanup e32ced22c7fb3f081990414768cb7a03dc34841b1386b5e7ef7eff5ee39b65b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:19:14.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:13 smithi194 bash[71384]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-c 2024-03-16T09:19:14.402 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mon.c.service' 2024-03-16T09:19:14.426 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:14 smithi194 podman[71399]: 2024-03-16 09:19:14.19428397 +0000 UTC m=+0.671192489 container remove e32ced22c7fb3f081990414768cb7a03dc34841b1386b5e7ef7eff5ee39b65b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mon-c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-16T09:19:14.427 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:14 smithi194 systemd[1]: ceph-273be2a4-e374-11ee-95c9-87774f69a715@mon.c.service: Deactivated successfully. 2024-03-16T09:19:14.427 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:14 smithi194 systemd[1]: Stopped Ceph mon.c for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:19:14.427 INFO:journalctl@ceph.mon.c.smithi194.stdout:Mar 16 09:19:14 smithi194 systemd[1]: ceph-273be2a4-e374-11ee-95c9-87774f69a715@mon.c.service: Consumed 8.271s CPU time. 2024-03-16T09:19:14.463 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:19:14.463 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-16T09:19:14.464 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-16T09:19:14.464 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@mgr.x 2024-03-16T09:19:14.779 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:19:14 smithi112 systemd[1]: Stopping Ceph mgr.x for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:19:15.092 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:19:14 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x[32541]: 2024-03-16T09:19:14.776+0000 7f501d833700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:19:15.093 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:19:14 smithi112 podman[90378]: 2024-03-16 09:19:14.808521647 +0000 UTC m=+0.155300475 container died f6fd1d82bafdeaf2bbfa913aee3cf7cc7b944a6e83373aa97b34c3af951b8d8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux ) 2024-03-16T09:19:15.456 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:19:15 smithi112 podman[90378]: 2024-03-16 09:19:15.329061939 +0000 UTC m=+0.675840764 container cleanup f6fd1d82bafdeaf2bbfa913aee3cf7cc7b944a6e83373aa97b34c3af951b8d8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-16T09:19:15.456 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:19:15 smithi112 bash[90378]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x 2024-03-16T09:19:15.842 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Mar 16 09:19:15 smithi112 podman[90392]: 2024-03-16 09:19:15.650343685 +0000 UTC m=+0.842024038 container remove f6fd1d82bafdeaf2bbfa913aee3cf7cc7b944a6e83373aa97b34c3af951b8d8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-x, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-16T09:19:15.890 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mgr.x.service' 2024-03-16T09:19:15.928 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:19:15.929 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-16T09:19:15.929 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-16T09:19:15.929 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@mgr.y 2024-03-16T09:19:16.236 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:19:15 smithi159 systemd[1]: Stopping Ceph mgr.y for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:19:16.596 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:19:16 smithi159 podman[69648]: 2024-03-16 09:19:16.280609407 +0000 UTC m=+0.161029185 container died c35bb3f40b25447ab792b2cf84cc19f8cb295b8de82c8b750aede2d411a36cc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-16T09:19:16.931 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:19:16 smithi159 podman[69648]: 2024-03-16 09:19:16.693869407 +0000 UTC m=+0.574289175 container cleanup c35bb3f40b25447ab792b2cf84cc19f8cb295b8de82c8b750aede2d411a36cc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-16T09:19:16.931 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 16 09:19:16 smithi159 bash[69648]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-y 2024-03-16T09:19:17.068 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mgr.y.service' 2024-03-16T09:19:17.106 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:19:17.106 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-16T09:19:17.106 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-16T09:19:17.106 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@mgr.z 2024-03-16T09:19:17.402 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:19:17 smithi194 systemd[1]: Stopping Ceph mgr.z for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:19:17.733 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:19:17 smithi194 podman[71517]: 2024-03-16 09:19:17.426364298 +0000 UTC m=+0.139718118 container died d6c887a04c5502d37d0f5f1842b75a6f327993eb7d4681df0b1a110023961ebd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:19:18.117 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:19:17 smithi194 podman[71517]: 2024-03-16 09:19:17.83969745 +0000 UTC m=+0.553051275 container cleanup d6c887a04c5502d37d0f5f1842b75a6f327993eb7d4681df0b1a110023961ebd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5) 2024-03-16T09:19:18.117 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:19:17 smithi194 bash[71517]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z 2024-03-16T09:19:18.119 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Mar 16 09:19:18 smithi194 podman[71531]: 2024-03-16 09:19:18.116135931 +0000 UTC m=+0.687878581 container remove d6c887a04c5502d37d0f5f1842b75a6f327993eb7d4681df0b1a110023961ebd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mgr-z, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:19:18.256 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mgr.z.service' 2024-03-16T09:19:18.295 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:19:18.295 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-16T09:19:18.296 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-16T09:19:18.296 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.0 2024-03-16T09:19:18.592 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:19:18 smithi112 systemd[1]: Stopping Ceph osd.0 for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:19:19.092 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:19:18 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-0[47070]: 2024-03-16T09:19:18.608+0000 7f9067afe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:19:19.092 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:19:18 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-0[47070]: 2024-03-16T09:19:18.608+0000 7f9067afe700 -1 osd.0 91 *** Got signal Terminated *** 2024-03-16T09:19:19.092 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:19:18 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-0[47070]: 2024-03-16T09:19:18.608+0000 7f9067afe700 -1 osd.0 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:19:24.445 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:19:24 smithi112 podman[90500]: 2024-03-16 09:19:24.141471265 +0000 UTC m=+5.660572059 container died d7067641f71fef05f3da6513d67bd426e248e8d751e805b2a21fe2d911acd74a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-16T09:19:24.842 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:19:24 smithi112 podman[90500]: 2024-03-16 09:19:24.643122351 +0000 UTC m=+6.162223188 container cleanup d7067641f71fef05f3da6513d67bd426e248e8d751e805b2a21fe2d911acd74a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:19:24.842 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:19:24 smithi112 bash[90500]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-0 2024-03-16T09:19:25.271 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:19:24 smithi112 podman[90517]: 2024-03-16 09:19:24.844959445 +0000 UTC m=+0.697384098 container remove d7067641f71fef05f3da6513d67bd426e248e8d751e805b2a21fe2d911acd74a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-16T09:19:25.552 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:19:25 smithi112 podman[90611]: 2024-03-16 09:19:25.172253397 +0000 UTC m=+0.018189294 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:19:25.552 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:19:25 smithi112 podman[90611]: 2024-03-16 09:19:25.313231892 +0000 UTC m=+0.159167822 container create 7003b713c2df56b834638c024948d2e239d0b74faa57a8a56af144f15fcd0135 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=) 2024-03-16T09:19:25.842 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:19:25 smithi112 podman[90611]: 2024-03-16 09:19:25.550164286 +0000 UTC m=+0.396100198 container init 7003b713c2df56b834638c024948d2e239d0b74faa57a8a56af144f15fcd0135 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-16T09:19:25.842 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:19:25 smithi112 podman[90611]: 2024-03-16 09:19:25.553838512 +0000 UTC m=+0.399774404 container start 7003b713c2df56b834638c024948d2e239d0b74faa57a8a56af144f15fcd0135 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:19:25.843 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:19:25 smithi112 podman[90611]: 2024-03-16 09:19:25.649599142 +0000 UTC m=+0.495535045 container attach 7003b713c2df56b834638c024948d2e239d0b74faa57a8a56af144f15fcd0135 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-0-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-16T09:19:26.212 INFO:journalctl@ceph.osd.0.smithi112.stdout:Mar 16 09:19:25 smithi112 podman[90611]: 2024-03-16 09:19:25.845844455 +0000 UTC m=+0.691780344 container died 7003b713c2df56b834638c024948d2e239d0b74faa57a8a56af144f15fcd0135 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-16T09:19:26.597 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.0.service' 2024-03-16T09:19:26.641 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:19:26.641 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-16T09:19:26.641 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-16T09:19:26.642 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.1 2024-03-16T09:19:26.958 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:19:26 smithi159 systemd[1]: Stopping Ceph osd.1 for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:19:27.237 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:19:26 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-1[41333]: 2024-03-16T09:19:26.955+0000 7fdf205dd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:19:27.237 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:19:26 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-1[41333]: 2024-03-16T09:19:26.955+0000 7fdf205dd700 -1 osd.1 91 *** Got signal Terminated *** 2024-03-16T09:19:27.237 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:19:26 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-1[41333]: 2024-03-16T09:19:26.955+0000 7fdf205dd700 -1 osd.1 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:19:32.626 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:19:32 smithi159 podman[69780]: 2024-03-16 09:19:32.325151555 +0000 UTC m=+5.491331221 container died 19f89b4b51d2e433e088ec07a88fdb7b8a17f857cc872bbfcc061b523ef876ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-1, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-16T09:19:32.935 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:19:32 smithi159 podman[69780]: 2024-03-16 09:19:32.732362619 +0000 UTC m=+5.898542313 container cleanup 19f89b4b51d2e433e088ec07a88fdb7b8a17f857cc872bbfcc061b523ef876ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-16T09:19:32.935 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:19:32 smithi159 bash[69780]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-1 2024-03-16T09:19:33.236 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:19:32 smithi159 podman[69800]: 2024-03-16 09:19:32.93364848 +0000 UTC m=+0.604902191 container remove 19f89b4b51d2e433e088ec07a88fdb7b8a17f857cc872bbfcc061b523ef876ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-1, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD) 2024-03-16T09:19:33.621 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:19:33 smithi159 podman[69892]: 2024-03-16 09:19:33.24392387 +0000 UTC m=+0.017538991 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:19:33.622 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:19:33 smithi159 podman[69892]: 2024-03-16 09:19:33.376198437 +0000 UTC m=+0.149813575 container create ab13788822a111efc80a6d68752ebc70b13b53344762eafe5bf23b00ac6d4ab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-16T09:19:33.915 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:19:33 smithi159 podman[69892]: 2024-03-16 09:19:33.62041999 +0000 UTC m=+0.394035116 container init ab13788822a111efc80a6d68752ebc70b13b53344762eafe5bf23b00ac6d4ab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:19:33.915 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:19:33 smithi159 podman[69892]: 2024-03-16 09:19:33.62363053 +0000 UTC m=+0.397245651 container start ab13788822a111efc80a6d68752ebc70b13b53344762eafe5bf23b00ac6d4ab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-1-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-16T09:19:33.915 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:19:33 smithi159 podman[69892]: 2024-03-16 09:19:33.720575617 +0000 UTC m=+0.494190753 container attach ab13788822a111efc80a6d68752ebc70b13b53344762eafe5bf23b00ac6d4ab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-1-deactivate, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-16T09:19:34.236 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 16 09:19:33 smithi159 podman[69892]: 2024-03-16 09:19:33.91335884 +0000 UTC m=+0.686973962 container died ab13788822a111efc80a6d68752ebc70b13b53344762eafe5bf23b00ac6d4ab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-16T09:19:34.606 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.1.service' 2024-03-16T09:19:34.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:19:34.649 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-16T09:19:34.650 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-16T09:19:34.650 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.2 2024-03-16T09:19:34.962 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:19:34 smithi194 systemd[1]: Stopping Ceph osd.2 for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:19:35.403 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:19:34 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-2[41896]: 2024-03-16T09:19:34.959+0000 7f39ffed0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:19:35.404 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:19:34 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-2[41896]: 2024-03-16T09:19:34.959+0000 7f39ffed0700 -1 osd.2 91 *** Got signal Terminated *** 2024-03-16T09:19:35.404 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:19:34 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-2[41896]: 2024-03-16T09:19:34.959+0000 7f39ffed0700 -1 osd.2 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:19:40.738 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:19:40 smithi194 podman[71640]: 2024-03-16 09:19:40.42599174 +0000 UTC m=+5.580791518 container died 5acab86b1c5d64e9a5a19a511d1d43966443b2a62ec7488623bd03798875bbfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:19:41.029 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:19:40 smithi194 podman[71640]: 2024-03-16 09:19:40.835014861 +0000 UTC m=+5.989814631 container cleanup 5acab86b1c5d64e9a5a19a511d1d43966443b2a62ec7488623bd03798875bbfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-16T09:19:41.030 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:19:40 smithi194 bash[71640]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-2 2024-03-16T09:19:41.402 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:19:41 smithi194 podman[71657]: 2024-03-16 09:19:41.028074177 +0000 UTC m=+0.597191536 container remove 5acab86b1c5d64e9a5a19a511d1d43966443b2a62ec7488623bd03798875bbfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:19:41.717 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:19:41 smithi194 podman[71747]: 2024-03-16 09:19:41.34500461 +0000 UTC m=+0.018426500 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:19:41.718 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:19:41 smithi194 podman[71747]: 2024-03-16 09:19:41.470665536 +0000 UTC m=+0.144087420 container create c0440b939a42c2a927b8bd91a60b707a2a0916d83aaa6dc59afbc46741e5a21d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-16T09:19:42.014 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:19:41 smithi194 podman[71747]: 2024-03-16 09:19:41.71569765 +0000 UTC m=+0.389119544 container init c0440b939a42c2a927b8bd91a60b707a2a0916d83aaa6dc59afbc46741e5a21d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:19:42.014 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:19:41 smithi194 podman[71747]: 2024-03-16 09:19:41.719062596 +0000 UTC m=+0.392484476 container start c0440b939a42c2a927b8bd91a60b707a2a0916d83aaa6dc59afbc46741e5a21d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-16T09:19:42.014 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:19:41 smithi194 podman[71747]: 2024-03-16 09:19:41.807027801 +0000 UTC m=+0.480449693 container attach c0440b939a42c2a927b8bd91a60b707a2a0916d83aaa6dc59afbc46741e5a21d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-2-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-16T09:19:42.276 INFO:journalctl@ceph.osd.2.smithi194.stdout:Mar 16 09:19:42 smithi194 podman[71747]: 2024-03-16 09:19:42.012766432 +0000 UTC m=+0.686188313 container died c0440b939a42c2a927b8bd91a60b707a2a0916d83aaa6dc59afbc46741e5a21d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-03-16T09:19:42.653 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.2.service' 2024-03-16T09:19:42.695 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:19:42.695 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-16T09:19:42.695 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-16T09:19:42.695 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.3 2024-03-16T09:19:43.001 INFO:journalctl@ceph.osd.3.smithi112.stdout:Mar 16 09:19:42 smithi112 systemd[1]: Stopping Ceph osd.3 for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:19:43.342 INFO:journalctl@ceph.osd.3.smithi112.stdout:Mar 16 09:19:42 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-3[53870]: 2024-03-16T09:19:42.997+0000 7f1095ce9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:19:43.342 INFO:journalctl@ceph.osd.3.smithi112.stdout:Mar 16 09:19:42 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-3[53870]: 2024-03-16T09:19:42.997+0000 7f1095ce9700 -1 osd.3 91 *** Got signal Terminated *** 2024-03-16T09:19:43.342 INFO:journalctl@ceph.osd.3.smithi112.stdout:Mar 16 09:19:42 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-3[53870]: 2024-03-16T09:19:42.997+0000 7f1095ce9700 -1 osd.3 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:19:45.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:44 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:44.701+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:19:45.487 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:45 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4[48007]: 2024-03-16T09:19:45.023+0000 7fb231cde700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.383552+0000 front 2024-03-16T09:19:21.383602+0000 (oldest deadline 2024-03-16T09:19:44.283468+0000) 2024-03-16T09:19:46.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:45 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:45.718+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:19:46.236 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:46 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4[48007]: 2024-03-16T09:19:46.065+0000 7fb231cde700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.383552+0000 front 2024-03-16T09:19:21.383602+0000 (oldest deadline 2024-03-16T09:19:44.283468+0000) 2024-03-16T09:19:46.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:45 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:45.872+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:19:47.092 INFO:journalctl@ceph.osd.3.smithi112.stdout:Mar 16 09:19:46 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-3[53870]: 2024-03-16T09:19:46.721+0000 7f10922dd700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:23.101655+0000 front 2024-03-16T09:19:23.101611+0000 (oldest deadline 2024-03-16T09:19:46.601766+0000) 2024-03-16T09:19:47.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:46 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:46.737+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:19:47.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:46 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:46.962+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:19:47.237 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:47 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4[48007]: 2024-03-16T09:19:47.056+0000 7fb231cde700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.383552+0000 front 2024-03-16T09:19:21.383602+0000 (oldest deadline 2024-03-16T09:19:44.283468+0000) 2024-03-16T09:19:47.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:46 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:46.844+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:19:47.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:19:47 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:19:47.519+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:19:48.092 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:47 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:47.829+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:19:48.092 INFO:journalctl@ceph.osd.3.smithi112.stdout:Mar 16 09:19:47 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-3[53870]: 2024-03-16T09:19:47.760+0000 7f10922dd700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:23.101655+0000 front 2024-03-16T09:19:23.101611+0000 (oldest deadline 2024-03-16T09:19:46.601766+0000) 2024-03-16T09:19:48.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:47 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:47.704+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:19:48.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:47 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:47.942+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:19:48.487 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:48 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4[48007]: 2024-03-16T09:19:48.089+0000 7fb231cde700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.383552+0000 front 2024-03-16T09:19:21.383602+0000 (oldest deadline 2024-03-16T09:19:44.283468+0000) 2024-03-16T09:19:48.785 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:48 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:48.705+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:19:48.785 INFO:journalctl@ceph.osd.3.smithi112.stdout:Mar 16 09:19:48 smithi112 podman[90727]: 2024-03-16 09:19:48.504968197 +0000 UTC m=+5.617923366 container died 59c37f4a81d05232adc5156166dafaed8fa4431c945090110ba5e14b7f64c688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-3, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-16T09:19:48.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:19:48 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:19:48.471+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:19:48.902 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:48 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:19:48.719+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.297503+0000 front 2024-03-16T09:19:22.297449+0000 (oldest deadline 2024-03-16T09:19:48.197559+0000) 2024-03-16T09:19:49.092 INFO:journalctl@ceph.osd.3.smithi112.stdout:Mar 16 09:19:48 smithi112 podman[90727]: 2024-03-16 09:19:48.990887714 +0000 UTC m=+6.103842937 container cleanup 59c37f4a81d05232adc5156166dafaed8fa4431c945090110ba5e14b7f64c688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:19:49.093 INFO:journalctl@ceph.osd.3.smithi112.stdout:Mar 16 09:19:48 smithi112 bash[90727]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-3 2024-03-16T09:19:49.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:48 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:48.912+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:19:49.128 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:48 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:48.821+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:19:49.486 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:49 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4[48007]: 2024-03-16T09:19:49.126+0000 7fb231cde700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.383552+0000 front 2024-03-16T09:19:21.383602+0000 (oldest deadline 2024-03-16T09:19:44.283468+0000) 2024-03-16T09:19:49.661 INFO:journalctl@ceph.osd.3.smithi112.stdout:Mar 16 09:19:49 smithi112 podman[90744]: 2024-03-16 09:19:49.385038533 +0000 UTC m=+0.874410351 container remove 59c37f4a81d05232adc5156166dafaed8fa4431c945090110ba5e14b7f64c688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-3, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-16T09:19:49.727 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:19:49 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:19:49.434+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:19:50.024 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:49 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:49.658+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:19:50.024 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:49 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:49.863+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:19:50.024 INFO:journalctl@ceph.osd.3.smithi112.stdout:Mar 16 09:19:49 smithi112 podman[90838]: 2024-03-16 09:19:49.66101221 +0000 UTC m=+0.017624426 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:19:50.024 INFO:journalctl@ceph.osd.3.smithi112.stdout:Mar 16 09:19:49 smithi112 podman[90838]: 2024-03-16 09:19:49.786035243 +0000 UTC m=+0.142647436 container create 3518d55b4f9d480ff33865d9f2ab6b285aaf9ec05c41b7a63bf8d74236b6ba9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-16T09:19:50.144 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:49 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:49.849+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:19:50.152 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:49 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:19:49.724+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.297503+0000 front 2024-03-16T09:19:22.297449+0000 (oldest deadline 2024-03-16T09:19:48.197559+0000) 2024-03-16T09:19:50.318 INFO:journalctl@ceph.osd.3.smithi112.stdout:Mar 16 09:19:50 smithi112 podman[90838]: 2024-03-16 09:19:50.022298946 +0000 UTC m=+0.378911179 container init 3518d55b4f9d480ff33865d9f2ab6b285aaf9ec05c41b7a63bf8d74236b6ba9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-16T09:19:50.318 INFO:journalctl@ceph.osd.3.smithi112.stdout:Mar 16 09:19:50 smithi112 podman[90838]: 2024-03-16 09:19:50.025781056 +0000 UTC m=+0.382393259 container start 3518d55b4f9d480ff33865d9f2ab6b285aaf9ec05c41b7a63bf8d74236b6ba9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:19:50.318 INFO:journalctl@ceph.osd.3.smithi112.stdout:Mar 16 09:19:50 smithi112 podman[90838]: 2024-03-16 09:19:50.130370186 +0000 UTC m=+0.486982389 container attach 3518d55b4f9d480ff33865d9f2ab6b285aaf9ec05c41b7a63bf8d74236b6ba9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-16T09:19:50.486 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:50 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4[48007]: 2024-03-16T09:19:50.141+0000 7fb231cde700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.383552+0000 front 2024-03-16T09:19:21.383602+0000 (oldest deadline 2024-03-16T09:19:44.283468+0000) 2024-03-16T09:19:50.590 INFO:journalctl@ceph.osd.3.smithi112.stdout:Mar 16 09:19:50 smithi112 podman[90838]: 2024-03-16 09:19:50.316760457 +0000 UTC m=+0.673372663 container died 3518d55b4f9d480ff33865d9f2ab6b285aaf9ec05c41b7a63bf8d74236b6ba9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-16T09:19:50.768 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:19:50 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:19:50.474+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:19:50.842 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:50 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:50.670+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:19:50.923 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.3.service' 2024-03-16T09:19:50.970 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:19:50.970 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-16T09:19:50.970 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-16T09:19:50.970 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.4 2024-03-16T09:19:51.152 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:50 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:19:50.765+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.297503+0000 front 2024-03-16T09:19:22.297449+0000 (oldest deadline 2024-03-16T09:19:48.197559+0000) 2024-03-16T09:19:51.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:50 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:50.879+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:19:51.237 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:51 smithi159 systemd[1]: Stopping Ceph osd.4 for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:19:51.237 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:51 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4[48007]: 2024-03-16T09:19:51.126+0000 7fb231cde700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.383552+0000 front 2024-03-16T09:19:21.383602+0000 (oldest deadline 2024-03-16T09:19:44.283468+0000) 2024-03-16T09:19:51.342 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:50 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:50.888+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:19:51.737 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:51 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4[48007]: 2024-03-16T09:19:51.277+0000 7fb2396f2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:19:51.737 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:51 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4[48007]: 2024-03-16T09:19:51.277+0000 7fb2396f2700 -1 osd.4 91 *** Got signal Terminated *** 2024-03-16T09:19:51.737 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:51 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4[48007]: 2024-03-16T09:19:51.277+0000 7fb2396f2700 -1 osd.4 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:19:51.842 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:51 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:51.694+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:19:51.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:19:51 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:19:51.478+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:19:51.902 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:51 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:19:51.719+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.297503+0000 front 2024-03-16T09:19:22.297449+0000 (oldest deadline 2024-03-16T09:19:48.197559+0000) 2024-03-16T09:19:52.149 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:51 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:51.830+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:19:52.342 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:51 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:51.929+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:19:52.487 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:52 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4[48007]: 2024-03-16T09:19:52.147+0000 7fb231cde700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.383552+0000 front 2024-03-16T09:19:21.383602+0000 (oldest deadline 2024-03-16T09:19:44.283468+0000) 2024-03-16T09:19:52.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:19:52 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:19:52.512+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:19:52.902 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:52 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:19:52.734+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.297503+0000 front 2024-03-16T09:19:22.297449+0000 (oldest deadline 2024-03-16T09:19:48.197559+0000) 2024-03-16T09:19:53.092 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:52 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:52.900+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:19:53.093 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:52 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:52.727+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:19:53.093 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:52 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:52.727+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:19:53.199 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:52 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:52.857+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:19:53.487 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:53 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4[48007]: 2024-03-16T09:19:53.197+0000 7fb231cde700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.383552+0000 front 2024-03-16T09:19:21.383602+0000 (oldest deadline 2024-03-16T09:19:44.283468+0000) 2024-03-16T09:19:53.902 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:53 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:19:53.751+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.297503+0000 front 2024-03-16T09:19:22.297449+0000 (oldest deadline 2024-03-16T09:19:48.197559+0000) 2024-03-16T09:19:53.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:19:53 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:19:53.559+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:19:54.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:53 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:53.740+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:19:54.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:53 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:53.740+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:19:54.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:53 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:53.924+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:19:54.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:53 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:53.924+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:19:54.181 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:19:54 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:19:54.009+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:19:54.181 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:53 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:53.812+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:19:54.181 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:53 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:53.812+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:19:54.402 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:19:53 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:19:53.918+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:19:54.486 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:54 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4[48007]: 2024-03-16T09:19:54.178+0000 7fb231cde700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.383552+0000 front 2024-03-16T09:19:21.383602+0000 (oldest deadline 2024-03-16T09:19:44.283468+0000) 2024-03-16T09:19:54.881 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:54 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:19:54.752+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.297503+0000 front 2024-03-16T09:19:22.297449+0000 (oldest deadline 2024-03-16T09:19:48.197559+0000) 2024-03-16T09:19:54.882 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:19:54 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:19:54.520+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:19:54.993 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:54 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:54.738+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:19:54.993 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:54 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:54.738+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:19:54.994 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:54 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:54.894+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:19:54.994 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:54 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:54.894+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:19:55.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:19:54 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:19:54.879+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:19:55.159 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:19:54 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:19:54.992+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:19:55.159 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:54 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:54.764+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:19:55.159 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:54 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:54.764+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:19:55.486 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:55 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4[48007]: 2024-03-16T09:19:55.156+0000 7fb231cde700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.383552+0000 front 2024-03-16T09:19:21.383602+0000 (oldest deadline 2024-03-16T09:19:44.283468+0000) 2024-03-16T09:19:55.791 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:19:55 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:19:55.474+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:19:56.034 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:55 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:55.770+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:19:56.034 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:55 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:55.770+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:19:56.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:55 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:55.700+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:19:56.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:55 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:55.700+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:19:56.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:55 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:55.863+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:19:56.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:55 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:55.863+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:19:56.152 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:55 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:19:55.788+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.297503+0000 front 2024-03-16T09:19:22.297449+0000 (oldest deadline 2024-03-16T09:19:48.197559+0000) 2024-03-16T09:19:56.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:19:55 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:19:55.832+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:19:56.487 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:56 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4[48007]: 2024-03-16T09:19:56.187+0000 7fb231cde700 -1 osd.4 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.383552+0000 front 2024-03-16T09:19:21.383602+0000 (oldest deadline 2024-03-16T09:19:44.283468+0000) 2024-03-16T09:19:56.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:19:56 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:19:56.031+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:19:56.823 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:19:56 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:19:56.462+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:19:56.823 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:19:56 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:19:56.462+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:19:57.072 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:56 smithi159 podman[69996]: 2024-03-16 09:19:56.817200432 +0000 UTC m=+5.655696431 container died 828aa1d92d65fc1a761b480f24d0c03180d9292a5c982c362764d41df5c978f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:19:57.072 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:19:56 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:19:56.996+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:19:57.073 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:56 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:56.778+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:19:57.073 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:56 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:56.778+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:19:57.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:56 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:56.723+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:19:57.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:56 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:56.723+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:19:57.092 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:56 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:56.849+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:19:57.092 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:56 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:56.849+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:19:57.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:19:56 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:19:56.828+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:19:57.152 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:56 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:19:56.821+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.297503+0000 front 2024-03-16T09:19:22.297449+0000 (oldest deadline 2024-03-16T09:19:48.197559+0000) 2024-03-16T09:19:57.152 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:56 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:19:56.821+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.698652+0000 front 2024-03-16T09:19:28.698670+0000 (oldest deadline 2024-03-16T09:19:56.299040+0000) 2024-03-16T09:19:57.473 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:57 smithi159 podman[69996]: 2024-03-16 09:19:57.170040946 +0000 UTC m=+6.008536924 container cleanup 828aa1d92d65fc1a761b480f24d0c03180d9292a5c982c362764d41df5c978f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-16T09:19:57.473 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:57 smithi159 bash[69996]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4 2024-03-16T09:19:57.734 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:57 smithi159 podman[70013]: 2024-03-16 09:19:57.471375712 +0000 UTC m=+0.651297117 container remove 828aa1d92d65fc1a761b480f24d0c03180d9292a5c982c362764d41df5c978f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-16T09:19:57.831 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:19:57 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:19:57.477+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:19:57.831 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:19:57 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:19:57.477+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:19:57.987 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:19:57 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:19:57.979+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:19:57.987 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:57 smithi159 podman[70103]: 2024-03-16 09:19:57.686526012 +0000 UTC m=+0.017721584 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:19:57.987 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:57 smithi159 podman[70103]: 2024-03-16 09:19:57.814056107 +0000 UTC m=+0.145251684 container create 4acc0a5cf63349de71978cbc2476ae51c1e1af75cec106e550953313b69c005f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-16T09:19:57.987 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:57 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:57.731+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:19:57.987 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:57 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:57.731+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:19:58.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:57 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:57.763+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:19:58.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:57 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:57.763+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:19:58.092 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:57 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:57.852+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:19:58.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:57 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:57.852+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:19:58.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:19:57 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:19:57.847+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:19:58.152 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:57 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:19:57.829+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.297503+0000 front 2024-03-16T09:19:22.297449+0000 (oldest deadline 2024-03-16T09:19:48.197559+0000) 2024-03-16T09:19:58.152 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:57 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:19:57.829+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.698652+0000 front 2024-03-16T09:19:28.698670+0000 (oldest deadline 2024-03-16T09:19:56.299040+0000) 2024-03-16T09:19:58.377 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:58 smithi159 podman[70103]: 2024-03-16 09:19:58.083273297 +0000 UTC m=+0.414468862 container init 4acc0a5cf63349de71978cbc2476ae51c1e1af75cec106e550953313b69c005f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-16T09:19:58.377 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:58 smithi159 podman[70103]: 2024-03-16 09:19:58.08640946 +0000 UTC m=+0.417605023 container start 4acc0a5cf63349de71978cbc2476ae51c1e1af75cec106e550953313b69c005f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-16T09:19:58.377 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:58 smithi159 podman[70103]: 2024-03-16 09:19:58.183302438 +0000 UTC m=+0.514498021 container attach 4acc0a5cf63349de71978cbc2476ae51c1e1af75cec106e550953313b69c005f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-16T09:19:58.671 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 16 09:19:58 smithi159 podman[70103]: 2024-03-16 09:19:58.375444595 +0000 UTC m=+0.706640156 container died 4acc0a5cf63349de71978cbc2476ae51c1e1af75cec106e550953313b69c005f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-03-16T09:19:58.806 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:19:58 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:19:58.500+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:19:58.806 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:19:58 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:19:58.500+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:19:58.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:58 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:58.761+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:19:58.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:58 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:58.761+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:19:58.974 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.4.service' 2024-03-16T09:19:59.016 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:19:59.016 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-16T09:19:59.016 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-16T09:19:59.016 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.5 2024-03-16T09:19:59.061 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:19:58 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:19:58.876+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:19:59.061 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:58 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:19:58.803+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.297503+0000 front 2024-03-16T09:19:22.297449+0000 (oldest deadline 2024-03-16T09:19:48.197559+0000) 2024-03-16T09:19:59.061 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:58 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:19:58.803+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.698652+0000 front 2024-03-16T09:19:28.698670+0000 (oldest deadline 2024-03-16T09:19:56.299040+0000) 2024-03-16T09:19:59.061 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:59 smithi194 systemd[1]: Stopping Ceph osd.5 for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:19:59.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:58 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:58.794+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:19:59.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:58 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:58.794+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:19:59.092 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:58 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:58.864+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:19:59.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:58 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:58.864+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:19:59.236 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:19:59 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:19:59.016+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:19:59.652 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:19:59 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:19:59.531+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:19:59.652 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:19:59 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:19:59.531+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:19:59.652 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:59 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:19:59.387+0000 7feed10a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:19:59.652 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:59 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:19:59.387+0000 7feed10a4700 -1 osd.5 91 *** Got signal Terminated *** 2024-03-16T09:19:59.652 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:59 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:19:59.387+0000 7feed10a4700 -1 osd.5 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:20:00.092 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:59 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:59.834+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:00.092 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:19:59 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:19:59.834+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:00.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:59 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:59.818+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:20:00.093 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:19:59 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:19:59.818+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:20:00.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:19:59 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:19:59.851+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:00.152 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:59 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:19:59.785+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.297503+0000 front 2024-03-16T09:19:22.297449+0000 (oldest deadline 2024-03-16T09:19:48.197559+0000) 2024-03-16T09:20:00.152 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:19:59 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:19:59.785+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.698652+0000 front 2024-03-16T09:19:28.698670+0000 (oldest deadline 2024-03-16T09:19:56.299040+0000) 2024-03-16T09:20:00.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:00 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:00.013+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:00.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:59 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:59.808+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:00.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:19:59 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:19:59.808+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:00.794 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:00 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:00.499+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:00.794 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:00 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:00.499+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:01.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:00 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:00.793+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:20:01.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:00 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:00.793+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:20:01.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:00 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:00.884+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:01.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:00 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:00.884+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:01.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:00 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:00.822+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:01.152 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:00 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:20:00.792+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.297503+0000 front 2024-03-16T09:19:22.297449+0000 (oldest deadline 2024-03-16T09:19:48.197559+0000) 2024-03-16T09:20:01.152 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:00 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:20:00.792+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.698652+0000 front 2024-03-16T09:19:28.698670+0000 (oldest deadline 2024-03-16T09:19:56.299040+0000) 2024-03-16T09:20:01.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:00 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:00.981+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:01.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:00 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:00.851+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:01.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:00 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:00.851+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:01.806 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:01 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:01.526+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:01.806 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:01 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:01.526+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:01.807 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:01 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:01.526+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:02.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:01 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:01.800+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:20:02.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:01 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:01.800+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:20:02.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:01 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:01.856+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:02.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:01 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:01.856+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:02.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:01 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:01.804+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:02.152 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:01 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:20:01.805+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.297503+0000 front 2024-03-16T09:19:22.297449+0000 (oldest deadline 2024-03-16T09:19:48.197559+0000) 2024-03-16T09:20:02.152 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:01 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:20:01.805+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.698652+0000 front 2024-03-16T09:19:28.698670+0000 (oldest deadline 2024-03-16T09:19:56.299040+0000) 2024-03-16T09:20:02.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:01 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:01.979+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:02.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:01 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:01.822+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:02.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:01 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:01.822+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:02.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:02 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:02.786+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:02.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:02 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:02.786+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:02.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:02 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:02.558+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:02.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:02 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:02.558+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:02.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:02 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:02.558+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:02.903 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:02 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:20:02.784+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.297503+0000 front 2024-03-16T09:19:22.297449+0000 (oldest deadline 2024-03-16T09:19:48.197559+0000) 2024-03-16T09:20:02.903 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:02 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:20:02.784+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.698652+0000 front 2024-03-16T09:19:28.698670+0000 (oldest deadline 2024-03-16T09:19:56.299040+0000) 2024-03-16T09:20:02.903 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:02 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:20:02.784+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.299369+0000 front 2024-03-16T09:19:36.299394+0000 (oldest deadline 2024-03-16T09:20:02.100103+0000) 2024-03-16T09:20:03.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:02 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:02.771+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:20:03.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:02 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:02.771+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:20:03.092 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:02 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:02.867+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:03.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:02 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:02.867+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:03.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:02 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:02.949+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:03.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:02 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:02.949+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:38.502466+0000 front 2024-03-16T09:19:38.502520+0000 (oldest deadline 2024-03-16T09:20:02.602422+0000) 2024-03-16T09:20:03.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:02 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:02.831+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:03.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:02 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:02.831+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:03.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:02 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:02.831+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:03.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:03 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:03.743+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:03.903 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:03 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:03.743+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:03.903 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:03 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:20:03.756+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.297503+0000 front 2024-03-16T09:19:22.297449+0000 (oldest deadline 2024-03-16T09:19:48.197559+0000) 2024-03-16T09:20:03.903 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:03 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:20:03.756+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.698652+0000 front 2024-03-16T09:19:28.698670+0000 (oldest deadline 2024-03-16T09:19:56.299040+0000) 2024-03-16T09:20:03.903 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:03 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5[48904]: 2024-03-16T09:20:03.756+0000 7feecd698700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.299369+0000 front 2024-03-16T09:19:36.299394+0000 (oldest deadline 2024-03-16T09:20:02.100103+0000) 2024-03-16T09:20:03.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:03 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:03.587+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:03.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:03 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:03.587+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:03.904 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:03 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:03.587+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:04.093 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:03 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:03.812+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:20:04.093 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:03 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:03.812+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:20:04.093 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:03 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:03.812+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.478454+0000 front 2024-03-16T09:19:39.478396+0000 (oldest deadline 2024-03-16T09:20:03.578427+0000) 2024-03-16T09:20:04.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:03 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:03.823+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:04.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:03 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:03.823+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:04.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:03 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:03.823+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:04.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:03 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:03.900+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:04.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:03 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:03.900+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:38.502466+0000 front 2024-03-16T09:19:38.502520+0000 (oldest deadline 2024-03-16T09:20:02.602422+0000) 2024-03-16T09:20:04.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:03 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:03.879+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:04.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:03 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:03.879+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:04.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:03 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:03.879+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:04.902 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:04 smithi194 podman[71864]: 2024-03-16 09:20:04.833631489 +0000 UTC m=+5.636779458 container died 1a93c2469af9062798f82a671835627993ce29ed54f8428ef73a5c6e2c86f64a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux ) 2024-03-16T09:20:04.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:04 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:04.735+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:04.903 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:04 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:04.735+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:04.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:04 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:04.628+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:04.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:04 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:04.628+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:04.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:04 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:04.628+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:05.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:04 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:04.879+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:05.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:04 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:04.879+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:38.502466+0000 front 2024-03-16T09:19:38.502520+0000 (oldest deadline 2024-03-16T09:20:02.602422+0000) 2024-03-16T09:20:05.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:04 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:04.831+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:05.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:04 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:04.831+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:05.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:04 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:04.831+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:05.342 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:04 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:04.844+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:20:05.343 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:04 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:04.844+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:20:05.343 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:04 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:04.844+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.478454+0000 front 2024-03-16T09:19:39.478396+0000 (oldest deadline 2024-03-16T09:20:03.578427+0000) 2024-03-16T09:20:05.343 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:04 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:04.844+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:05.343 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:04 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:04.844+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:05.343 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:04 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:04.844+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:05.402 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:05 smithi194 podman[71864]: 2024-03-16 09:20:05.205122635 +0000 UTC m=+6.008270604 container cleanup 1a93c2469af9062798f82a671835627993ce29ed54f8428ef73a5c6e2c86f64a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-16T09:20:05.402 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:05 smithi194 bash[71864]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5 2024-03-16T09:20:05.817 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:05 smithi194 podman[71882]: 2024-03-16 09:20:05.498421098 +0000 UTC m=+0.658695823 container remove 1a93c2469af9062798f82a671835627993ce29ed54f8428ef73a5c6e2c86f64a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-03-16T09:20:05.818 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:05 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:05.725+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:05.818 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:05 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:05.726+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:05.818 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:05 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:05.676+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:05.818 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:05 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:05.676+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:05.818 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:05 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:05.676+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:06.087 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:05 smithi194 podman[71971]: 2024-03-16 09:20:05.718429408 +0000 UTC m=+0.017951128 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:20:06.087 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:05 smithi194 podman[71971]: 2024-03-16 09:20:05.832506461 +0000 UTC m=+0.132028182 container create 7fd0bfce530fa56d4530ad8caf2b0941f99b5c635cf37aa7dfaff8fa7b8b64c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:20:06.095 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:05 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:05.835+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:20:06.095 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:05 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:05.835+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:20:06.095 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:05 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:05.835+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.478454+0000 front 2024-03-16T09:19:39.478396+0000 (oldest deadline 2024-03-16T09:20:03.578427+0000) 2024-03-16T09:20:06.095 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:05 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:05.819+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:06.095 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:05 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:05.819+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:06.095 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:05 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:05.819+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:06.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:05 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:05.926+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:06.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:05 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:05.926+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:38.502466+0000 front 2024-03-16T09:19:38.502520+0000 (oldest deadline 2024-03-16T09:20:02.602422+0000) 2024-03-16T09:20:06.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:05 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:05.817+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:06.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:05 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:05.817+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:06.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:05 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:05.817+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:06.372 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:06 smithi194 podman[71971]: 2024-03-16 09:20:06.084834378 +0000 UTC m=+0.384356113 container init 7fd0bfce530fa56d4530ad8caf2b0941f99b5c635cf37aa7dfaff8fa7b8b64c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-16T09:20:06.372 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:06 smithi194 podman[71971]: 2024-03-16 09:20:06.087991811 +0000 UTC m=+0.387513540 container start 7fd0bfce530fa56d4530ad8caf2b0941f99b5c635cf37aa7dfaff8fa7b8b64c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-03-16T09:20:06.372 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:06 smithi194 podman[71971]: 2024-03-16 09:20:06.185022599 +0000 UTC m=+0.484544339 container attach 7fd0bfce530fa56d4530ad8caf2b0941f99b5c635cf37aa7dfaff8fa7b8b64c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-16T09:20:06.652 INFO:journalctl@ceph.osd.5.smithi194.stdout:Mar 16 09:20:06 smithi194 podman[71971]: 2024-03-16 09:20:06.370821426 +0000 UTC m=+0.670343152 container died 7fd0bfce530fa56d4530ad8caf2b0941f99b5c635cf37aa7dfaff8fa7b8b64c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:20:06.999 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:06 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:06.752+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:06.999 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:06 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:06.752+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:06.999 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:06 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:06.693+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:06.999 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:06 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:06.693+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:06.999 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:06 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:06.693+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:07.023 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.5.service' 2024-03-16T09:20:07.065 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:20:07.066 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-16T09:20:07.066 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-16T09:20:07.066 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.6 2024-03-16T09:20:07.092 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:06 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:06.787+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:07.092 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:06 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:06.787+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:07.092 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:06 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:06.787+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:07.093 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:06 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:06.877+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:20:07.093 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:06 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:06.877+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:20:07.093 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:06 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:06.877+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.478454+0000 front 2024-03-16T09:19:39.478396+0000 (oldest deadline 2024-03-16T09:20:03.578427+0000) 2024-03-16T09:20:07.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:06 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:06.934+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:07.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:06 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:06.934+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:38.502466+0000 front 2024-03-16T09:19:38.502520+0000 (oldest deadline 2024-03-16T09:20:02.602422+0000) 2024-03-16T09:20:07.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:06 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:06.770+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:07.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:06 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:06.770+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:07.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:06 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:06.770+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:07.365 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:07 smithi112 systemd[1]: Stopping Ceph osd.6 for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:20:07.756 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:07 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:07.362+0000 7f4736c3a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:20:07.757 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:07 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:07.362+0000 7f4736c3a700 -1 osd.6 91 *** Got signal Terminated *** 2024-03-16T09:20:07.757 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:07 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:07.362+0000 7f4736c3a700 -1 osd.6 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:20:08.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:07 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:07.866+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:20:08.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:07 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:07.866+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:20:08.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:07 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:07.866+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.478454+0000 front 2024-03-16T09:19:39.478396+0000 (oldest deadline 2024-03-16T09:20:03.578427+0000) 2024-03-16T09:20:08.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:07 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:07.754+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:08.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:07 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:07.754+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:08.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:07 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:07.754+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:08.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:07 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:07.739+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:08.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:07 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:07.739+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:08.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:07 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:07.707+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:08.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:07 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:07.707+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:08.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:07 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:07.707+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:08.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:07 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:07.963+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:08.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:07 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:07.963+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:38.502466+0000 front 2024-03-16T09:19:38.502520+0000 (oldest deadline 2024-03-16T09:20:02.602422+0000) 2024-03-16T09:20:08.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:07 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:07.790+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:08.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:07 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:07.790+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:08.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:07 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:07.790+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:09.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:08.820+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:20:09.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:08.820+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:20:09.093 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:08.820+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.478454+0000 front 2024-03-16T09:19:39.478396+0000 (oldest deadline 2024-03-16T09:20:03.578427+0000) 2024-03-16T09:20:09.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:08.719+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:09.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:08.719+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:09.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:08 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:08.719+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:09.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:08 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:08.750+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:09.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:08 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:08.750+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:09.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:08 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:08.695+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:09.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:08 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:08.695+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:09.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:08 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:08.695+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:09.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:09 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:09.009+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:09.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:09 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:09.009+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:38.502466+0000 front 2024-03-16T09:19:38.502520+0000 (oldest deadline 2024-03-16T09:20:02.602422+0000) 2024-03-16T09:20:09.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:08 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:08.793+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:09.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:08 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:08.793+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:09.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:08 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:08.793+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:10.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:09 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:09.828+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:20:10.093 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:09 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:09.828+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:20:10.093 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:09 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:09.828+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.478454+0000 front 2024-03-16T09:19:39.478396+0000 (oldest deadline 2024-03-16T09:20:03.578427+0000) 2024-03-16T09:20:10.093 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:09 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:09.828+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.079517+0000 front 2024-03-16T09:19:47.079478+0000 (oldest deadline 2024-03-16T09:20:09.379613+0000) 2024-03-16T09:20:10.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:09 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:09.736+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:10.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:09 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:09.736+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:10.094 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:09 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:09.736+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:10.094 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:09 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:09.736+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:10.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:09 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:09.794+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:10.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:09 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:09.794+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:10.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:09 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:09.794+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:42.977192+0000 front 2024-03-16T09:19:42.977205+0000 (oldest deadline 2024-03-16T09:20:08.877122+0000) 2024-03-16T09:20:10.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:09 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:09.694+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:10.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:09 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:09.694+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:10.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:09 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:09.694+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:10.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:09 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:09.994+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:10.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:09 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:09.994+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:38.502466+0000 front 2024-03-16T09:19:38.502520+0000 (oldest deadline 2024-03-16T09:20:02.602422+0000) 2024-03-16T09:20:10.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:09 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:09.773+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:10.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:09 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:09.773+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:10.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:09 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:09.773+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:10.987 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:10 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:10.726+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:10.987 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:10 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:10.726+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:10.987 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:10 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:10.726+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:11.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:10 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:10.856+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:20:11.093 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:10 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:10.856+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:20:11.093 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:10 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:10.856+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.478454+0000 front 2024-03-16T09:19:39.478396+0000 (oldest deadline 2024-03-16T09:20:03.578427+0000) 2024-03-16T09:20:11.093 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:10 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:10.856+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.079517+0000 front 2024-03-16T09:19:47.079478+0000 (oldest deadline 2024-03-16T09:20:09.379613+0000) 2024-03-16T09:20:11.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:10 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:10.707+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:11.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:10 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:10.707+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:11.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:10 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:10.707+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:11.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:10 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:10.707+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:11.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:10 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:10.817+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:11.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:10 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:10.817+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:11.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:10 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:10.817+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:42.977192+0000 front 2024-03-16T09:19:42.977205+0000 (oldest deadline 2024-03-16T09:20:08.877122+0000) 2024-03-16T09:20:11.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:10 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:10.655+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:11.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:10 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:10.655+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:11.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:10 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:10.655+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:11.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:10 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:10.994+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:11.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:10 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:10.994+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:38.502466+0000 front 2024-03-16T09:19:38.502520+0000 (oldest deadline 2024-03-16T09:20:02.602422+0000) 2024-03-16T09:20:11.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:10 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:10.994+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:46.103634+0000 front 2024-03-16T09:19:46.103548+0000 (oldest deadline 2024-03-16T09:20:10.803523+0000) 2024-03-16T09:20:11.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:11 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:11.611+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:11.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:11 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:11.611+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:11.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:11 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:11.611+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:11.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:11 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:11.802+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:11.903 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:11 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:11.802+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:11.903 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:11 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:11.802+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:42.977192+0000 front 2024-03-16T09:19:42.977205+0000 (oldest deadline 2024-03-16T09:20:08.877122+0000) 2024-03-16T09:20:11.987 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:11 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:11.728+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:11.987 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:11 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:11.728+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:11.987 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:11 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:11.728+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:12.092 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:11 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:11.899+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:22.575374+0000 front 2024-03-16T09:19:22.575307+0000 (oldest deadline 2024-03-16T09:19:44.275622+0000) 2024-03-16T09:20:12.093 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:11 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:11.899+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:26.576567+0000 front 2024-03-16T09:19:26.576624+0000 (oldest deadline 2024-03-16T09:19:52.476663+0000) 2024-03-16T09:20:12.093 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:11 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:11.899+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.478454+0000 front 2024-03-16T09:19:39.478396+0000 (oldest deadline 2024-03-16T09:20:03.578427+0000) 2024-03-16T09:20:12.093 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:11 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6[60885]: 2024-03-16T09:20:11.899+0000 7f473322e700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.079517+0000 front 2024-03-16T09:19:47.079478+0000 (oldest deadline 2024-03-16T09:20:09.379613+0000) 2024-03-16T09:20:12.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:11 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:11.665+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:12.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:11 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:11.665+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:12.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:11 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:11.665+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:12.093 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:11 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:11.665+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:12.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:12 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:12.001+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:12.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:12 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:12.001+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:38.502466+0000 front 2024-03-16T09:19:38.502520+0000 (oldest deadline 2024-03-16T09:20:02.602422+0000) 2024-03-16T09:20:12.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:12 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:12.001+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:46.103634+0000 front 2024-03-16T09:19:46.103548+0000 (oldest deadline 2024-03-16T09:20:10.803523+0000) 2024-03-16T09:20:12.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:12 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:12.799+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:12.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:12 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:12.799+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:12.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:12 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:12.799+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:42.977192+0000 front 2024-03-16T09:19:42.977205+0000 (oldest deadline 2024-03-16T09:20:08.877122+0000) 2024-03-16T09:20:12.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:12 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:12.577+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:12.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:12 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:12.577+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:12.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:12 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:12.577+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:12.999 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:12 smithi112 podman[90961]: 2024-03-16 09:20:12.692088973 +0000 UTC m=+5.439278334 container died c1310282009d07956a7659b496bef6fc2b6ea010af6f8fa418c4028c1ba5e616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-16T09:20:12.999 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:12 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:12.631+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:12.999 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:12 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:12.631+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:12.999 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:12 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:12.631+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:12.999 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:12 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:12.631+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:13.021 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:12 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:12.742+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:13.021 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:12 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:12.742+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:13.021 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:12 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:12.742+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:13.021 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:12 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:12.742+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:13.342 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:13 smithi112 podman[90961]: 2024-03-16 09:20:13.179779623 +0000 UTC m=+5.926968988 container cleanup c1310282009d07956a7659b496bef6fc2b6ea010af6f8fa418c4028c1ba5e616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-16T09:20:13.342 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:13 smithi112 bash[90961]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6 2024-03-16T09:20:13.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:13 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:13.019+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:13.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:13 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:13.019+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:38.502466+0000 front 2024-03-16T09:19:38.502520+0000 (oldest deadline 2024-03-16T09:20:02.602422+0000) 2024-03-16T09:20:13.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:13 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:13.019+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:46.103634+0000 front 2024-03-16T09:19:46.103548+0000 (oldest deadline 2024-03-16T09:20:10.803523+0000) 2024-03-16T09:20:13.758 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:13 smithi112 podman[90979]: 2024-03-16 09:20:13.456750806 +0000 UTC m=+0.760544200 container remove c1310282009d07956a7659b496bef6fc2b6ea010af6f8fa418c4028c1ba5e616 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:20:13.758 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:13 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:13.596+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:13.758 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:13 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:13.596+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:13.758 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:13 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:13.596+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:13.758 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:13 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:13.596+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:13.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:13 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:13.773+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:13.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:13 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:13.773+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:13.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:13 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:13.773+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:42.977192+0000 front 2024-03-16T09:19:42.977205+0000 (oldest deadline 2024-03-16T09:20:08.877122+0000) 2024-03-16T09:20:13.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:13 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:13.571+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:13.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:13 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:13.571+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:13.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:13 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:13.571+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:14.034 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:13 smithi112 podman[91070]: 2024-03-16 09:20:13.658944827 +0000 UTC m=+0.018050239 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:20:14.034 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:13 smithi112 podman[91070]: 2024-03-16 09:20:13.796757668 +0000 UTC m=+0.155863067 container create 53ffb82fc4fa49321bca1eb8422fcb8989129c4759ae3de47f054c55378fe0a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True) 2024-03-16T09:20:14.034 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:14 smithi112 podman[91070]: 2024-03-16 09:20:14.032927475 +0000 UTC m=+0.392032887 container init 53ffb82fc4fa49321bca1eb8422fcb8989129c4759ae3de47f054c55378fe0a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:20:14.060 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:13 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:13.743+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:14.060 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:13 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:13.743+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:14.060 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:13 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:13.743+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:14.060 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:13 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:13.743+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:14.326 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:14 smithi112 podman[91070]: 2024-03-16 09:20:14.036154898 +0000 UTC m=+0.395260300 container start 53ffb82fc4fa49321bca1eb8422fcb8989129c4759ae3de47f054c55378fe0a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307) 2024-03-16T09:20:14.326 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:14 smithi112 podman[91070]: 2024-03-16 09:20:14.14125598 +0000 UTC m=+0.500361397 container attach 53ffb82fc4fa49321bca1eb8422fcb8989129c4759ae3de47f054c55378fe0a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-03-16T09:20:14.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:14 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:14.058+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:14.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:14 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:14.058+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:38.502466+0000 front 2024-03-16T09:19:38.502520+0000 (oldest deadline 2024-03-16T09:20:02.602422+0000) 2024-03-16T09:20:14.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:14 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:14.058+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:46.103634+0000 front 2024-03-16T09:19:46.103548+0000 (oldest deadline 2024-03-16T09:20:10.803523+0000) 2024-03-16T09:20:14.592 INFO:journalctl@ceph.osd.6.smithi112.stdout:Mar 16 09:20:14 smithi112 podman[91070]: 2024-03-16 09:20:14.324834148 +0000 UTC m=+0.683939539 container died 53ffb82fc4fa49321bca1eb8422fcb8989129c4759ae3de47f054c55378fe0a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-16T09:20:14.593 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:14 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:14.570+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:14.593 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:14 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:14.570+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:14.593 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:14 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:14.570+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:14.593 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:14 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:14.570+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:14.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:14 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:14.726+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:14.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:14 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:14.726+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:14.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:14 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:14.726+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:42.977192+0000 front 2024-03-16T09:19:42.977205+0000 (oldest deadline 2024-03-16T09:20:08.877122+0000) 2024-03-16T09:20:14.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:14 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:14.616+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:14.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:14 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:14.616+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:14.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:14 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:14.616+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:15.033 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.6.service' 2024-03-16T09:20:15.108 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:14 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:14.737+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:15.108 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:14 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:14.737+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:15.108 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:14 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:14.737+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:15.108 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:14 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:14.737+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:15.115 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:20:15.115 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-16T09:20:15.116 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-16T09:20:15.116 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.7 2024-03-16T09:20:15.425 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:15 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:15.105+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:15.425 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:15 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:15.105+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:38.502466+0000 front 2024-03-16T09:19:38.502520+0000 (oldest deadline 2024-03-16T09:20:02.602422+0000) 2024-03-16T09:20:15.425 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:15 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:15.105+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:46.103634+0000 front 2024-03-16T09:19:46.103548+0000 (oldest deadline 2024-03-16T09:20:10.803523+0000) 2024-03-16T09:20:15.426 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:15 smithi159 systemd[1]: Stopping Ceph osd.7 for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:20:15.426 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:15 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:15.422+0000 7f36771e2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:20:15.426 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:15 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:15.422+0000 7f36771e2700 -1 osd.7 91 *** Got signal Terminated *** 2024-03-16T09:20:15.426 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:15 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:15.422+0000 7f36771e2700 -1 osd.7 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:20:15.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:15 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:15.541+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:15.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:15 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:15.541+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:15.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:15 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:15.541+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:15.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:15 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:15.541+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:16.127 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:15 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:15.749+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:16.127 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:15 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:15.749+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:16.127 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:15 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:15.749+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:16.127 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:15 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:15.749+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:16.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:15 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:15.750+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:16.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:15 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:15.750+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:16.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:15 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:15.750+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:42.977192+0000 front 2024-03-16T09:19:42.977205+0000 (oldest deadline 2024-03-16T09:20:08.877122+0000) 2024-03-16T09:20:16.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:15 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:15.661+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:16.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:15 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:15.661+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:16.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:15 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:15.661+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:16.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:16 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:16.121+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:16.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:16 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:16.121+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:38.502466+0000 front 2024-03-16T09:19:38.502520+0000 (oldest deadline 2024-03-16T09:20:02.602422+0000) 2024-03-16T09:20:16.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:16 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:16.121+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:46.103634+0000 front 2024-03-16T09:19:46.103548+0000 (oldest deadline 2024-03-16T09:20:10.803523+0000) 2024-03-16T09:20:16.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:16 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:16.532+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:16.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:16 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:16.532+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:16.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:16 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:16.532+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:16.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:16 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:16.532+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:17.053 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:16 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:16.710+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:17.054 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:16 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:16.710+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:17.054 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:16 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:16.710+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:17.054 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:16 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:16.716+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:17.054 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:16 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:16.716+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:17.054 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:16 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:16.716+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:42.977192+0000 front 2024-03-16T09:19:42.977205+0000 (oldest deadline 2024-03-16T09:20:08.877122+0000) 2024-03-16T09:20:17.122 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:16 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:16.756+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:17.122 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:16 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:16.756+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:17.122 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:16 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:16.756+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:17.122 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:16 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:16.756+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:17.486 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:17 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:17.120+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:17.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:17 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:17.120+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:38.502466+0000 front 2024-03-16T09:19:38.502520+0000 (oldest deadline 2024-03-16T09:20:02.602422+0000) 2024-03-16T09:20:17.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:17 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:17.120+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:46.103634+0000 front 2024-03-16T09:19:46.103548+0000 (oldest deadline 2024-03-16T09:20:10.803523+0000) 2024-03-16T09:20:17.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:17 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:17.542+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:17.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:17 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:17.542+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:17.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:17 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:17.542+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:17.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:17 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:17.542+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:18.067 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:17 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:17.721+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:18.067 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:17 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:17.721+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:18.067 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:17 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:17.721+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:18.067 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:17 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:17.759+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:18.068 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:17 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:17.759+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:18.068 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:17 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:17.759+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:42.977192+0000 front 2024-03-16T09:19:42.977205+0000 (oldest deadline 2024-03-16T09:20:08.877122+0000) 2024-03-16T09:20:18.144 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:17 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:17.789+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:18.144 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:17 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:17.789+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:18.144 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:17 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:17.789+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:18.144 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:17 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:17.789+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:18.486 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:18 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:18.141+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:18.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:18 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:18.141+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:38.502466+0000 front 2024-03-16T09:19:38.502520+0000 (oldest deadline 2024-03-16T09:20:02.602422+0000) 2024-03-16T09:20:18.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:18 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:18.141+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:46.103634+0000 front 2024-03-16T09:19:46.103548+0000 (oldest deadline 2024-03-16T09:20:10.803523+0000) 2024-03-16T09:20:18.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:18 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:18.141+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.304543+0000 front 2024-03-16T09:19:54.304778+0000 (oldest deadline 2024-03-16T09:20:17.204808+0000) 2024-03-16T09:20:18.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:18 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:18.528+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:18.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:18 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:18.528+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:18.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:18 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:18.528+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:18.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:18 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:18.528+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:19.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:18 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:18.731+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:19.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:18 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:18.731+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:19.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:18 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:18.731+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:42.977192+0000 front 2024-03-16T09:19:42.977205+0000 (oldest deadline 2024-03-16T09:20:08.877122+0000) 2024-03-16T09:20:19.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:18 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:18.756+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:19.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:18 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:18.756+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:19.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:18 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:18.756+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:19.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:18 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:18.756+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:19.154 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:18 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:18.801+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:19.154 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:18 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:18.801+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:19.154 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:18 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:18.801+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:19.155 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:18 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:18.801+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:19.155 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:18 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:18.801+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:19.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:19.152+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:19.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:19.152+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:38.502466+0000 front 2024-03-16T09:19:38.502520+0000 (oldest deadline 2024-03-16T09:20:02.602422+0000) 2024-03-16T09:20:19.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:19.152+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:46.103634+0000 front 2024-03-16T09:19:46.103548+0000 (oldest deadline 2024-03-16T09:20:10.803523+0000) 2024-03-16T09:20:19.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:19.152+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.304543+0000 front 2024-03-16T09:19:54.304778+0000 (oldest deadline 2024-03-16T09:20:17.204808+0000) 2024-03-16T09:20:19.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:19 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:19.500+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:19.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:19 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:19.500+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:19.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:19 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:19.500+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:19.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:19 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:19.500+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:19.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:19 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:19.500+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:55.259165+0000 front 2024-03-16T09:19:55.259123+0000 (oldest deadline 2024-03-16T09:20:18.759086+0000) 2024-03-16T09:20:20.151 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:19.809+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:20.151 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:19.809+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:20.151 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:19.809+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:20.151 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:19.809+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:20.151 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:19 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:19.809+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:20.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:19 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:19.712+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:20.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:19 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:19.712+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:20.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:19 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:19.712+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:42.977192+0000 front 2024-03-16T09:19:42.977205+0000 (oldest deadline 2024-03-16T09:20:08.877122+0000) 2024-03-16T09:20:20.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:19 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:19.712+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.178164+0000 front 2024-03-16T09:19:54.178126+0000 (oldest deadline 2024-03-16T09:20:18.877935+0000) 2024-03-16T09:20:20.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:19 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:19.726+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:20.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:19 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:19.726+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:20.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:19 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:19.726+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:20.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:19 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:19.726+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:20.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:20 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:20.149+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:29.200751+0000 front 2024-03-16T09:19:29.200776+0000 (oldest deadline 2024-03-16T09:19:53.900883+0000) 2024-03-16T09:20:20.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:20 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:20.149+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:38.502466+0000 front 2024-03-16T09:19:38.502520+0000 (oldest deadline 2024-03-16T09:20:02.602422+0000) 2024-03-16T09:20:20.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:20 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:20.149+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:46.103634+0000 front 2024-03-16T09:19:46.103548+0000 (oldest deadline 2024-03-16T09:20:10.803523+0000) 2024-03-16T09:20:20.487 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:20 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7[54595]: 2024-03-16T09:20:20.149+0000 7f366f7ce700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.304543+0000 front 2024-03-16T09:19:54.304778+0000 (oldest deadline 2024-03-16T09:20:17.204808+0000) 2024-03-16T09:20:20.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:20 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:20.517+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:20.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:20 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:20.517+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:20.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:20 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:20.517+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:20.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:20 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:20.517+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:20.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:20 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:20.517+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:55.259165+0000 front 2024-03-16T09:19:55.259123+0000 (oldest deadline 2024-03-16T09:20:18.759086+0000) 2024-03-16T09:20:21.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:20 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:20.663+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:21.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:20 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:20.663+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:21.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:20 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:20.663+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:42.977192+0000 front 2024-03-16T09:19:42.977205+0000 (oldest deadline 2024-03-16T09:20:08.877122+0000) 2024-03-16T09:20:21.152 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:20 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:20.663+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.178164+0000 front 2024-03-16T09:19:54.178126+0000 (oldest deadline 2024-03-16T09:20:18.877935+0000) 2024-03-16T09:20:21.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:20 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:20.683+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:21.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:20 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:20.683+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:21.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:20 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:20.683+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:21.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:20 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:20.683+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:21.237 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:20 smithi159 podman[70209]: 2024-03-16 09:20:20.954549572 +0000 UTC m=+5.637610331 container died 9d88e43df177067ab133abd8d2f665d781e23c91e6c1cd655882f30c003412e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:20:21.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:20 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:20.799+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:21.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:20 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:20.799+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:21.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:20 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:20.799+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:21.238 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:20 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:20.799+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:21.238 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:20 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:20.799+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:21.601 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:21 smithi159 podman[70209]: 2024-03-16 09:20:21.364644814 +0000 UTC m=+6.047705571 container cleanup 9d88e43df177067ab133abd8d2f665d781e23c91e6c1cd655882f30c003412e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:20:21.601 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:21 smithi159 bash[70209]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7 2024-03-16T09:20:21.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:21 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:21.506+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:21.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:21 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:21.506+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:21.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:21 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:21.506+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:21.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:21 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:21.506+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:21.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:21 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:21.506+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:55.259165+0000 front 2024-03-16T09:19:55.259123+0000 (oldest deadline 2024-03-16T09:20:18.759086+0000) 2024-03-16T09:20:21.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:21 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:21.621+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:21.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:21 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:21.621+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:21.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:21 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:21.621+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:42.977192+0000 front 2024-03-16T09:19:42.977205+0000 (oldest deadline 2024-03-16T09:20:08.877122+0000) 2024-03-16T09:20:21.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:21 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:21.621+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.178164+0000 front 2024-03-16T09:19:54.178126+0000 (oldest deadline 2024-03-16T09:20:18.877935+0000) 2024-03-16T09:20:21.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:21 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:21.657+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:21.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:21 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:21.657+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:21.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:21 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:21.657+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:21.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:21 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:21.657+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:21.987 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:21 smithi159 podman[70227]: 2024-03-16 09:20:21.599285168 +0000 UTC m=+0.647398967 container remove 9d88e43df177067ab133abd8d2f665d781e23c91e6c1cd655882f30c003412e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-16T09:20:21.987 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:21 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:21.777+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:21.987 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:21 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:21.777+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:21.987 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:21 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:21.777+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:21.987 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:21 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:21.777+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:21.987 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:21 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:21.777+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:22.305 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:22 smithi159 podman[70323]: 2024-03-16 09:20:22.025340033 +0000 UTC m=+0.110487563 container create 655db1ea289c6d3e2a31bed46a3dc7692cc99c7b0e89c90f375619c9ccbfa162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0) 2024-03-16T09:20:22.305 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:22 smithi159 podman[70323]: 2024-03-16 09:20:21.932461632 +0000 UTC m=+0.017609170 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:20:22.305 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:22 smithi159 podman[70323]: 2024-03-16 09:20:22.219390896 +0000 UTC m=+0.304538439 container init 655db1ea289c6d3e2a31bed46a3dc7692cc99c7b0e89c90f375619c9ccbfa162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-16T09:20:22.305 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:22 smithi159 podman[70323]: 2024-03-16 09:20:22.222562316 +0000 UTC m=+0.307709846 container start 655db1ea289c6d3e2a31bed46a3dc7692cc99c7b0e89c90f375619c9ccbfa162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:20:22.305 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:22 smithi159 podman[70323]: 2024-03-16 09:20:22.302616412 +0000 UTC m=+0.387763962 container attach 655db1ea289c6d3e2a31bed46a3dc7692cc99c7b0e89c90f375619c9ccbfa162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2) 2024-03-16T09:20:22.737 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 16 09:20:22 smithi159 podman[70323]: 2024-03-16 09:20:22.507160963 +0000 UTC m=+0.592308486 container died 655db1ea289c6d3e2a31bed46a3dc7692cc99c7b0e89c90f375619c9ccbfa162 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-16T09:20:22.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:22 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:22.487+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:22.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:22 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:22.487+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:22.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:22 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:22.487+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:22.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:22 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:22.487+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:22.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:22 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:22.487+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:55.259165+0000 front 2024-03-16T09:19:55.259123+0000 (oldest deadline 2024-03-16T09:20:18.759086+0000) 2024-03-16T09:20:22.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:22 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:22.627+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:22.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:22 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:22.627+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:22.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:22 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:22.627+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:42.977192+0000 front 2024-03-16T09:19:42.977205+0000 (oldest deadline 2024-03-16T09:20:08.877122+0000) 2024-03-16T09:20:22.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:22 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:22.627+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.178164+0000 front 2024-03-16T09:19:54.178126+0000 (oldest deadline 2024-03-16T09:20:18.877935+0000) 2024-03-16T09:20:22.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:22 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:22.631+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:22.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:22 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:22.631+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:22.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:22 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:22.631+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:22.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:22 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:22.631+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:23.075 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:22 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:22.790+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:23.075 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:22 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:22.790+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:23.075 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:22 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:22.790+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:23.075 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:22 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:22.790+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:23.075 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:22 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:22.790+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:23.094 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.7.service' 2024-03-16T09:20:23.134 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:20:23.135 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-16T09:20:23.135 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-16T09:20:23.135 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.8 2024-03-16T09:20:23.460 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:23 smithi194 systemd[1]: Stopping Ceph osd.8 for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:20:23.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:23 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:23.461+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:23.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:23 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:23.461+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:23.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:23 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:23.461+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:23.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:23 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:23.461+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:23.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:23 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:23.461+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:55.259165+0000 front 2024-03-16T09:19:55.259123+0000 (oldest deadline 2024-03-16T09:20:18.759086+0000) 2024-03-16T09:20:23.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:23 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:23.650+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:23.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:23 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:23.650+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:23.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:23 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:23.650+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:23.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:23 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:23.650+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:23.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:23 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:23.457+0000 7fb79bd51700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:20:23.903 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:23 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:23.457+0000 7fb79bd51700 -1 osd.8 91 *** Got signal Terminated *** 2024-03-16T09:20:23.903 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:23 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:23.457+0000 7fb79bd51700 -1 osd.8 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:20:23.903 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:23 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:23.582+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:23.903 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:23 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:23.582+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:23.903 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:23 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:23.582+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:42.977192+0000 front 2024-03-16T09:19:42.977205+0000 (oldest deadline 2024-03-16T09:20:08.877122+0000) 2024-03-16T09:20:23.903 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:23 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:23.582+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.178164+0000 front 2024-03-16T09:19:54.178126+0000 (oldest deadline 2024-03-16T09:20:18.877935+0000) 2024-03-16T09:20:24.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:23 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:23.805+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:24.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:23 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:23.805+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:24.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:23 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:23.805+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:24.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:23 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:23.805+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:24.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:23 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:23.805+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:24.743 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:24 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:24.423+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:24.744 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:24 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:24.423+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:24.744 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:24 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:24.423+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:24.744 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:24 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:24.423+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:24.744 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:24 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:24.423+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:55.259165+0000 front 2024-03-16T09:19:55.259123+0000 (oldest deadline 2024-03-16T09:20:18.759086+0000) 2024-03-16T09:20:24.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:24 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:24.536+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:24.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:24 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:24.536+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:24.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:24 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:24.536+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:42.977192+0000 front 2024-03-16T09:19:42.977205+0000 (oldest deadline 2024-03-16T09:20:08.877122+0000) 2024-03-16T09:20:24.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:24 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:24.536+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.178164+0000 front 2024-03-16T09:19:54.178126+0000 (oldest deadline 2024-03-16T09:20:18.877935+0000) 2024-03-16T09:20:24.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:24 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:24.696+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:24.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:24 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:24.696+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:24.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:24 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:24.696+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:24.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:24 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:24.696+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:25.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:24 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:24.803+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:25.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:24 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:24.803+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:25.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:24 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:24.803+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:25.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:24 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:24.803+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:25.238 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:24 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:24.803+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:25.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:25 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:25.467+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:25.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:25 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:25.467+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:25.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:25 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:25.467+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:25.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:25 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:25.467+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:25.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:25 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:25.467+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:55.259165+0000 front 2024-03-16T09:19:55.259123+0000 (oldest deadline 2024-03-16T09:20:18.759086+0000) 2024-03-16T09:20:25.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:25 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:25.531+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:25.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:25 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:25.531+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:25.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:25 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:25.531+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:42.977192+0000 front 2024-03-16T09:19:42.977205+0000 (oldest deadline 2024-03-16T09:20:08.877122+0000) 2024-03-16T09:20:25.902 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:25 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:25.531+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.178164+0000 front 2024-03-16T09:19:54.178126+0000 (oldest deadline 2024-03-16T09:20:18.877935+0000) 2024-03-16T09:20:25.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:25 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:25.661+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:25.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:25 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:25.661+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:25.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:25 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:25.661+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:25.903 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:25 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:25.661+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:26.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:25 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:25.822+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:26.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:25 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:25.822+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:26.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:25 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:25.822+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:26.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:25 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:25.822+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:26.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:25 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:25.822+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:26.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:26 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:26.479+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:26.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:26 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:26.479+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:26.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:26 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:26.479+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:26.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:26 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:26.479+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:26.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:26 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:26.479+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:55.259165+0000 front 2024-03-16T09:19:55.259123+0000 (oldest deadline 2024-03-16T09:20:18.759086+0000) 2024-03-16T09:20:26.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:26 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:26.479+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.660301+0000 front 2024-03-16T09:20:01.660147+0000 (oldest deadline 2024-03-16T09:20:26.360064+0000) 2024-03-16T09:20:26.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:26 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:26.710+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:26.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:26 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:26.710+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:26.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:26 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:26.710+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:26.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:26 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:26.710+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:26.903 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:26 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:26.498+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:26.903 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:26 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:26.498+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:26.903 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:26 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:26.498+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:42.977192+0000 front 2024-03-16T09:19:42.977205+0000 (oldest deadline 2024-03-16T09:20:08.877122+0000) 2024-03-16T09:20:26.903 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:26 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:26.498+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.178164+0000 front 2024-03-16T09:19:54.178126+0000 (oldest deadline 2024-03-16T09:20:18.877935+0000) 2024-03-16T09:20:27.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:26 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:26.830+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:27.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:26 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:26.830+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:27.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:26 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:26.830+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:27.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:26 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:26.830+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:27.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:26 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:26.830+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:27.743 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:27 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:27.484+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:30.874548+0000 front 2024-03-16T09:19:30.874517+0000 (oldest deadline 2024-03-16T09:19:53.174574+0000) 2024-03-16T09:20:27.743 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:27 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:27.484+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:37.176145+0000 front 2024-03-16T09:19:37.176238+0000 (oldest deadline 2024-03-16T09:20:02.476280+0000) 2024-03-16T09:20:27.743 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:27 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:27.484+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:42.977192+0000 front 2024-03-16T09:19:42.977205+0000 (oldest deadline 2024-03-16T09:20:08.877122+0000) 2024-03-16T09:20:27.743 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:27 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8[55495]: 2024-03-16T09:20:27.484+0000 7fb798345700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.178164+0000 front 2024-03-16T09:19:54.178126+0000 (oldest deadline 2024-03-16T09:20:18.877935+0000) 2024-03-16T09:20:27.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:27 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:27.450+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:27.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:27 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:27.450+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:27.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:27 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:27.450+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:27.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:27 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:27.450+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:27.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:27 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:27.450+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:55.259165+0000 front 2024-03-16T09:19:55.259123+0000 (oldest deadline 2024-03-16T09:20:18.759086+0000) 2024-03-16T09:20:27.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:27 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:27.450+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.660301+0000 front 2024-03-16T09:20:01.660147+0000 (oldest deadline 2024-03-16T09:20:26.360064+0000) 2024-03-16T09:20:28.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:27 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:27.741+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:28.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:27 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:27.741+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:28.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:27 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:27.741+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:28.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:27 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:27.741+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:28.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:27 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:27.839+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:28.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:27 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:27.839+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:28.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:27 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:27.839+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:28.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:27 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:27.839+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:28.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:27 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:27.839+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:28.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:27 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:27.839+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.124479+0000 front 2024-03-16T09:20:01.124530+0000 (oldest deadline 2024-03-16T09:20:27.024453+0000) 2024-03-16T09:20:28.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:28 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:28.404+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:28.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:28 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:28.404+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:28.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:28 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:28.404+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:28.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:28 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:28.404+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:28.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:28 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:28.404+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:55.259165+0000 front 2024-03-16T09:19:55.259123+0000 (oldest deadline 2024-03-16T09:20:18.759086+0000) 2024-03-16T09:20:28.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:28 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:28.404+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.660301+0000 front 2024-03-16T09:20:01.660147+0000 (oldest deadline 2024-03-16T09:20:26.360064+0000) 2024-03-16T09:20:29.102 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:28 smithi194 podman[72088]: 2024-03-16 09:20:28.819987375 +0000 UTC m=+5.486275827 container died 65ea366ef2798d9914dd0957a80779afa25f73f80855142b7499c9b0d3134540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-16T09:20:29.102 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:28 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:28.755+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:29.102 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:28 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:28.755+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:29.102 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:28 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:28.755+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:29.102 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:28 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:28.755+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:29.102 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:28 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:28.755+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:29.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:28 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:28.868+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:29.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:28 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:28.868+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:29.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:28 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:28.868+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:29.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:28 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:28.868+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:29.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:28 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:28.868+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:29.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:28 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:28.868+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.124479+0000 front 2024-03-16T09:20:01.124530+0000 (oldest deadline 2024-03-16T09:20:27.024453+0000) 2024-03-16T09:20:29.402 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:29 smithi194 podman[72088]: 2024-03-16 09:20:29.166081262 +0000 UTC m=+5.832369706 container cleanup 65ea366ef2798d9914dd0957a80779afa25f73f80855142b7499c9b0d3134540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-16T09:20:29.402 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:29 smithi194 bash[72088]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8 2024-03-16T09:20:29.778 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:29 smithi194 podman[72105]: 2024-03-16 09:20:29.425566878 +0000 UTC m=+0.604773181 container remove 65ea366ef2798d9914dd0957a80779afa25f73f80855142b7499c9b0d3134540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:20:29.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:29 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:29.405+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:29.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:29 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:29.406+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:29.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:29 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:29.406+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:29.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:29 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:29.406+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:29.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:29 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:29.406+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:55.259165+0000 front 2024-03-16T09:19:55.259123+0000 (oldest deadline 2024-03-16T09:20:18.759086+0000) 2024-03-16T09:20:29.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:29 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:29.406+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.660301+0000 front 2024-03-16T09:20:01.660147+0000 (oldest deadline 2024-03-16T09:20:26.360064+0000) 2024-03-16T09:20:30.105 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:29 smithi194 podman[72195]: 2024-03-16 09:20:29.679265063 +0000 UTC m=+0.017923520 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:20:30.106 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:29 smithi194 podman[72195]: 2024-03-16 09:20:29.893252183 +0000 UTC m=+0.231910634 container create 3e84d3473e38d63566066f7c6b2f1e2cb9c1e7aafe211e73a5cfd074d40c4f62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307) 2024-03-16T09:20:30.106 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:30 smithi194 podman[72195]: 2024-03-16 09:20:30.103745194 +0000 UTC m=+0.442403656 container init 3e84d3473e38d63566066f7c6b2f1e2cb9c1e7aafe211e73a5cfd074d40c4f62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-03-16T09:20:30.106 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:29 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:29.786+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:30.106 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:29 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:29.786+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:30.106 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:29 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:29.786+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:30.106 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:29 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:29.786+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:30.106 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:29 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:29.786+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:30.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:29 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:29.868+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:30.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:29 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:29.868+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:30.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:29 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:29.868+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:30.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:29 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:29.868+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:30.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:29 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:29.868+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:30.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:29 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:29.868+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.124479+0000 front 2024-03-16T09:20:01.124530+0000 (oldest deadline 2024-03-16T09:20:27.024453+0000) 2024-03-16T09:20:30.398 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:30 smithi194 podman[72195]: 2024-03-16 09:20:30.106957371 +0000 UTC m=+0.445615820 container start 3e84d3473e38d63566066f7c6b2f1e2cb9c1e7aafe211e73a5cfd074d40c4f62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307) 2024-03-16T09:20:30.399 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:30 smithi194 podman[72195]: 2024-03-16 09:20:30.19547379 +0000 UTC m=+0.534132262 container attach 3e84d3473e38d63566066f7c6b2f1e2cb9c1e7aafe211e73a5cfd074d40c4f62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:20:30.652 INFO:journalctl@ceph.osd.8.smithi194.stdout:Mar 16 09:20:30 smithi194 podman[72195]: 2024-03-16 09:20:30.397296953 +0000 UTC m=+0.735955433 container died 3e84d3473e38d63566066f7c6b2f1e2cb9c1e7aafe211e73a5cfd074d40c4f62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-16T09:20:30.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:30 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:30.386+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:30.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:30 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:30.387+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:30.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:30 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:30.387+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:30.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:30 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:30.387+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:30.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:30 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:30.387+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:55.259165+0000 front 2024-03-16T09:19:55.259123+0000 (oldest deadline 2024-03-16T09:20:18.759086+0000) 2024-03-16T09:20:30.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:30 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:30.387+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.660301+0000 front 2024-03-16T09:20:01.660147+0000 (oldest deadline 2024-03-16T09:20:26.360064+0000) 2024-03-16T09:20:30.993 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:30 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:30.788+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:30.993 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:30 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:30.788+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:30.993 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:30 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:30.788+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:30.993 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:30 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:30.788+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:30.993 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:30 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:30.788+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:31.017 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.8.service' 2024-03-16T09:20:31.062 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:20:31.063 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-16T09:20:31.063 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-16T09:20:31.063 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.9 2024-03-16T09:20:31.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:30 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:30.866+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:31.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:30 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:30.866+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:31.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:30 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:30.866+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:31.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:30 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:30.866+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:31.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:30 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:30.866+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:31.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:30 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:30.866+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.124479+0000 front 2024-03-16T09:20:01.124530+0000 (oldest deadline 2024-03-16T09:20:27.024453+0000) 2024-03-16T09:20:31.386 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:31 smithi112 systemd[1]: Stopping Ceph osd.9 for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:20:31.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:31 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:31.384+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:31.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:31 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:31.384+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:31.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:31 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:31.384+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:31.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:31 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:31.384+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:31.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:31 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:31.384+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:55.259165+0000 front 2024-03-16T09:19:55.259123+0000 (oldest deadline 2024-03-16T09:20:18.759086+0000) 2024-03-16T09:20:31.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:31 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:31.384+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.660301+0000 front 2024-03-16T09:20:01.660147+0000 (oldest deadline 2024-03-16T09:20:26.360064+0000) 2024-03-16T09:20:31.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:31 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:31.423+0000 7fb119e58700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:20:31.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:31 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:31.423+0000 7fb119e58700 -1 osd.9 91 *** Got signal Terminated *** 2024-03-16T09:20:31.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:31 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:31.423+0000 7fb119e58700 -1 osd.9 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:20:32.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:31 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:31.811+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:32.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:31 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:31.811+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:32.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:31 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:31.811+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:32.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:31 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:31.811+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:32.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:31 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:31.811+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:32.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:31 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:31.896+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:32.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:31 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:31.896+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:32.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:31 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:31.896+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:32.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:31 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:31.896+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:32.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:31 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:31.896+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:32.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:31 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:31.896+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.124479+0000 front 2024-03-16T09:20:01.124530+0000 (oldest deadline 2024-03-16T09:20:27.024453+0000) 2024-03-16T09:20:32.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:32 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:32.346+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:32.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:32 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:32.346+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:32.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:32 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:32.346+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:32.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:32 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:32.346+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:32.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:32 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:32.346+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:55.259165+0000 front 2024-03-16T09:19:55.259123+0000 (oldest deadline 2024-03-16T09:20:18.759086+0000) 2024-03-16T09:20:32.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:32 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:32.346+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.660301+0000 front 2024-03-16T09:20:01.660147+0000 (oldest deadline 2024-03-16T09:20:26.360064+0000) 2024-03-16T09:20:33.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:32 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:32.857+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:33.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:32 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:32.857+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:33.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:32 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:32.857+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:33.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:32 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:32.857+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:33.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:32 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:32.857+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:33.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:32 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:32.886+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:33.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:32 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:32.886+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:33.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:32 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:32.886+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:33.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:32 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:32.886+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:33.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:32 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:32.886+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:33.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:32 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:32.886+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.124479+0000 front 2024-03-16T09:20:01.124530+0000 (oldest deadline 2024-03-16T09:20:27.024453+0000) 2024-03-16T09:20:33.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:33 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:33.373+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:33.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:33 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:33.373+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:33.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:33 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:33.373+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:33.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:33 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:33.373+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:33.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:33 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:33.373+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:55.259165+0000 front 2024-03-16T09:19:55.259123+0000 (oldest deadline 2024-03-16T09:20:18.759086+0000) 2024-03-16T09:20:33.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:33 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:33.373+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.660301+0000 front 2024-03-16T09:20:01.660147+0000 (oldest deadline 2024-03-16T09:20:26.360064+0000) 2024-03-16T09:20:34.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:33 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:33.864+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:34.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:33 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:33.864+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:34.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:33 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:33.864+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:34.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:33 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:33.864+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:34.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:33 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:33.864+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:34.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:33 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:33.864+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:10.700745+0000 front 2024-03-16T09:20:10.700791+0000 (oldest deadline 2024-03-16T09:20:33.000763+0000) 2024-03-16T09:20:34.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:33 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:33.838+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:34.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:33 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:33.838+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:34.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:33 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:33.838+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:34.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:33 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:33.838+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:34.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:33 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:33.838+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:34.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:33 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:33.838+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.124479+0000 front 2024-03-16T09:20:01.124530+0000 (oldest deadline 2024-03-16T09:20:27.024453+0000) 2024-03-16T09:20:34.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:34 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:34.366+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:34.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:34 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:34.366+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:34.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:34 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:34.366+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:34.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:34 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:34.366+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:34.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:34 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:34.366+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:55.259165+0000 front 2024-03-16T09:19:55.259123+0000 (oldest deadline 2024-03-16T09:20:18.759086+0000) 2024-03-16T09:20:34.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:34 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:34.366+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.660301+0000 front 2024-03-16T09:20:01.660147+0000 (oldest deadline 2024-03-16T09:20:26.360064+0000) 2024-03-16T09:20:35.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:34 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:34.876+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:35.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:34 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:34.876+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:35.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:34 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:34.876+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:35.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:34 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:34.876+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:35.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:34 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:34.876+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:35.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:34 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:34.876+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:10.700745+0000 front 2024-03-16T09:20:10.700791+0000 (oldest deadline 2024-03-16T09:20:33.000763+0000) 2024-03-16T09:20:35.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:34 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:34.862+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:35.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:34 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:34.862+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:35.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:34 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:34.862+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:35.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:34 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:34.862+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:35.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:34 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:34.862+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:35.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:34 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:34.862+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.124479+0000 front 2024-03-16T09:20:01.124530+0000 (oldest deadline 2024-03-16T09:20:27.024453+0000) 2024-03-16T09:20:35.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:34 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:34.862+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:09.325528+0000 front 2024-03-16T09:20:09.325565+0000 (oldest deadline 2024-03-16T09:20:34.025457+0000) 2024-03-16T09:20:35.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:35 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:35.360+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:35.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:35 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:35.360+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:35.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:35 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:35.360+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:35.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:35 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:35.360+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:35.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:35 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:35.360+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:55.259165+0000 front 2024-03-16T09:19:55.259123+0000 (oldest deadline 2024-03-16T09:20:18.759086+0000) 2024-03-16T09:20:35.843 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:35 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:35.360+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.660301+0000 front 2024-03-16T09:20:01.660147+0000 (oldest deadline 2024-03-16T09:20:26.360064+0000) 2024-03-16T09:20:36.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:35 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:35.893+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:36.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:35 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:35.893+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:36.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:35 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:35.893+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:36.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:35 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:35.893+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:36.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:35 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:35.893+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:36.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:35 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:35.893+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:10.700745+0000 front 2024-03-16T09:20:10.700791+0000 (oldest deadline 2024-03-16T09:20:33.000763+0000) 2024-03-16T09:20:36.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:35 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:35.848+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:36.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:35 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:35.848+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:36.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:35 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:35.848+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:36.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:35 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:35.848+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:36.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:35 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:35.848+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:36.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:35 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:35.848+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.124479+0000 front 2024-03-16T09:20:01.124530+0000 (oldest deadline 2024-03-16T09:20:27.024453+0000) 2024-03-16T09:20:36.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:35 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:35.848+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:09.325528+0000 front 2024-03-16T09:20:09.325565+0000 (oldest deadline 2024-03-16T09:20:34.025457+0000) 2024-03-16T09:20:36.592 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:36 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:36.319+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.553232+0000 front 2024-03-16T09:19:21.553206+0000 (oldest deadline 2024-03-16T09:19:46.153986+0000) 2024-03-16T09:20:36.592 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:36 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:36.319+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.455388+0000 front 2024-03-16T09:19:31.455493+0000 (oldest deadline 2024-03-16T09:19:53.755155+0000) 2024-03-16T09:20:36.592 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:36 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:36.319+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:39.056805+0000 front 2024-03-16T09:19:39.056843+0000 (oldest deadline 2024-03-16T09:20:03.156191+0000) 2024-03-16T09:20:36.592 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:36 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:36.319+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:47.157634+0000 front 2024-03-16T09:19:47.157697+0000 (oldest deadline 2024-03-16T09:20:08.857778+0000) 2024-03-16T09:20:36.593 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:36 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:36.319+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:55.259165+0000 front 2024-03-16T09:19:55.259123+0000 (oldest deadline 2024-03-16T09:20:18.759086+0000) 2024-03-16T09:20:36.593 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:36 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9[67469]: 2024-03-16T09:20:36.319+0000 7fb11644c700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.660301+0000 front 2024-03-16T09:20:01.660147+0000 (oldest deadline 2024-03-16T09:20:26.360064+0000) 2024-03-16T09:20:37.076 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:36 smithi112 podman[91189]: 2024-03-16 09:20:36.7994583 +0000 UTC m=+5.546296013 container died 30e2ed2f1586406e23f9bb06e6de94aea070ccb01ad5ebf54e99cb351e9bcf72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS) 2024-03-16T09:20:37.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:36 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:36.861+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:37.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:36 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:36.861+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:37.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:36 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:36.861+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:37.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:36 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:36.861+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:37.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:36 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:36.861+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:37.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:36 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:36.861+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.124479+0000 front 2024-03-16T09:20:01.124530+0000 (oldest deadline 2024-03-16T09:20:27.024453+0000) 2024-03-16T09:20:37.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:36 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:36.861+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:09.325528+0000 front 2024-03-16T09:20:09.325565+0000 (oldest deadline 2024-03-16T09:20:34.025457+0000) 2024-03-16T09:20:37.342 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:37 smithi112 podman[91189]: 2024-03-16 09:20:37.248274809 +0000 UTC m=+5.995112528 container cleanup 30e2ed2f1586406e23f9bb06e6de94aea070ccb01ad5ebf54e99cb351e9bcf72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:20:37.342 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:37 smithi112 bash[91189]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9 2024-03-16T09:20:37.402 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:36 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:36.921+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:37.402 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:36 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:36.921+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:37.402 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:36 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:36.921+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:37.402 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:36 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:36.921+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:37.403 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:36 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:36.921+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:37.403 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:36 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:36.921+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:10.700745+0000 front 2024-03-16T09:20:10.700791+0000 (oldest deadline 2024-03-16T09:20:33.000763+0000) 2024-03-16T09:20:37.842 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:37 smithi112 podman[91207]: 2024-03-16 09:20:37.541330817 +0000 UTC m=+0.741444002 container remove 30e2ed2f1586406e23f9bb06e6de94aea070ccb01ad5ebf54e99cb351e9bcf72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:20:38.112 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:37 smithi112 podman[91299]: 2024-03-16 09:20:37.763200428 +0000 UTC m=+0.017617397 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:20:38.113 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:37 smithi112 podman[91299]: 2024-03-16 09:20:37.867281665 +0000 UTC m=+0.121698616 container create 1ee30798456b3c862be94bbb5574dd0c503372c43cb452d684026dc1b03ee751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-16T09:20:38.113 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:38 smithi112 podman[91299]: 2024-03-16 09:20:38.111095365 +0000 UTC m=+0.365512330 container init 1ee30798456b3c862be94bbb5574dd0c503372c43cb452d684026dc1b03ee751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-16T09:20:38.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:37 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:37.886+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:38.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:37 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:37.886+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:38.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:37 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:37.886+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:38.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:37 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:37.886+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:38.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:37 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:37.886+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:38.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:37 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:37.886+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:10.700745+0000 front 2024-03-16T09:20:10.700791+0000 (oldest deadline 2024-03-16T09:20:33.000763+0000) 2024-03-16T09:20:38.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:37 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:37.910+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:38.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:37 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:37.910+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:38.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:37 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:37.910+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:38.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:37 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:37.910+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:38.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:37 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:37.910+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:38.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:37 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:37.910+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.124479+0000 front 2024-03-16T09:20:01.124530+0000 (oldest deadline 2024-03-16T09:20:27.024453+0000) 2024-03-16T09:20:38.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:37 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:37.910+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:09.325528+0000 front 2024-03-16T09:20:09.325565+0000 (oldest deadline 2024-03-16T09:20:34.025457+0000) 2024-03-16T09:20:38.383 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:38 smithi112 podman[91299]: 2024-03-16 09:20:38.114277196 +0000 UTC m=+0.368694161 container start 1ee30798456b3c862be94bbb5574dd0c503372c43cb452d684026dc1b03ee751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:20:38.383 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:38 smithi112 podman[91299]: 2024-03-16 09:20:38.194711296 +0000 UTC m=+0.449128253 container attach 1ee30798456b3c862be94bbb5574dd0c503372c43cb452d684026dc1b03ee751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9-deactivate, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-16T09:20:38.708 INFO:journalctl@ceph.osd.9.smithi112.stdout:Mar 16 09:20:38 smithi112 podman[91299]: 2024-03-16 09:20:38.416617986 +0000 UTC m=+0.671034943 container died 1ee30798456b3c862be94bbb5574dd0c503372c43cb452d684026dc1b03ee751 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-9-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:20:38.975 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.9.service' 2024-03-16T09:20:39.015 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:20:39.016 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-16T09:20:39.016 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-16T09:20:39.016 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.10 2024-03-16T09:20:39.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:38 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:38.868+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:39.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:38 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:38.868+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:39.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:38 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:38.868+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:39.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:38 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:38.868+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:39.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:38 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:38.868+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:39.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:38 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:38.868+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:10.700745+0000 front 2024-03-16T09:20:10.700791+0000 (oldest deadline 2024-03-16T09:20:33.000763+0000) 2024-03-16T09:20:39.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:38 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:38.863+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:39.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:38 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:38.863+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:39.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:38 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:38.863+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:39.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:38 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:38.863+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:39.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:38 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:38.863+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:39.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:38 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:38.863+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.124479+0000 front 2024-03-16T09:20:01.124530+0000 (oldest deadline 2024-03-16T09:20:27.024453+0000) 2024-03-16T09:20:39.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:38 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:38.863+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:09.325528+0000 front 2024-03-16T09:20:09.325565+0000 (oldest deadline 2024-03-16T09:20:34.025457+0000) 2024-03-16T09:20:39.238 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:39 smithi159 systemd[1]: Stopping Ceph osd.10 for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:20:39.736 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:39 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:39.308+0000 7fe44502c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:20:39.737 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:39 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:39.308+0000 7fe44502c700 -1 osd.10 91 *** Got signal Terminated *** 2024-03-16T09:20:39.737 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:39 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:39.308+0000 7fe44502c700 -1 osd.10 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:20:40.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:39 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:39.828+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:40.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:39 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:39.828+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:40.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:39 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:39.828+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:40.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:39 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:39.828+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:40.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:39 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:39.828+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:40.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:39 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:39.828+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:10.700745+0000 front 2024-03-16T09:20:10.700791+0000 (oldest deadline 2024-03-16T09:20:33.000763+0000) 2024-03-16T09:20:40.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:39 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:39.833+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:40.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:39 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:39.833+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:40.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:39 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:39.833+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:40.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:39 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:39.833+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:40.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:39 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:39.833+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:40.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:39 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:39.833+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.124479+0000 front 2024-03-16T09:20:01.124530+0000 (oldest deadline 2024-03-16T09:20:27.024453+0000) 2024-03-16T09:20:40.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:39 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:39.833+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:09.325528+0000 front 2024-03-16T09:20:09.325565+0000 (oldest deadline 2024-03-16T09:20:34.025457+0000) 2024-03-16T09:20:41.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:40 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:40.824+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:41.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:40 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:40.824+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:41.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:40 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:40.824+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:41.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:40 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:40.824+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:41.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:40 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:40.824+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:41.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:40 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:40.824+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:10.700745+0000 front 2024-03-16T09:20:10.700791+0000 (oldest deadline 2024-03-16T09:20:33.000763+0000) 2024-03-16T09:20:41.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:40 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:40.784+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:41.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:40 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:40.784+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:41.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:40 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:40.784+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:41.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:40 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:40.784+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:41.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:40 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:40.784+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:41.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:40 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:40.784+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.124479+0000 front 2024-03-16T09:20:01.124530+0000 (oldest deadline 2024-03-16T09:20:27.024453+0000) 2024-03-16T09:20:41.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:40 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:40.784+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:09.325528+0000 front 2024-03-16T09:20:09.325565+0000 (oldest deadline 2024-03-16T09:20:34.025457+0000) 2024-03-16T09:20:42.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:41 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:41.789+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:42.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:41 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:41.789+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:42.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:41 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:41.789+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:42.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:41 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:41.789+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:42.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:41 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:41.789+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:42.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:41 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:41.789+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:10.700745+0000 front 2024-03-16T09:20:10.700791+0000 (oldest deadline 2024-03-16T09:20:33.000763+0000) 2024-03-16T09:20:42.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:41 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:41.794+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:42.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:41 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:41.794+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:42.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:41 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:41.794+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:42.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:41 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:41.794+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:42.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:41 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:41.794+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:42.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:41 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:41.794+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.124479+0000 front 2024-03-16T09:20:01.124530+0000 (oldest deadline 2024-03-16T09:20:27.024453+0000) 2024-03-16T09:20:42.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:41 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:41.794+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:09.325528+0000 front 2024-03-16T09:20:09.325565+0000 (oldest deadline 2024-03-16T09:20:34.025457+0000) 2024-03-16T09:20:42.756 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:42 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:42.753+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:42.757 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:42 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:42.753+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:42.757 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:42 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:42.753+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:42.757 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:42 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:42.753+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:42.757 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:42 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:42.753+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:42.757 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:42 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:42.753+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.124479+0000 front 2024-03-16T09:20:01.124530+0000 (oldest deadline 2024-03-16T09:20:27.024453+0000) 2024-03-16T09:20:42.757 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:42 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:42.753+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:09.325528+0000 front 2024-03-16T09:20:09.325565+0000 (oldest deadline 2024-03-16T09:20:34.025457+0000) 2024-03-16T09:20:43.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:42 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:42.783+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:43.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:42 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:42.783+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:43.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:42 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:42.783+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:43.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:42 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:42.783+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:43.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:42 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:42.783+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:43.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:42 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:42.783+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:10.700745+0000 front 2024-03-16T09:20:10.700791+0000 (oldest deadline 2024-03-16T09:20:33.000763+0000) 2024-03-16T09:20:44.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:43 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:43.828+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:44.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:43 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:43.828+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:44.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:43 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:43.828+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:44.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:43 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:43.828+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:44.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:43 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:43.828+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:44.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:43 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:43.828+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:10.700745+0000 front 2024-03-16T09:20:10.700791+0000 (oldest deadline 2024-03-16T09:20:33.000763+0000) 2024-03-16T09:20:44.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:43 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:43.790+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:21.216754+0000 front 2024-03-16T09:19:21.216915+0000 (oldest deadline 2024-03-16T09:19:45.316671+0000) 2024-03-16T09:20:44.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:43 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:43.790+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:28.817792+0000 front 2024-03-16T09:19:28.817821+0000 (oldest deadline 2024-03-16T09:19:53.517956+0000) 2024-03-16T09:20:44.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:43 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:43.790+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:36.719995+0000 front 2024-03-16T09:19:36.720177+0000 (oldest deadline 2024-03-16T09:20:02.520714+0000) 2024-03-16T09:20:44.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:43 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:43.790+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-03-16T09:19:44.221829+0000 front 2024-03-16T09:19:44.221733+0000 (oldest deadline 2024-03-16T09:20:11.822447+0000) 2024-03-16T09:20:44.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:43 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:43.790+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:54.123452+0000 front 2024-03-16T09:19:54.123434+0000 (oldest deadline 2024-03-16T09:20:18.223490+0000) 2024-03-16T09:20:44.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:43 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:43.790+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:01.124479+0000 front 2024-03-16T09:20:01.124530+0000 (oldest deadline 2024-03-16T09:20:27.024453+0000) 2024-03-16T09:20:44.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:43 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10[61265]: 2024-03-16T09:20:43.790+0000 7fe441620700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:09.325528+0000 front 2024-03-16T09:20:09.325565+0000 (oldest deadline 2024-03-16T09:20:34.025457+0000) 2024-03-16T09:20:44.928 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:44 smithi159 podman[70442]: 2024-03-16 09:20:44.645612959 +0000 UTC m=+5.439526632 container died 6a402381ea0efc93a740ccd30fc465d533264c45743f03f18a62998758b4009e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-16T09:20:45.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:44 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:44.831+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:45.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:44 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:44.831+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:45.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:44 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:44.831+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:45.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:44 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:44.831+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:45.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:44 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:44.831+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:45.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:44 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:44.831+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:10.700745+0000 front 2024-03-16T09:20:10.700791+0000 (oldest deadline 2024-03-16T09:20:33.000763+0000) 2024-03-16T09:20:45.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:44 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:44.831+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-16T09:20:18.702527+0000 front 2024-03-16T09:20:18.702565+0000 (oldest deadline 2024-03-16T09:20:44.602461+0000) 2024-03-16T09:20:45.236 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:45 smithi159 podman[70442]: 2024-03-16 09:20:45.025087114 +0000 UTC m=+5.819000785 container cleanup 6a402381ea0efc93a740ccd30fc465d533264c45743f03f18a62998758b4009e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-16T09:20:45.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:45 smithi159 bash[70442]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10 2024-03-16T09:20:45.556 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:45 smithi159 podman[70460]: 2024-03-16 09:20:45.259503675 +0000 UTC m=+0.611401882 container remove 6a402381ea0efc93a740ccd30fc465d533264c45743f03f18a62998758b4009e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-16T09:20:45.822 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:45 smithi159 podman[70552]: 2024-03-16 09:20:45.458184945 +0000 UTC m=+0.017726317 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:20:45.822 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:45 smithi159 podman[70552]: 2024-03-16 09:20:45.568546524 +0000 UTC m=+0.128087883 container create 831187255b227330762593e48d7e089e1cfd9d2f56e2c46f5e351f51a5cec1fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10-deactivate, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux ) 2024-03-16T09:20:45.823 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:45 smithi159 podman[70552]: 2024-03-16 09:20:45.754212295 +0000 UTC m=+0.313753655 container init 831187255b227330762593e48d7e089e1cfd9d2f56e2c46f5e351f51a5cec1fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-16T09:20:45.823 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:45 smithi159 podman[70552]: 2024-03-16 09:20:45.757335014 +0000 UTC m=+0.316876368 container start 831187255b227330762593e48d7e089e1cfd9d2f56e2c46f5e351f51a5cec1fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-16T09:20:45.823 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:45 smithi159 podman[70552]: 2024-03-16 09:20:45.820770306 +0000 UTC m=+0.380311685 container attach 831187255b227330762593e48d7e089e1cfd9d2f56e2c46f5e351f51a5cec1fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-16T09:20:46.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:45 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:45.860+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:46.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:45 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:45.860+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:46.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:45 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:45.860+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:46.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:45 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:45.860+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:46.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:45 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:45.860+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:46.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:45 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:45.860+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:10.700745+0000 front 2024-03-16T09:20:10.700791+0000 (oldest deadline 2024-03-16T09:20:33.000763+0000) 2024-03-16T09:20:46.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:45 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:45.860+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-16T09:20:18.702527+0000 front 2024-03-16T09:20:18.702565+0000 (oldest deadline 2024-03-16T09:20:44.602461+0000) 2024-03-16T09:20:46.237 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:46 smithi159 podman[70552]: 2024-03-16 09:20:46.052204631 +0000 UTC m=+0.611745998 container died 831187255b227330762593e48d7e089e1cfd9d2f56e2c46f5e351f51a5cec1fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-16T09:20:46.591 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 16 09:20:46 smithi159 podman[70579]: 2024-03-16 09:20:46.573954365 +0000 UTC m=+0.516105536 container remove 831187255b227330762593e48d7e089e1cfd9d2f56e2c46f5e351f51a5cec1fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-10-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-16T09:20:46.596 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.10.service' 2024-03-16T09:20:46.637 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:20:46.637 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-16T09:20:46.637 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-16T09:20:46.637 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.11 2024-03-16T09:20:46.947 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:46 smithi194 systemd[1]: Stopping Ceph osd.11 for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:20:46.947 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:46 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:46.838+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:46.947 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:46 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:46.838+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:46.947 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:46 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:46.838+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:46.948 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:46 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:46.838+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:46.948 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:46 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:46.838+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:46.948 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:46 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:46.838+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:10.700745+0000 front 2024-03-16T09:20:10.700791+0000 (oldest deadline 2024-03-16T09:20:33.000763+0000) 2024-03-16T09:20:46.948 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:46 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:46.838+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-16T09:20:18.702527+0000 front 2024-03-16T09:20:18.702565+0000 (oldest deadline 2024-03-16T09:20:44.602461+0000) 2024-03-16T09:20:47.402 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:46 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:46.942+0000 7f2893b8a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:20:47.402 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:46 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:46.942+0000 7f2893b8a700 -1 osd.11 91 *** Got signal Terminated *** 2024-03-16T09:20:47.402 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:46 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:46.942+0000 7f2893b8a700 -1 osd.11 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-16T09:20:48.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:47 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:47.849+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:48.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:47 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:47.849+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:48.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:47 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:47.849+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:48.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:47 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:47.849+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:48.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:47 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:47.849+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:48.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:47 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:47.849+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:10.700745+0000 front 2024-03-16T09:20:10.700791+0000 (oldest deadline 2024-03-16T09:20:33.000763+0000) 2024-03-16T09:20:48.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:47 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:47.849+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-16T09:20:18.702527+0000 front 2024-03-16T09:20:18.702565+0000 (oldest deadline 2024-03-16T09:20:44.602461+0000) 2024-03-16T09:20:49.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:48 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:48.824+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:49.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:48 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:48.824+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:49.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:48 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:48.824+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:49.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:48 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:48.824+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:49.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:48 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:48.824+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:49.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:48 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:48.824+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:10.700745+0000 front 2024-03-16T09:20:10.700791+0000 (oldest deadline 2024-03-16T09:20:33.000763+0000) 2024-03-16T09:20:49.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:48 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:48.824+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-16T09:20:18.702527+0000 front 2024-03-16T09:20:18.702565+0000 (oldest deadline 2024-03-16T09:20:44.602461+0000) 2024-03-16T09:20:50.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:49 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:49.797+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:50.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:49 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:49.797+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:50.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:49 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:49.797+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:50.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:49 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:49.797+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:50.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:49 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:49.797+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:50.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:49 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:49.797+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:10.700745+0000 front 2024-03-16T09:20:10.700791+0000 (oldest deadline 2024-03-16T09:20:33.000763+0000) 2024-03-16T09:20:50.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:49 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:49.797+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-16T09:20:18.702527+0000 front 2024-03-16T09:20:18.702565+0000 (oldest deadline 2024-03-16T09:20:44.602461+0000) 2024-03-16T09:20:50.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:49 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:49.797+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6820 osd.8 since back 2024-03-16T09:20:24.602862+0000 front 2024-03-16T09:20:24.602840+0000 (oldest deadline 2024-03-16T09:20:49.302827+0000) 2024-03-16T09:20:51.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:50 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:50.764+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:51.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:50 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:50.764+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:51.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:50 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:50.764+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:51.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:50 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:50.764+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:51.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:50 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:50.764+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:51.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:50 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:50.764+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:10.700745+0000 front 2024-03-16T09:20:10.700791+0000 (oldest deadline 2024-03-16T09:20:33.000763+0000) 2024-03-16T09:20:51.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:50 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:50.764+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-16T09:20:18.702527+0000 front 2024-03-16T09:20:18.702565+0000 (oldest deadline 2024-03-16T09:20:44.602461+0000) 2024-03-16T09:20:51.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:50 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:50.764+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6820 osd.8 since back 2024-03-16T09:20:24.602862+0000 front 2024-03-16T09:20:24.602840+0000 (oldest deadline 2024-03-16T09:20:49.302827+0000) 2024-03-16T09:20:52.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:51 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:51.735+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-03-16T09:19:20.893806+0000 front 2024-03-16T09:19:20.893734+0000 (oldest deadline 2024-03-16T09:19:46.793687+0000) 2024-03-16T09:20:52.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:51 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:51.735+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-16T09:19:31.495029+0000 front 2024-03-16T09:19:31.494900+0000 (oldest deadline 2024-03-16T09:19:55.595086+0000) 2024-03-16T09:20:52.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:51 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:51.735+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-03-16T09:19:35.595487+0000 front 2024-03-16T09:19:35.595515+0000 (oldest deadline 2024-03-16T09:20:01.495633+0000) 2024-03-16T09:20:52.152 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:51 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:51.735+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-16T09:19:53.197836+0000 front 2024-03-16T09:19:53.198025+0000 (oldest deadline 2024-03-16T09:20:18.497916+0000) 2024-03-16T09:20:52.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:51 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:51.735+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-03-16T09:20:02.499378+0000 front 2024-03-16T09:20:02.499354+0000 (oldest deadline 2024-03-16T09:20:28.399441+0000) 2024-03-16T09:20:52.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:51 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:51.735+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-03-16T09:20:10.700745+0000 front 2024-03-16T09:20:10.700791+0000 (oldest deadline 2024-03-16T09:20:33.000763+0000) 2024-03-16T09:20:52.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:51 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:51.735+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-16T09:20:18.702527+0000 front 2024-03-16T09:20:18.702565+0000 (oldest deadline 2024-03-16T09:20:44.602461+0000) 2024-03-16T09:20:52.153 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:51 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11[62426]: 2024-03-16T09:20:51.735+0000 7f289017e700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.194:6820 osd.8 since back 2024-03-16T09:20:24.602862+0000 front 2024-03-16T09:20:24.602840+0000 (oldest deadline 2024-03-16T09:20:49.302827+0000) 2024-03-16T09:20:52.561 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:52 smithi194 podman[72314]: 2024-03-16 09:20:52.302060893 +0000 UTC m=+5.472301139 container died 3e2a934bfd053f5f49465abae69ce2915950caadbe7e8640d3060c5d0b3af49f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-16T09:20:52.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:52 smithi194 podman[72314]: 2024-03-16 09:20:52.659234352 +0000 UTC m=+5.829474587 container cleanup 3e2a934bfd053f5f49465abae69ce2915950caadbe7e8640d3060c5d0b3af49f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-03-16T09:20:52.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:52 smithi194 bash[72314]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11 2024-03-16T09:20:53.272 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:52 smithi194 podman[72343]: 2024-03-16 09:20:52.944197195 +0000 UTC m=+0.640726384 container remove 3e2a934bfd053f5f49465abae69ce2915950caadbe7e8640d3060c5d0b3af49f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5) 2024-03-16T09:20:53.601 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:53 smithi194 podman[72434]: 2024-03-16 09:20:53.270767292 +0000 UTC m=+0.106645316 container create d3b063c9ef00d7b9cdf852234bee99415a98ed9fc85181ed727344b91fc6c45d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-16T09:20:53.601 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:53 smithi194 podman[72434]: 2024-03-16 09:20:53.181360689 +0000 UTC m=+0.017238721 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-16T09:20:53.601 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:53 smithi194 podman[72434]: 2024-03-16 09:20:53.516223968 +0000 UTC m=+0.352101990 container init d3b063c9ef00d7b9cdf852234bee99415a98ed9fc85181ed727344b91fc6c45d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-16T09:20:53.601 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:53 smithi194 podman[72434]: 2024-03-16 09:20:53.519334795 +0000 UTC m=+0.355212827 container start d3b063c9ef00d7b9cdf852234bee99415a98ed9fc85181ed727344b91fc6c45d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-16T09:20:53.601 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:53 smithi194 podman[72434]: 2024-03-16 09:20:53.599267494 +0000 UTC m=+0.435145539 container attach d3b063c9ef00d7b9cdf852234bee99415a98ed9fc85181ed727344b91fc6c45d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-16T09:20:53.902 INFO:journalctl@ceph.osd.11.smithi194.stdout:Mar 16 09:20:53 smithi194 podman[72434]: 2024-03-16 09:20:53.809524631 +0000 UTC m=+0.645402658 container died d3b063c9ef00d7b9cdf852234bee99415a98ed9fc85181ed727344b91fc6c45d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-16T09:20:54.475 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@osd.11.service' 2024-03-16T09:20:54.517 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:20:54.517 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-16T09:20:54.517 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-16T09:20:54.517 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.a 2024-03-16T09:20:54.811 INFO:journalctl@ceph.mds.a.smithi112.stdout:Mar 16 09:20:54 smithi112 systemd[1]: Stopping Ceph mds.a for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:20:55.092 INFO:journalctl@ceph.mds.a.smithi112.stdout:Mar 16 09:20:54 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-a[74132]: 2024-03-16T09:20:54.807+0000 7f0090c2d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:20:55.092 INFO:journalctl@ceph.mds.a.smithi112.stdout:Mar 16 09:20:54 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-a[74132]: 2024-03-16T09:20:54.807+0000 7f0090c2d700 -1 mds.a *** got signal Terminated *** 2024-03-16T09:20:56.694 INFO:journalctl@ceph.mds.a.smithi112.stdout:Mar 16 09:20:56 smithi112 podman[91415]: 2024-03-16 09:20:56.421250151 +0000 UTC m=+1.709061618 container died 5b2e1e3e3a94d17fdb14bc25c39a24d53fdbbb282daecd0aff0db78b00a3465a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-16T09:20:57.039 INFO:journalctl@ceph.mds.a.smithi112.stdout:Mar 16 09:20:56 smithi112 podman[91415]: 2024-03-16 09:20:56.791730225 +0000 UTC m=+2.079541686 container cleanup 5b2e1e3e3a94d17fdb14bc25c39a24d53fdbbb282daecd0aff0db78b00a3465a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-a, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-16T09:20:57.039 INFO:journalctl@ceph.mds.a.smithi112.stdout:Mar 16 09:20:56 smithi112 bash[91415]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-a 2024-03-16T09:20:57.183 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.a.service' 2024-03-16T09:20:57.244 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:20:57.245 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-16T09:20:57.245 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-16T09:20:57.245 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.d 2024-03-16T09:20:57.573 INFO:journalctl@ceph.mds.d.smithi112.stdout:Mar 16 09:20:57 smithi112 systemd[1]: Stopping Ceph mds.d for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:20:57.842 INFO:journalctl@ceph.mds.d.smithi112.stdout:Mar 16 09:20:57 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-d[73169]: 2024-03-16T09:20:57.570+0000 7f4f6a193700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:20:57.842 INFO:journalctl@ceph.mds.d.smithi112.stdout:Mar 16 09:20:57 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-d[73169]: 2024-03-16T09:20:57.570+0000 7f4f6a193700 -1 mds.d *** got signal Terminated *** 2024-03-16T09:21:00.581 INFO:journalctl@ceph.mds.d.smithi112.stdout:Mar 16 09:21:00 smithi112 podman[91546]: 2024-03-16 09:21:00.307993075 +0000 UTC m=+2.840361724 container died efe476354faaaa9f913e9952b493f7925ef6970ff08f4f2998e6d441ac0dce88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-d, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:21:00.842 INFO:journalctl@ceph.mds.d.smithi112.stdout:Mar 16 09:21:00 smithi112 podman[91546]: 2024-03-16 09:21:00.662337345 +0000 UTC m=+3.194706003 container cleanup efe476354faaaa9f913e9952b493f7925ef6970ff08f4f2998e6d441ac0dce88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-16T09:21:00.843 INFO:journalctl@ceph.mds.d.smithi112.stdout:Mar 16 09:21:00 smithi112 bash[91546]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-d 2024-03-16T09:21:01.088 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.d.service' 2024-03-16T09:21:01.124 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:21:01.125 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-16T09:21:01.125 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-16T09:21:01.125 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.g 2024-03-16T09:21:01.592 INFO:journalctl@ceph.mds.g.smithi112.stdout:Mar 16 09:21:01 smithi112 systemd[1]: Stopping Ceph mds.g for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:21:01.592 INFO:journalctl@ceph.mds.g.smithi112.stdout:Mar 16 09:21:01 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-g[73784]: 2024-03-16T09:21:01.447+0000 7f79d62a0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:21:01.592 INFO:journalctl@ceph.mds.g.smithi112.stdout:Mar 16 09:21:01 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-g[73784]: 2024-03-16T09:21:01.447+0000 7f79d62a0700 -1 mds.g *** got signal Terminated *** 2024-03-16T09:21:05.751 INFO:journalctl@ceph.mds.g.smithi112.stdout:Mar 16 09:21:05 smithi112 podman[91666]: 2024-03-16 09:21:05.45509196 +0000 UTC m=+4.111588693 container died 32db2fe1a8052831f49e25713b419c1af5d8644a57feedd81a247ee4e6a71006 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-g, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-16T09:21:06.077 INFO:journalctl@ceph.mds.g.smithi112.stdout:Mar 16 09:21:05 smithi112 podman[91666]: 2024-03-16 09:21:05.840639367 +0000 UTC m=+4.497136099 container cleanup 32db2fe1a8052831f49e25713b419c1af5d8644a57feedd81a247ee4e6a71006 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-g, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-16T09:21:06.077 INFO:journalctl@ceph.mds.g.smithi112.stdout:Mar 16 09:21:05 smithi112 bash[91666]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-g 2024-03-16T09:21:06.179 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.g.service' 2024-03-16T09:21:06.215 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:21:06.215 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-16T09:21:06.215 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-16T09:21:06.215 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.j 2024-03-16T09:21:06.342 INFO:journalctl@ceph.mds.j.smithi112.stdout:Mar 16 09:21:06 smithi112 systemd[1]: Stopping Ceph mds.j for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:21:06.842 INFO:journalctl@ceph.mds.j.smithi112.stdout:Mar 16 09:21:06 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-j[72822]: 2024-03-16T09:21:06.527+0000 7f58265ba700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:21:06.842 INFO:journalctl@ceph.mds.j.smithi112.stdout:Mar 16 09:21:06 smithi112 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-j[72822]: 2024-03-16T09:21:06.527+0000 7f58265ba700 -1 mds.j *** got signal Terminated *** 2024-03-16T09:21:09.721 INFO:journalctl@ceph.mds.j.smithi112.stdout:Mar 16 09:21:09 smithi112 podman[91788]: 2024-03-16 09:21:09.425098681 +0000 UTC m=+2.996942796 container died ed78459aba8d1370934756262d67207e8c3d631996620a3b38bddb406060e0d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-j, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:21:09.721 INFO:journalctl@ceph.mds.j.smithi112.stdout:Mar 16 09:21:09 smithi112 podman[91788]: 2024-03-16 09:21:09.719703991 +0000 UTC m=+3.291548112 container cleanup ed78459aba8d1370934756262d67207e8c3d631996620a3b38bddb406060e0d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-j, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=) 2024-03-16T09:21:10.049 INFO:journalctl@ceph.mds.j.smithi112.stdout:Mar 16 09:21:09 smithi112 bash[91788]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-j 2024-03-16T09:21:10.049 INFO:journalctl@ceph.mds.j.smithi112.stdout:Mar 16 09:21:09 smithi112 podman[91802]: 2024-03-16 09:21:09.920723479 +0000 UTC m=+0.494008506 container remove ed78459aba8d1370934756262d67207e8c3d631996620a3b38bddb406060e0d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-j, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5) 2024-03-16T09:21:10.063 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.j.service' 2024-03-16T09:21:10.098 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:21:10.098 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-16T09:21:10.098 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-16T09:21:10.098 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.b 2024-03-16T09:21:10.385 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:21:10 smithi159 systemd[1]: Stopping Ceph mds.b for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:21:10.644 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:21:10 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-b[66504]: 2024-03-16T09:21:10.382+0000 7f0d4a0c9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:21:10.644 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:21:10 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-b[66504]: 2024-03-16T09:21:10.382+0000 7f0d4a0c9700 -1 mds.b *** got signal Terminated *** 2024-03-16T09:21:14.685 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:21:14 smithi159 podman[70659]: 2024-03-16 09:21:14.390030164 +0000 UTC m=+4.108272800 container died 404cd1638c026731e0a48e118967429336a2f3ba033c5698f950644b61a80aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-16T09:21:14.987 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:21:14 smithi159 podman[70659]: 2024-03-16 09:21:14.782921045 +0000 UTC m=+4.501163680 container cleanup 404cd1638c026731e0a48e118967429336a2f3ba033c5698f950644b61a80aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-16T09:21:14.987 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 16 09:21:14 smithi159 bash[70659]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-b 2024-03-16T09:21:15.173 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.b.service' 2024-03-16T09:21:15.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:21:15.212 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-16T09:21:15.212 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-16T09:21:15.212 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.e 2024-03-16T09:21:15.737 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:21:15 smithi159 systemd[1]: Stopping Ceph mds.e for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:21:15.737 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:21:15 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-e[65884]: 2024-03-16T09:21:15.519+0000 7f62028c8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:21:15.737 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:21:15 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-e[65884]: 2024-03-16T09:21:15.519+0000 7f62028c8700 -1 mds.e *** got signal Terminated *** 2024-03-16T09:21:19.237 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:21:18 smithi159 podman[70793]: 2024-03-16 09:21:18.983942877 +0000 UTC m=+3.558600950 container died f5bbe49efcbb5e5650369bd3bd3ed6036a3184dec6c9deb8d1cc1bfcc19f4fcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:21:19.514 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:21:19 smithi159 podman[70793]: 2024-03-16 09:21:19.32762718 +0000 UTC m=+3.902285242 container cleanup f5bbe49efcbb5e5650369bd3bd3ed6036a3184dec6c9deb8d1cc1bfcc19f4fcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-e, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:21:19.514 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:21:19 smithi159 bash[70793]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-e 2024-03-16T09:21:19.752 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.e.service' 2024-03-16T09:21:19.775 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:21:19 smithi159 podman[70807]: 2024-03-16 09:21:19.512039882 +0000 UTC m=+0.524526081 container remove f5bbe49efcbb5e5650369bd3bd3ed6036a3184dec6c9deb8d1cc1bfcc19f4fcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-e, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-16T09:21:19.775 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:21:19 smithi159 systemd[1]: ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.e.service: Deactivated successfully. 2024-03-16T09:21:19.775 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:21:19 smithi159 systemd[1]: Stopped Ceph mds.e for 273be2a4-e374-11ee-95c9-87774f69a715. 2024-03-16T09:21:19.775 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 16 09:21:19 smithi159 systemd[1]: ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.e.service: Consumed 1.452s CPU time. 2024-03-16T09:21:19.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:21:19.787 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-16T09:21:19.787 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-16T09:21:19.787 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.h 2024-03-16T09:21:20.100 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:21:19 smithi159 systemd[1]: Stopping Ceph mds.h for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:21:20.487 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:21:20 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:21:20.097+0000 7f87ea9c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:21:20.487 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:21:20 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h[66197]: 2024-03-16T09:21:20.097+0000 7f87ea9c4700 -1 mds.h *** got signal Terminated *** 2024-03-16T09:21:23.249 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:21:22 smithi159 podman[70913]: 2024-03-16 09:21:22.98731949 +0000 UTC m=+2.992333190 container died d9c2c9c807002a6d4930484051a7e339810f326fa25291b9c62b048bf3820d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-16T09:21:23.249 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:21:23 smithi159 podman[70913]: 2024-03-16 09:21:23.247743731 +0000 UTC m=+3.252757433 container cleanup d9c2c9c807002a6d4930484051a7e339810f326fa25291b9c62b048bf3820d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:21:23.604 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:21:23 smithi159 bash[70913]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h 2024-03-16T09:21:23.604 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 16 09:21:23 smithi159 podman[70928]: 2024-03-16 09:21:23.481896133 +0000 UTC m=+0.491190854 container remove d9c2c9c807002a6d4930484051a7e339810f326fa25291b9c62b048bf3820d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-16T09:21:23.616 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.h.service' 2024-03-16T09:21:23.650 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:21:23.650 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-16T09:21:23.650 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-16T09:21:23.650 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.k 2024-03-16T09:21:23.969 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 16 09:21:23 smithi159 systemd[1]: Stopping Ceph mds.k for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:21:24.237 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 16 09:21:23 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-k[65562]: 2024-03-16T09:21:23.966+0000 7ff5135e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:21:24.237 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 16 09:21:23 smithi159 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-k[65562]: 2024-03-16T09:21:23.966+0000 7ff5135e3700 -1 mds.k *** got signal Terminated *** 2024-03-16T09:21:26.878 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 16 09:21:26 smithi159 podman[71033]: 2024-03-16 09:21:26.592226026 +0000 UTC m=+2.729445543 container died c4afbd131895d0a28145873a7f4837ab559b7400de202e222f80c28912fdae68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-k, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-03-16T09:21:27.212 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 16 09:21:26 smithi159 podman[71033]: 2024-03-16 09:21:26.96796758 +0000 UTC m=+3.105187098 container cleanup c4afbd131895d0a28145873a7f4837ab559b7400de202e222f80c28912fdae68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-k, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-16T09:21:27.212 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 16 09:21:26 smithi159 bash[71033]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-k 2024-03-16T09:21:27.334 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.k.service' 2024-03-16T09:21:27.367 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:21:27.368 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-16T09:21:27.368 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-16T09:21:27.368 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.c 2024-03-16T09:21:27.652 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:21:27 smithi194 systemd[1]: Stopping Ceph mds.c for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:21:27.914 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:21:27 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-c[67892]: 2024-03-16T09:21:27.684+0000 7fadad830700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:21:27.914 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:21:27 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-c[67892]: 2024-03-16T09:21:27.684+0000 7fadad830700 -1 mds.c *** got signal Terminated *** 2024-03-16T09:21:31.270 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:21:30 smithi194 podman[72539]: 2024-03-16 09:21:30.989999788 +0000 UTC m=+3.433139965 container died 42b46e581e529129b8f5c72293e3dc1f54012327be9e5cdc97d89793a9f71aa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS) 2024-03-16T09:21:31.620 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:21:31 smithi194 podman[72539]: 2024-03-16 09:21:31.367538559 +0000 UTC m=+3.810678741 container cleanup 42b46e581e529129b8f5c72293e3dc1f54012327be9e5cdc97d89793a9f71aa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-16T09:21:31.620 INFO:journalctl@ceph.mds.c.smithi194.stdout:Mar 16 09:21:31 smithi194 bash[72539]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-c 2024-03-16T09:21:31.792 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.c.service' 2024-03-16T09:21:31.831 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:21:31.831 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-16T09:21:31.831 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-16T09:21:31.831 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.f 2024-03-16T09:21:32.152 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 09:21:31 smithi194 systemd[1]: Stopping Ceph mds.f for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:21:32.654 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 09:21:32 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-f[67573]: 2024-03-16T09:21:32.154+0000 7f045fd49700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:21:32.654 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 09:21:32 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-f[67573]: 2024-03-16T09:21:32.154+0000 7f045fd49700 -1 mds.f *** got signal Terminated *** 2024-03-16T09:21:35.266 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 09:21:34 smithi194 podman[72674]: 2024-03-16 09:21:34.983446502 +0000 UTC m=+2.937827034 container died 0b0b811fcf98628c3a07f487dc958ba11cf8e22da8313668a498b72ef9ec24a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-f, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-16T09:21:35.616 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 09:21:35 smithi194 podman[72674]: 2024-03-16 09:21:35.346903718 +0000 UTC m=+3.301284243 container cleanup 0b0b811fcf98628c3a07f487dc958ba11cf8e22da8313668a498b72ef9ec24a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-f, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-16T09:21:35.617 INFO:journalctl@ceph.mds.f.smithi194.stdout:Mar 16 09:21:35 smithi194 bash[72674]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-f 2024-03-16T09:21:35.784 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.f.service' 2024-03-16T09:21:35.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:21:35.820 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-16T09:21:35.820 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-16T09:21:35.820 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.i 2024-03-16T09:21:36.136 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 09:21:35 smithi194 systemd[1]: Stopping Ceph mds.i for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:21:36.402 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 09:21:36 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-i[68207]: 2024-03-16T09:21:36.133+0000 7f5b0c577700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:21:36.402 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 09:21:36 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-i[68207]: 2024-03-16T09:21:36.133+0000 7f5b0c577700 -1 mds.i *** got signal Terminated *** 2024-03-16T09:21:39.220 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 09:21:38 smithi194 podman[72796]: 2024-03-16 09:21:38.981329684 +0000 UTC m=+2.948429922 container died bfe012df85d8197e7e0b8ac16aa75f86ee3c57073d9c6453b2af22d3963f1806 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-i, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-16T09:21:39.512 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 09:21:39 smithi194 podman[72796]: 2024-03-16 09:21:39.317620486 +0000 UTC m=+3.284720716 container cleanup bfe012df85d8197e7e0b8ac16aa75f86ee3c57073d9c6453b2af22d3963f1806 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-i, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-16T09:21:39.512 INFO:journalctl@ceph.mds.i.smithi194.stdout:Mar 16 09:21:39 smithi194 bash[72796]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-i 2024-03-16T09:21:39.732 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.i.service' 2024-03-16T09:21:39.766 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:21:39.767 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-16T09:21:39.767 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-16T09:21:39.767 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.l 2024-03-16T09:21:40.073 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 09:21:39 smithi194 systemd[1]: Stopping Ceph mds.l for 273be2a4-e374-11ee-95c9-87774f69a715... 2024-03-16T09:21:40.402 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 09:21:40 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-l[67259]: 2024-03-16T09:21:40.071+0000 7f2eddaf3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-16T09:21:40.402 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 09:21:40 smithi194 ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-l[67259]: 2024-03-16T09:21:40.071+0000 7f2eddaf3700 -1 mds.l *** got signal Terminated *** 2024-03-16T09:21:43.902 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 09:21:43 smithi194 podman[72917]: 2024-03-16 09:21:43.611942031 +0000 UTC m=+3.632903662 container died acce808a8f08258f56196c61fdcd92e0ff7a159e36a626b39445dfe8631cf5b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-16T09:21:44.232 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 09:21:44 smithi194 podman[72917]: 2024-03-16 09:21:44.004518132 +0000 UTC m=+4.025479756 container cleanup acce808a8f08258f56196c61fdcd92e0ff7a159e36a626b39445dfe8631cf5b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-l, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-16T09:21:44.232 INFO:journalctl@ceph.mds.l.smithi194.stdout:Mar 16 09:21:44 smithi194 bash[72917]: ceph-273be2a4-e374-11ee-95c9-87774f69a715-mds-l 2024-03-16T09:21:44.358 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-273be2a4-e374-11ee-95c9-87774f69a715@mds.l.service' 2024-03-16T09:21:44.392 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-16T09:21:44.393 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-16T09:21:44.393 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 273be2a4-e374-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-16T09:21:44.584 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:21:47.005 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 273be2a4-e374-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-16T09:21:47.198 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:21:49.677 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 273be2a4-e374-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-16T09:21:49.866 INFO:teuthology.orchestra.run.smithi194.stdout:Deleting cluster with fsid: 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:21:52.274 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T09:21:52.308 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T09:21:52.343 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-16T09:21:52.371 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-16T09:21:52.374 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602314/remote/smithi112/crash 2024-03-16T09:21:52.374 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/crash -- . 2024-03-16T09:21:52.404 INFO:teuthology.orchestra.run.smithi112.stderr:tar: /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-16T09:21:52.404 INFO:teuthology.orchestra.run.smithi112.stderr:tar: Error is not recoverable: exiting now 2024-03-16T09:21:52.406 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602314/remote/smithi159/crash 2024-03-16T09:21:52.407 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/crash -- . 2024-03-16T09:21:52.473 INFO:teuthology.orchestra.run.smithi159.stderr:tar: /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-16T09:21:52.473 INFO:teuthology.orchestra.run.smithi159.stderr:tar: Error is not recoverable: exiting now 2024-03-16T09:21:52.475 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602314/remote/smithi194/crash 2024-03-16T09:21:52.476 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/crash -- . 2024-03-16T09:21:52.516 INFO:teuthology.orchestra.run.smithi194.stderr:tar: /var/lib/ceph/273be2a4-e374-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-16T09:21:52.516 INFO:teuthology.orchestra.run.smithi194.stderr:tar: Error is not recoverable: exiting now 2024-03-16T09:21:52.517 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-16T09:21:52.517 DEBUG:teuthology.orchestra.run.smithi112:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-16T09:21:52.551 INFO:teuthology.orchestra.run.smithi112.stdout:2024-03-16T09:07:29.261095+0000 mon.a (mon.0) 211 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-16T09:21:52.552 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-16T09:21:52.552 INFO:tasks.cephadm:Compressing logs... 2024-03-16T09:21:52.552 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-16T09:21:52.594 DEBUG:teuthology.orchestra.run.smithi159:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-16T09:21:52.597 DEBUG:teuthology.orchestra.run.smithi194:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-16T09:21:52.620 INFO:teuthology.orchestra.run.smithi112.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log: No such file or directory 2024-03-16T09:21:52.621 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-03-16T09:21:52.621 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.6.log 2024-03-16T09:21:52.621 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mon.a.log 2024-03-16T09:21:52.621 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-16T09:21:52.621 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.g.log 2024-03-16T09:21:52.624 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.j.log 2024-03-16T09:21:52.624 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.3.log 2024-03-16T09:21:52.624 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mgr.x.log 2024-03-16T09:21:52.624 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.0.log 2024-03-16T09:21:52.624 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mgr.x.log: 90.2% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-16T09:21:52.638 INFO:teuthology.orchestra.run.smithi112.stderr: 94.6% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.g.log.gz 2024-03-16T09:21:52.638 INFO:teuthology.orchestra.run.smithi112.stderr: 94.6% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.j.log.gz 2024-03-16T09:21:52.638 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-16T09:21:52.638 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.log 2024-03-16T09:21:52.638 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-16T09:21:52.638 INFO:teuthology.orchestra.run.smithi112.stderr: 83.0% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-16T09:21:52.638 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.9.log 2024-03-16T09:21:52.638 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-volume.log: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-16T09:21:52.638 INFO:teuthology.orchestra.run.smithi112.stderr: 86.4% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-16T09:21:52.638 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.d.log 2024-03-16T09:21:52.638 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.a.log 2024-03-16T09:21:52.638 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.d.log.gz 2024-03-16T09:21:52.639 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.a.log: 94.6% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.a.log.gz 2024-03-16T09:21:52.642 INFO:teuthology.orchestra.run.smithi194.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-16T09:21:52.642 INFO:teuthology.orchestra.run.smithi194.stderr:: No such file or directory 2024-03-16T09:21:52.642 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.l.log 2024-03-16T09:21:52.642 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-16T09:21:52.642 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.i.log 2024-03-16T09:21:52.642 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.l.log: /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.11.log 2024-03-16T09:21:52.642 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mgr.z.log 2024-03-16T09:21:52.642 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.i.log: /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.8.log 2024-03-16T09:21:52.642 INFO:teuthology.orchestra.run.smithi194.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-16T09:21:52.642 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.5.log 2024-03-16T09:21:52.642 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.f.log 2024-03-16T09:21:52.643 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-16T09:21:52.643 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.f.log: 90.4% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-16T09:21:52.643 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.log 2024-03-16T09:21:52.643 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.cephadm.log: 94.8% 82.5% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-16T09:21:52.643 INFO:teuthology.orchestra.run.smithi194.stderr: -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.l.log.gz 2024-03-16T09:21:52.643 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.2.log 2024-03-16T09:21:52.643 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.c.log 2024-03-16T09:21:52.643 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.2.log: 92.2% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mgr.z.log.gz 2024-03-16T09:21:52.643 INFO:teuthology.orchestra.run.smithi194.stderr: 86.3% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-16T09:21:52.643 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-16T09:21:52.643 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mon.c.log 2024-03-16T09:21:52.644 INFO:teuthology.orchestra.run.smithi112.stderr: 91.0% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-16T09:21:52.647 INFO:teuthology.orchestra.run.smithi159.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-16T09:21:52.647 INFO:teuthology.orchestra.run.smithi159.stderr:: No such file or directory 2024-03-16T09:21:52.647 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.10.log 2024-03-16T09:21:52.647 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-16T09:21:52.647 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.b.log 2024-03-16T09:21:52.647 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.10.log: /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.4.log 2024-03-16T09:21:52.647 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.1.log 2024-03-16T09:21:52.647 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.b.log: /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.k.log 2024-03-16T09:21:52.647 INFO:teuthology.orchestra.run.smithi159.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-16T09:21:52.647 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.h.log 2024-03-16T09:21:52.648 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.7.log 2024-03-16T09:21:52.648 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.e.log 2024-03-16T09:21:52.648 INFO:teuthology.orchestra.run.smithi159.stderr: 90.4% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-16T09:21:52.648 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mgr.y.log 2024-03-16T09:21:52.648 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.e.log: 94.7% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.k.log.gz 2024-03-16T09:21:52.648 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-16T09:21:52.648 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mgr.y.log: 92.1% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mgr.y.log.gz 2024-03-16T09:21:52.648 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.log 2024-03-16T09:21:52.648 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.cephadm.log: 82.5% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-16T09:21:52.648 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-16T09:21:52.648 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.log: 86.3% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-16T09:21:52.648 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mon.b.log 2024-03-16T09:21:52.651 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-volume.log: /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-16T09:21:52.656 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-volume.log: /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-16T09:21:52.746 INFO:teuthology.orchestra.run.smithi159.stderr: 93.6% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.e.log.gz 2024-03-16T09:21:52.818 INFO:teuthology.orchestra.run.smithi112.stderr: 89.6% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mgr.x.log.gz 2024-03-16T09:21:52.877 INFO:teuthology.orchestra.run.smithi159.stderr: 91.9% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mon.b.log.gz 2024-03-16T09:21:52.944 INFO:teuthology.orchestra.run.smithi194.stderr: 91.9% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mon.c.log.gz 2024-03-16T09:21:53.158 INFO:teuthology.orchestra.run.smithi112.stderr: 90.6% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mon.a.log.gz 2024-03-16T09:21:53.401 INFO:teuthology.orchestra.run.smithi159.stderr: 93.6% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.h.log.gz 2024-03-16T09:21:53.762 INFO:teuthology.orchestra.run.smithi194.stderr: 95.4% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.f.log.gz 2024-03-16T09:21:54.586 INFO:teuthology.orchestra.run.smithi112.stderr: 92.2% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.9.log.gz 2024-03-16T09:21:55.042 INFO:teuthology.orchestra.run.smithi112.stderr: 91.7% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.6.log.gz 2024-03-16T09:21:55.091 INFO:teuthology.orchestra.run.smithi194.stderr: 95.1% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.c.log.gz 2024-03-16T09:21:55.224 INFO:teuthology.orchestra.run.smithi159.stderr: 92.4% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.4.log.gz 2024-03-16T09:21:55.228 INFO:teuthology.orchestra.run.smithi112.stderr: 91.8% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.0.log.gz 2024-03-16T09:21:55.463 INFO:teuthology.orchestra.run.smithi194.stderr: 91.7% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.5.log.gz 2024-03-16T09:21:55.759 INFO:teuthology.orchestra.run.smithi112.stderr: 91.8% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.3.log.gz 2024-03-16T09:21:55.762 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-03-16T09:21:55.762 INFO:teuthology.orchestra.run.smithi112.stderr:real 0m3.155s 2024-03-16T09:21:55.762 INFO:teuthology.orchestra.run.smithi112.stderr:user 0m10.520s 2024-03-16T09:21:55.762 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m0.388s 2024-03-16T09:21:55.805 INFO:teuthology.orchestra.run.smithi194.stderr: 91.8% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.11.log.gz 2024-03-16T09:21:55.854 INFO:teuthology.orchestra.run.smithi159.stderr: 92.0% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.10.log.gz 2024-03-16T09:21:55.872 INFO:teuthology.orchestra.run.smithi194.stderr: 91.9% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.8.log.gz 2024-03-16T09:21:56.025 INFO:teuthology.orchestra.run.smithi159.stderr: 92.2% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.7.log.gz 2024-03-16T09:21:56.449 INFO:teuthology.orchestra.run.smithi194.stderr: 92.3% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.2.log.gz 2024-03-16T09:21:56.489 INFO:teuthology.orchestra.run.smithi159.stderr: 92.3% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-osd.1.log.gz 2024-03-16T09:22:02.812 INFO:teuthology.orchestra.run.smithi194.stderr: 93.4% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.i.log.gz 2024-03-16T09:22:02.814 INFO:teuthology.orchestra.run.smithi194.stderr: 2024-03-16T09:22:02.814 INFO:teuthology.orchestra.run.smithi194.stderr:real 0m10.203s 2024-03-16T09:22:02.814 INFO:teuthology.orchestra.run.smithi194.stderr:user 0m26.390s 2024-03-16T09:22:02.814 INFO:teuthology.orchestra.run.smithi194.stderr:sys 0m0.785s 2024-03-16T09:22:19.806 INFO:teuthology.orchestra.run.smithi159.stderr: 93.1% -- replaced with /var/log/ceph/273be2a4-e374-11ee-95c9-87774f69a715/ceph-mds.b.log.gz 2024-03-16T09:22:19.808 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-03-16T09:22:19.808 INFO:teuthology.orchestra.run.smithi159.stderr:real 0m27.196s 2024-03-16T09:22:19.808 INFO:teuthology.orchestra.run.smithi159.stderr:user 0m40.085s 2024-03-16T09:22:19.808 INFO:teuthology.orchestra.run.smithi159.stderr:sys 0m1.271s 2024-03-16T09:22:19.808 INFO:tasks.cephadm:Archiving logs... 2024-03-16T09:22:19.809 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602314/remote/smithi112/log 2024-03-16T09:22:19.810 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-16T09:22:21.520 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602314/remote/smithi159/log 2024-03-16T09:22:21.521 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-16T09:22:27.405 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602314/remote/smithi194/log 2024-03-16T09:22:27.406 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-16T09:22:30.738 INFO:tasks.cephadm:Removing cluster... 2024-03-16T09:22:30.739 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 273be2a4-e374-11ee-95c9-87774f69a715 --force 2024-03-16T09:22:30.944 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:22:31.206 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 273be2a4-e374-11ee-95c9-87774f69a715 --force 2024-03-16T09:22:31.400 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:22:31.691 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 273be2a4-e374-11ee-95c9-87774f69a715 --force 2024-03-16T09:22:31.882 INFO:teuthology.orchestra.run.smithi194.stdout:Deleting cluster with fsid: 273be2a4-e374-11ee-95c9-87774f69a715 2024-03-16T09:22:32.210 INFO:tasks.cephadm:Removing cephadm ... 2024-03-16T09:22:32.210 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-16T09:22:32.228 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-16T09:22:32.246 DEBUG:teuthology.orchestra.run.smithi194:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-16T09:22:32.261 INFO:tasks.cephadm:Teardown complete 2024-03-16T09:22:32.262 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-16T09:22:32.274 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-16T09:22:32.274 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-16T09:22:32.277 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-16T09:22:32.290 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-16T09:22:32.398 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-16T09:22:32.408 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-16T09:22:32.409 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T09:22:32.411 DEBUG:teuthology.orchestra.run.smithi159:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T09:22:32.413 DEBUG:teuthology.orchestra.run.smithi194:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T09:22:32.424 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-03-16T09:22:32.425 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-03-16T09:22:32.427 INFO:teuthology.orchestra.run.smithi194.stderr:bash: line 1: ntpq: command not found 2024-03-16T09:22:32.428 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T09:22:32.428 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-03-16T09:22:32.428 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 13 +3320us[+3289us] +/- 79ms 2024-03-16T09:22:32.429 INFO:teuthology.orchestra.run.smithi112.stdout:^* hv02.front.sepia.ceph.com 3 7 377 12 -1926us[-1957us] +/- 56ms 2024-03-16T09:22:32.429 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 -457us[ -488us] +/- 69ms 2024-03-16T09:22:32.429 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:22:32.430 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T09:22:32.430 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-03-16T09:22:32.430 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 15 +3350us[+3347us] +/- 79ms 2024-03-16T09:22:32.430 INFO:teuthology.orchestra.run.smithi159.stdout:^* hv02.front.sepia.ceph.com 3 6 377 12 -1909us[-1913us] +/- 56ms 2024-03-16T09:22:32.430 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 -429us[ -429us] +/- 69ms 2024-03-16T09:22:32.430 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:22:32.430 INFO:teuthology.orchestra.run.smithi194.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T09:22:32.430 INFO:teuthology.orchestra.run.smithi194.stdout:=============================================================================== 2024-03-16T09:22:32.431 INFO:teuthology.orchestra.run.smithi194.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 13 +3332us[+3326us] +/- 79ms 2024-03-16T09:22:32.431 INFO:teuthology.orchestra.run.smithi194.stdout:^* hv02.front.sepia.ceph.com 3 6 377 11 -1911us[-1917us] +/- 56ms 2024-03-16T09:22:32.431 INFO:teuthology.orchestra.run.smithi194.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 -442us[ -449us] +/- 69ms 2024-03-16T09:22:32.431 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T09:22:32.432 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-16T09:22:32.444 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-16T09:22:32.444 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-16T09:22:32.475 DEBUG:teuthology.orchestra.run.smithi112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-16T09:22:32.555 DEBUG:teuthology.orchestra.run.smithi159:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-16T09:22:32.626 DEBUG:teuthology.orchestra.run.smithi194:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-16T09:22:32.696 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T09:22:32.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:22:32.728 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T09:22:32.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:22:32.758 DEBUG:teuthology.orchestra.run.smithi194:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-16T09:22:32.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:22:32.788 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-16T09:22:32.798 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-16T09:22:32.847 INFO:teuthology.task.internal:Duration was 1529.159050 seconds 2024-03-16T09:22:32.848 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-16T09:22:32.882 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-16T09:22:32.882 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-16T09:22:32.885 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-16T09:22:32.887 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-16T09:22:32.926 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T09:22:32.929 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T09:22:32.930 INFO:teuthology.orchestra.run.smithi194.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T09:22:33.433 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-16T09:22:33.433 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2024-03-16T09:22:33.434 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-16T09:22:33.454 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi159.front.sepia.ceph.com 2024-03-16T09:22:33.454 DEBUG:teuthology.orchestra.run.smithi159:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-16T09:22:33.485 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi194.front.sepia.ceph.com 2024-03-16T09:22:33.485 DEBUG:teuthology.orchestra.run.smithi194:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-16T09:22:33.509 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-16T09:22:33.509 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-16T09:22:33.511 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-16T09:22:33.528 DEBUG:teuthology.orchestra.run.smithi194:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-16T09:22:33.578 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-16T09:22:33.578 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-16T09:22:33.581 DEBUG:teuthology.orchestra.run.smithi159:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-16T09:22:33.596 DEBUG:teuthology.orchestra.run.smithi194:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-16T09:22:33.946 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-16T09:22:33.958 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-16T09:22:33.958 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-16T09:22:33.985 DEBUG:teuthology.orchestra.run.smithi159:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-16T09:22:34.012 DEBUG:teuthology.orchestra.run.smithi194:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-16T09:22:34.039 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-16T09:22:34.052 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-16T09:22:34.055 DEBUG:teuthology.orchestra.run.smithi159:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-16T09:22:34.057 DEBUG:teuthology.orchestra.run.smithi194:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-16T09:22:34.078 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2024-03-16T09:22:34.080 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = core 2024-03-16T09:22:34.104 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern = core 2024-03-16T09:22:34.130 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-16T09:22:34.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:22:34.148 DEBUG:teuthology.orchestra.run.smithi159:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-16T09:22:34.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:22:34.166 DEBUG:teuthology.orchestra.run.smithi194:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-16T09:22:34.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:22:34.184 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-16T09:22:34.194 INFO:teuthology.task.internal:Transferring archived files... 2024-03-16T09:22:34.195 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602314/remote/smithi112 2024-03-16T09:22:34.195 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-16T09:22:34.238 DEBUG:teuthology.misc:Transferring archived files from smithi159:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602314/remote/smithi159 2024-03-16T09:22:34.238 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-16T09:22:34.275 DEBUG:teuthology.misc:Transferring archived files from smithi194:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602314/remote/smithi194 2024-03-16T09:22:34.275 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-16T09:22:34.311 INFO:teuthology.task.internal:Removing archive directory... 2024-03-16T09:22:34.312 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-16T09:22:34.314 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-16T09:22:34.316 DEBUG:teuthology.orchestra.run.smithi194:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-16T09:22:34.361 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-16T09:22:34.374 INFO:teuthology.task.internal:Not uploading archives. 2024-03-16T09:22:34.374 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-16T09:22:34.416 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-16T09:22:34.416 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-16T09:22:34.419 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-16T09:22:34.421 DEBUG:teuthology.orchestra.run.smithi194:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-16T09:22:34.431 INFO:teuthology.orchestra.run.smithi112.stdout: 265605 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 16 09:22 /home/ubuntu/cephtest 2024-03-16T09:22:34.433 INFO:teuthology.orchestra.run.smithi159.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 16 09:22 /home/ubuntu/cephtest 2024-03-16T09:22:34.435 INFO:teuthology.orchestra.run.smithi194.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 16 09:22 /home/ubuntu/cephtest 2024-03-16T09:22:34.436 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-16T09:22:34.450 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-16T09:22:34.515 INFO:teuthology.nuke:Checking targets against current locks 2024-03-16T09:22:34.541 DEBUG:teuthology.nuke:shortname: smithi112 2024-03-16T09:22:34.542 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T09:22:34.564 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602314', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 08:47:57.911507', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKMcQjK1rvItRmL4/d6pqQrhpzHSWR91fnX6TdnTY7XC2SubTf7JMs3KMhAohQyTwOx/gFFsyX2+r7kJkLGBd0Y='} 2024-03-16T09:22:34.582 DEBUG:teuthology.nuke:shortname: smithi159 2024-03-16T09:22:34.582 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T09:22:34.607 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602314', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 08:47:57.908446', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHk06dUmFhNWeMG+1AKfUShHlDg3tc4cOLl+wvCchzg4pNCgiMKmcq/SqbIonqu15hbqyZd+I95iTnmJuuafkfA='} 2024-03-16T09:22:34.608 DEBUG:teuthology.nuke:shortname: smithi194 2024-03-16T09:22:34.608 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T09:22:34.626 INFO:teuthology.orchestra.console.smithi112:Power off 2024-03-16T09:22:34.626 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T09:22:34.639 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi194.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602314', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 08:47:57.910261', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIUfg8irZqXC6jjvMXnUMcmg4amOQLA+dO82iErVMDRIdet6YICjTwsDO97TYxaP7VihKgqFGgElf/IZHWjhPhU='} 2024-03-16T09:22:34.651 DEBUG:teuthology.orchestra.console.smithi112:power off output: Chassis Power Control: Down/Off 2024-03-16T09:22:34.652 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T09:22:34.668 INFO:teuthology.orchestra.console.smithi159:Power off 2024-03-16T09:22:34.668 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T09:22:34.683 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-03-16T09:22:34.693 DEBUG:teuthology.orchestra.console.smithi159:power off output: Chassis Power Control: Down/Off 2024-03-16T09:22:34.693 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T09:22:34.712 INFO:teuthology.orchestra.console.smithi194:Power off 2024-03-16T09:22:34.712 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T09:22:34.722 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2024-03-16T09:22:34.734 DEBUG:teuthology.orchestra.console.smithi194:power off output: Chassis Power Control: Down/Off 2024-03-16T09:22:34.734 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T09:22:34.758 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is on 2024-03-16T09:22:38.684 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T09:22:38.722 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T09:22:38.759 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T09:22:38.794 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-03-16T09:22:38.832 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2024-03-16T09:22:38.868 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is on 2024-03-16T09:22:42.799 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T09:22:42.833 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T09:22:42.869 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T09:22:42.910 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-03-16T09:22:42.942 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off 2024-03-16T09:22:42.978 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is on 2024-03-16T09:22:43.043 INFO:teuthology.orchestra.console.smithi159:Power off completed 2024-03-16T09:22:46.915 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T09:22:46.979 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T09:22:47.026 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-03-16T09:22:47.091 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is off 2024-03-16T09:22:47.127 INFO:teuthology.orchestra.console.smithi112:Power off completed 2024-03-16T09:22:47.192 INFO:teuthology.orchestra.console.smithi194:Power off completed 2024-03-16T09:22:47.296 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/norstats}} duration: 1529.159049987793 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2024-03-16T09:22:47.297 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T09:22:47.382 INFO:teuthology.run:FAIL